Antony Henao Roqueme

🌟Hello Everyone!🌟 I'm happy to share my new Portfolio Website😀. Contains some EDA 🔍 projects, Kaggle competition 💥 notebooks, and some AI 🤖 projects. Feel free to explore my repositories and Kaggle contributions. 🔗 GitHub: https://lnkd.in/eqKtnncQ 🔗 Kaggle: https://lnkd.in/exZmyU_Q #datascienceportfolio #PortfolioLaunch #DataScience #GitHub #Kaggle #MachineLearning #DeepLearning #ArtificialIntelligence #Math #Portfolio #ARCgames #ClimSim #NaturalLanguageProcessing

10

2 comentarios

I finished the Skill truck Machine Learning Fundamentals with Python! Included the following courses and proyects: Courses: ·      SUPERVISED LEARNING WITH SCIKIT-LEARN ·      UNSUPERVISED LEARNING IN PYTHON ·      LINEAR CLASSIFIERS IN PYTHON ·      INTRODUCTION TO DEEP LEARNING WITH PYTORCH Projects: ·       Predictive Modeling for Agriculture ·       Clustering Antartic Penguin Species

3

Check out the retrieval techniques of RAG. Really useful !!!

5

For understanding the #RAG.

2

🔹 What do you do if your model is not working well? 🔹 What changes can you make to improve model performance?   If 𝐦𝐨𝐝𝐞𝐥 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 is disappointing, there are three main levers that we can pull to try improving the performance.   The first and most powerful lever is 𝐜𝐡𝐚𝐧𝐠𝐢𝐧𝐠 𝐭𝐡𝐞 𝐝𝐚𝐭𝐚 that the model is using. We can add more features to the model or transform the features that we’ve already included. In my experience, this is the most powerful of the levers.   Another lever we can pull is 𝐜𝐡𝐚𝐧𝐠𝐢𝐧𝐠 𝐭𝐡𝐞 𝐭𝐲𝐩𝐞 𝐨𝐟 𝐦𝐨𝐝𝐞𝐥or the type of feature selection. If we see that a regression model isn’t working well, we can try a decision tree, for example. We can also try using a penalized regression model that performs feature selection automatically during the modeling process.   The other lever is tuning the 𝐡𝐲𝐩𝐞𝐫𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫𝐬. A hyperparameter is like a setting knob on a model. It adjusts the model's rules so changes in the hyperparameters produce models with very different results.   Any combination of three levers may be used to improve model performance. Depending on what data is accessible, it may not be feasible to 𝐚𝐝𝐝 𝐦𝐨𝐫𝐞 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬, so the data scientist must rely on 𝐡𝐲𝐩𝐞𝐫𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫 𝐭𝐮𝐧𝐢𝐧𝐠 and 𝐦𝐨𝐝𝐞𝐥 𝐬𝐞𝐥𝐞𝐜𝐭𝐢𝐨𝐧 to improve the quality of predictions.   Keys questions to ask the data science team about model tuning include: 🔹 Can you describe the hyperparameter tuning process? 🔹 How did you determine the optimal hyperparameter values for the chosen models?

8

This week I found an interesting article that I would like to share with you, This article compares three data preprocessing platforms: - Pandas - Polars - PySpark Pandas: is the most famous of the three and is expected in the repertoire of every young data scientist, so once you face an issue you know it is probably resolved. Polars: is written in Rust, but luckily you do not need to know Rust to use it, it is designed to be very fast, Polars provides two APIs one is eager and the other is lazy. Pyspark: is a Python API to Apache Spark, Spark is designed to process big data in a distributed environment. The writer did timing benchmarks of different sizes on the three platforms (Polars in both eager and lazy mode) and found the following, 1- Polars is way faster than the other alternatives. 2- Polars Lazy mode is consistently better than the Eager mode in execution time. 3- PySpark is a huge waste in the low-to-mid data sizes, as it has a large setup time. 4- Moving up in the data size Polars and Pandas execution time increases linearly, while PySpark execution time almost remains the same. 5- It is known that PySpark starts to shine in the large data domain, i.e. in the terabytes, but the author did not run such benchmarks yet at the time he wrote the article Conclusion: As a data scientist, Pandas will do fine with your regular workload, but if you are looking to enhance the performance of your data processing you would probably want to give Polars a go, but expect to find a lack of community support when you face an issue as the tool is relatively speaking new, If you are going to deal with large data PySpark should be your choice, Finally to get more details and the exact benchmark results I suggest you read the original article for yourself (link below). https://lnkd.in/dGAdW4U6

3

2 comentarios

Spatial Data Infrastructure (SDI) People: Fun discovery. There is a knowledge-graph framework (Ontop) that can publish your existing external relational databases to RDF, SPARQL, GEOSPARQL, etc. No need to convert or import. It means legacy relational systems can be gracefully included in semantic web technology that allow for MASSIVE query-spaces that span multiple semantic webs!

1

2 comentarios

🎉 Just completed my Python course on Kaggle! 🚀 Curious about Python's impact on Machine Learning? Let's discuss! Python's pivotal role in Machine Learning cannot be overstated. As a versatile and powerful language, it is the backbone of numerous ML projects. Here are a few reasons why Python is indispensable for Machine Learning: · Ease of learning: Python's simple syntax makes it accessible to beginners and experts alike. · Rich libraries: Extensive libraries like TensorFlow, PyTorch, and scikit-learn accelerate development. · Community support: A robust community means ample resources and support. · Integration: Seamless integration with other technologies and tools. Do you agree with me? 👍🏼 #Python #MachineLearning #AI #Programming #DataScience #Tech #Coding #DeepLearning #ArtificialIntelligence #LinkedInLearning

1

Data wrangling with python pandas. - Reading txt files into python pandas data frame. - Cleaning dataframe. - Basics of exploratory data analysis. - Concatenating dataframes. - Exporting dataframe to CSV https://lnkd.in/eB9J5vTf

6

Retrieval-Augmented Generation (RAG) is a powerful technique, combining the strengths of retrieval-based systems and generative models to create more accurate and contextually relevant responses. RAG integrates two primary components: -Retrieval Module: This component searches a vast corpus of documents to find the most relevant pieces of information based on a given query. Instead of generating responses from scratch, it leverages existing knowledge to provide accurate and detailed information. -Generation Module: After retrieving relevant documents, the generative model, typically a transformer-based model like GPT, processes this information to generate coherent and contextually appropriate responses. Why RAG is Important Improved Accuracy: By retrieving specific information from a large dataset, RAG ensures that the generated responses are grounded in factual data. This reduces the risk of generating incorrect or nonsensical information. Contextual Relevance: Embedding techniques allow the retrieval module to understand the semantics and context of both the query and documents. This results in more relevant and meaningful answers. Scalability: RAG systems can efficiently handle vast amounts of data. The retrieval process, driven by advanced search algorithms and embeddings, ensures quick and accurate access to necessary information. Flexibility: RAG can adapt to various domains and applications, from customer support and chatbots to complex question-answering systems, making it a versatile solution for many AI-driven tasks. #machinelearning #artificialintelligence #largelanguagemodel #llm

3

Llama 3.3 is out! 70B performs as well as 3.2 495B! > Llama 3.3 70B model offering similar performance to the Llama 3.1 405B model, allowing developers to achieve greater quality and performance on text-based applications at a fraction of the cost.

3

Goldfish: A Breakthrough Framework for Long-Video Understanding Key Innovations: - Handling Long Videos: Goldfish addresses challenges in processing lengthy videos, such as noise, redundancy, and computational limits. - Efficient Retrieval: Introduces a mechanism to identify the most relevant clips for a query, enhancing efficiency. - MiniGPT4-Video: A new model that generates detailed, descriptive summaries of video clips, facilitating effective retrieval. This can probably be used to fetch which parts of the video contains a given query(??) - TVQA-long Benchmark: A benchmark specifically designed to evaluate long-video understanding. How It Works: - Video Segmentation: Long videos are divided into manageable clips. - Clip Encoding: MiniGPT4-Video creates descriptive summaries for each clip, leveraging visual and textual data (if available). - Retrieval: Relevant clips are identified based on the query and generated descriptions. - Answer Generation: A language model processes the retrieved clips to deliver the final response. Significance: Goldfish marks a pivotal advancement in video understanding, offering efficient and accurate analysis of long-form video content. Its applications span video summarization, search, and personalized recommendations. https://lnkd.in/giJibDez

15

1 comentario

"LLMs are but RecSys in Disguise" 🥸 - Chang She, Pandas, LanceDB ⚡This blog post explores Retrieval-Augmented Generation (RAG), a method that combines information retrieval with language generation for Recommendation Systems. It examines RAG's potential promising approach to improve recommendation quality and discusses its advantages and limitations. 🗝️ Key Points: 🔸 LLMs for Recommendations: Large language models (LLMs) offer a new approach to recommendations by understanding and generating human-like text. However, directly prompting LLMs for recommendations can be ineffective due to limited context memory, hallucinations, bias, and slow inference speed. 🔸 RAG to the Rescue: RAG addresses these limitations by retrieving relevant information before generating recommendations. This allows LLMs to leverage a broader knowledge base and produce more accurate and personalized suggestions. 🔹 Building a RAG System: Here's a breakdown of building a RAG system: 🔸Data Ingestion and Retrieval: Store candidate items and search for the most relevant ones using vector search techniques. 🔹 Encoding User Queries: Encode user queries into semantic vectors to find similar items in the database. 🔸 LLM for Reranking: Use an LLM like GPT-3.5 to refine the retrieved items and generate the final recommendations. 🔸 RAG's Benefits: 🔹 Improved Performance: Studies show that RAG can enhance the performance of various recommendation models. 🔸 Long-Term Memory: RAG enables LLMs to effectively utilize a vast amount of data, overcoming context length limitations. 🔹 Multimodal Data Support: RAG can potentially handle various data formats like text, images, and videos. 🔸 Dynamic Updates: The system can be continuously updated with new user interactions and data. 🔹 Challenges and Considerations: 🔸 Training and Maintenance: Careful training and fine-tuning are crucial for optimal RAG performance. Frequent retraining might be necessary for large datasets. 🔹 Long Context Models: While beneficial, using long context models for re-ranking can lead to slow output times, requiring careful engineering to minimize this trade-off. 🔸Hallucination: LLMs can still generate inaccurate recommendations. Techniques like temperature control and fuzzy string matching can help mitigate this issue. 🔹 Data Structure: The effectiveness of RAG relies heavily on the quality of your data embeddings. Poorly structured data can hinder RAG's ability to find the best matches. 🔸 Multimodal Data Research: The effectiveness of RAG with multimodal data requires further investigation. 👉Conclusion: RAG presents a promising approach for enhancing LLM-based RecSys. While challenges exist, RAG's ability to ✅improve recommendation accuracy, ✅ leverage long-term memory, and ✅potentially handle multimodal data makes it a valuable technique to explore. #RecSys #LLMs #GenAI https://lnkd.in/eCaiWAKD

3

Unraveling the Dynamics of Time Series Datasets: Best Analyzing Trends and Making Predictions Introduction: In the vast landscape o... https://lnkd.in/gCHT8sX9 #Datasets #OnlineLearningandAdaptiveForecasting Introduction: In the vast landscape of data analytics and machine learning, time series datasets hold a unique significance, representing a sequential collection of data points ordered by time. From financial markets to climate patterns and beyond, time series analysis enables us to unravel temporal patterns, understand trends, and make informed predictions. This comprehensive exploration delves ... Read more

4

A guide to choosing the correct classification model 🔷

3

RAG With Autoscaling: Better Performance With Lower Costs For pgvector Querying large datasets is slow at scale because scanning entire databases could be more efficient. Take, for example, running a vector similarity search operation on a 1 million-row dataset on an 8-CPU Neon instance. The query takes approximately 3 seconds to complete. Vector similarity search is essential for Retrieval-Augmented Generation (RAG) apps and allows large language models (LLM) to generate a better-quality output. However, a 3-second query execution time can negatively impact user experience. Article explores the challenges of scaling vector similarity searches and how to optimize for them using pgvector and Neon’s autoscaling capabilities. https://lnkd.in/dmU9TApM #LLM #RAG #Performance #autoscaling

3

Excellent article from Iván Palomares Carrascosa on Statology! For professionals without a profound statistics background, choosing the right statistical test to perform on their data might sometimes not be a straightforward task. This article provides a decision tree-based guide aimed at helping them navigate the problem of choosing the right test depending on the data and problem they are facing, and the hypothesis to be tested.

2