Justin Chavez

Get started with the most powerful LLM library: AdalFlow--The PyTorch library to build, eval, and auto-optimize any LLM tasks. Though only public for two months, AdalFlow has been widely adopted by tech companies in Silicon Valley, including LLM engineers from Meta and various top tech startups. 1️⃣ Create your first chatbot, RAG, and Agent—all in one Colab notebook https://lnkd.in/g23YDaHt 2️⃣ Deep dive on Generator https://lnkd.in/dCRhXg2U 3️⃣ LLM evaluation playbook https://lnkd.in/gTvD93mP 4️⃣ Get started with end-to-end Auto-optimization -- Question Answering: https://lnkd.in/gtECHuGi -- Classification: https://lnkd.in/g4py2e6w 👉 Github repo: https://lnkd.in/g-KN9-dV Happy building! 😊 #adalflow #artificialintelligence #machinelearning #llms

137

2 Comments

Microsoft Research introduced OmniParser – a breakthrough in multimodal large model for GUI agents, now available on Hugging Face! OmniParser is a compact screen parsing module that leverages the power of large vision-language models like GPT-4V to transform UI screenshots into structured elements. This enables AI agents to interact with user interfaces (UI) effectively using purely visual inputs. With OmniParser, researchers are able to tackle some of the toughest challenges in GUI-based agent technology: Reliable identification of interactive icons within the UI. Deep understanding of UI semantics to accurately map actions to specific regions on the screen. Powered by models like GPT-4V, OmniParser takes intelligent agents to the next level, enabling them to perform precise actions across diverse operating systems and applications. In fact, it recently achieved top performance in the WindowsAgentArena benchmark! 🎉 OmniParser is now available open-source on GitHub!  https://lnkd.in/gZge9yta It's an interesting read for vision language model researchers and enthusiasts. #MicrosoftResearch #AI #MachineLearning #VisionLanguageModels #OmniParser #GPT4V #HuggingFace #ArtificialIntelligence #UserInterface #ScreenParsing

11

1 Comment

Creating Retrieval Augmented Generation (RAG) with Langchain, Gradio, and ChromaDB on Hugging Face 🤖💻 I'm excited to share my latest YouTube video, in which I guide you through building a Retrieval Augmented Generation (RAG) model using Langchain, Anysacle, Gradio, and ChromaDB and then deploying it on Hugging Face. YouTube Video URL: https://lnkd.in/gyHi8qpg RAG is a powerful technique that combines the capabilities of pre-trained large language models (LLMs) with external data sources to generate more accurate and informative text. By accessing relevant information from knowledge bases or databases, RAG models can produce text that is not only fluent but also grounded in real-world knowledge. In this video, I cover the following key steps: 1. Indexing data sources into a vector database using ChromaDB and Langchain abstractions. We'll explore how to choose the right embedding model and chunking strategies to optimize retrieval. 🗂️ 2. Setting up a Gradio interface to accept user queries and dynamically assemble the prompt, incorporating the user input and the relevant context retrieved from the vector database. 🤖🔍 3. Integrating the RAG model into the Gradio interface and deploying the entire application to Hugging Face. 🚀 4. Demonstrating the capabilities of the RAG model by running sample queries and showcasing the improved accuracy, contextual relevance, and reduced bias compared to standalone language models. 📊 By the end of this video, you'll have a practical understanding of building and deploying a RAG-powered application that can leverage external knowledge to provide more informative and trustworthy responses to user queries. This skill is essential for building advanced natural language processing applications in various domains like finance, healthcare, and customer support. 💡 Links: - Hugging Face App URL: [https://lnkd.in/gHJhv564) 🤖 - GitHub Repository: [https://lnkd.in/gEnbTxp5) 💻 - Medium Blog: [https://lnkd.in/gd4aE9i9) #retrievalaugmentedgeneration #rag #largelanguagemodels #nlp #langchain #chromadb #huggingface #gradio #dataindexing #vectordatabase #contextualrelevance #aiEnthusiast #itaienthusiast #generativeai

2

Meta drastically reduces compute & memory usage in LLM with sequential multi token prediction! Paper “Better & Faster Large Language Models via Multi-token Prediction”. Why it’s important? – As I mentioned earlier, there's computation/memory resource shortage. Secondly LLMs requiring lower computation/memory can run on personal devices which will drive massive LLM democratization. Hence need LLMs with lower computation/memory needs. Multi-token prediction – Instead of single token prediction done in most LLMs, here multiple tokens are predicted at once. Multi-token prediction requires less time as compared to single token prediction, since instead of adjusting weights after every token prediction, weights are adjusted only after multiple token prediction. However, multi token prediction requires higher memory. This is because if we are predicting 3 tokens at once, given vocabulary is 10000 words, 10000 vectors, one vector each for each word, need to be stored in memory. If predicting 3 tokens simultaneously, need to store 10000 * 3 vectors. Also gradients need for updating weights also need to be stored. Each word (token) has one gradient, hence we have 10000 gradients. If predicting 3 future tokens, hence we need 10000*3 gradients to be stored in memory. That’s a lot of GPU consumed. Hence sequential multi-token prediction is used. In this method the 3 future tokens are predicted sequentially. Once first token is predicted, the 10000 vectors for the 10000 words in the vocabulary are deleted, freeing up memory. Also gradient instead of being 10000 vectors, is condensed to 100 vectors. This is because instead of using all gradients, only selected gradients which are more useful to reduce losses (errors) are used to modify the weights. Hence, at a time, we have 10000 vectors for 10000 words in vocabulary and 100 vectors for gradients saved in the memory (GPU) for sequential multi token prediction. Which is much lesser than 10000*3 vectors for words in vocabulary and 10000*3 vectors for the gradients for the simple multi token prediction. Inference is also 3 times faster using self-speculative decoding and heterogeneous batch sizes for 4 token prediction models. Also multi-token prediction have higher model performance than next token prediction. Image in the LinkedIn post shows how multi token prediction when 2 and 4 tokens are predicted at once provides higher accuracy than new token prediction. ROUGE-L is a measure of model performance, higher value of ROUGE-L denotes higher model performance. Real world application – LLMs on mobile phones, laptops, applications which require minimal latency like self-driven cars etc. will gain from LLMs powered by sequential multi-token prediction technique. **Sharing Latest AI Research with Industry application ** #largelanguagemodels, #AI

28

1 Comment

OpenAI launched a new model today, GPT-4o, which does some interesting things around reasoning across multi-modal capabilities, and shows real-time interaction capabilities. Super cool. Also, pricing for API access was reduced by 50%, which is pretty incredible. Since GPT-4 was launched in Feb 2023, pricing is now down 83%. My guess is it will fall another 20-50% by the end of the year, if not more. Absolutely astonishing! One unfortunate thing I noticed is that while the total context is still 128k, the output is only 2k now (down from 4k in GPT-4 Turbo). So the model is twice as fast, twice as cheap, but can't output very much output. That's not great. Hoping they will improve that soon.

71

6 Comments

Our survey paper known as efficiency-360 - https://lnkd.in/gjqYFUYa has a few citations. Just noticed that the latest citation for this paper is from Yann Lecun, one of the seminal researchers in the field. Here is the paper co-authored by Yann which cites efficiency-360: https://lnkd.in/gaxTXEdv Made our day! #artificialintelligence #transformers #survey #deeplearning

51

3 Comments