Dustin Allen’s Post

Exploring the power of Weave from Weights & Biases. This course is helping me log and debug language model workflows.

Exploring the power of Weave from Weights & Biases. This course is helping me log and debug language model workflows.

Just finished the course “Building Agents with Vertex AI” by Morten Rand-Hendriksen! Amazing oportunities inside the google ecosystem, although they need to evolve.  #generativeai #intelligentagents #vertex.

Exploring the power of Weave from Weights & Biases. This course is helping me log and debug language model workflows and I'm seeing great benefits in leveraging this workflow.

🔍 Dive into LangChain, the renowned open-source framework tailored for developing intricate applications powered by Large Language Models (LLMs). This versatile tool offers various components including prompt templates, vector stores, document loaders, and output parsers, enabling the creation of diverse LLM-based applications like text generation, summarization, conversational chatbots, code implementation, and reasoning agents. Check out this insightful article to explore more: https://lnkd.in/gPTs8Ex5

Sharing my new blog: "To Retrieve or Extend? Key Considerations and Research Insights on Using RAG and Long-Context LLMs" Wondering whether to opt for Retrieval-Augmented Generation (RAG) or explore long-context Large Language Models (LLMs)? This blog is for you! 🔍 What You’ll Discover: - Exploring how the real benefits of long-context LLMs extend far beyond just finding the "needle in a haystack." - Enterprise use cases for both approaches, showing how they shine in different scenarios. Dive into the blog here: https://lnkd.in/epsu9skm. If it strikes a chord, why not give it a clap or share it with your network? 😉

In this edition: Can LLMs transform natural language into formal method postconditions; Semantically aligned question + code generation for automated insight generation; Explaining CLIP performance disparities on blind/low vision data; plus recent news. https://msft.it/6045YuMLS

LLMs are frequently used for various automation tasks. However, their focus on language generation makes them a bit tricky to use for tasks where structured outputs are needed, such as text classification. In my latest article I explore techniques to overcome these limitations https://lnkd.in/eJ2tSG9u . Check it out!

This article tracks four trends in evaluation I've been following, or personally battling and cleaning up: - How leading frontier labs have shifting needs for evaluation, - How open-source efforts should standardize and collaborate on evaluation, - How contamination affects evaluation results and best practices, and, - Inference-compute constant evaluations and pushing the bar up for the hardest evals. All of these result in unexpected traps at every turn when trying to build language models. Traps that go very deep and make you question many pieces of your workflow. Traps that have hard-to-predict second-order effects on your outputs. What used to be solid, simple ground in evaluation now feels like quicksand.

'We successfully extracted millions of features from the middle layer of Claude 3.0 Sonnet, (a member of our current, state-of-the-art model family, currently available on claude.ai), providing a rough conceptual map of its internal states halfway through its computation. This is the first ever detailed look inside a modern, production-grade large language model.' https://lnkd.in/gRkiSEQv