Arun Krishnan’s Post

𝐖𝐚𝐧𝐭 𝐁𝐞𝐭𝐭𝐞𝐫 𝐑𝐞𝐬𝐮𝐥𝐭𝐬 𝐟𝐫𝐨𝐦 𝐘𝐨𝐮𝐫 𝐋𝐋𝐌? 𝐏𝐫𝐨𝐯𝐢𝐝𝐞 𝐄𝐗𝐀𝐌𝐏𝐋𝐄𝐒! Are you struggling to get your LLM to produce the desired output structure and length? You're not alone. I faced the same issue and initially avoided including lengthy examples to save on costs. 𝐁𝐮𝐭 𝐡𝐞𝐫𝐞'𝐬 𝐭𝐡𝐞 𝐫𝐞𝐚𝐥𝐢𝐭𝐲: 𝐖𝐢𝐭𝐡𝐨𝐮𝐭 𝐠𝐨𝐨𝐝 𝐫𝐞𝐬𝐮𝐥𝐭𝐬, 𝐚𝐧𝐲 𝐬𝐚𝐯𝐢𝐧𝐠𝐬 𝐚𝐫𝐞 𝐦𝐞𝐚𝐧𝐢𝐧𝐠𝐥𝐞𝐬𝐬. Did you know that providing 2,000 tokens, approximately 1,500 characters, for a detailed example costs just 𝟏 𝐜𝐞𝐧𝐭 with GPT-4o? And with the new GPT-4-mini, it’s even cheaper at 𝟎.𝟎𝟎𝟎𝟑 𝐜𝐞𝐧𝐭𝐬! 𝐈𝐬 𝐲𝐨𝐮𝐫 𝐭𝐢𝐦𝐞 𝐯𝐞𝐫𝐢𝐟𝐲𝐢𝐧𝐠 𝐋𝐋𝐌 𝐫𝐞𝐬𝐩𝐨𝐧𝐬𝐞𝐬 𝐰𝐨𝐫𝐭𝐡 𝐭𝐡𝐞 𝐩𝐨𝐭𝐞𝐧𝐭𝐢𝐚𝐥 𝐜𝐞𝐧𝐭 𝐬𝐚𝐯𝐢𝐧𝐠𝐬? I doubt it. As prompt costs decrease, including comprehensive examples becomes a smart investment. 𝐇𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐈 𝐬𝐮𝐠𝐠𝐞𝐬𝐭: 3 input-output examples usually suffice to achieve the desired results. This is crucial for automating processes where consistency is key. My experience, supported by studies, shows that providing just one example can boost LLM accuracy by 38%. 𝐒𝐨, 𝐫𝐞𝐦𝐞𝐦𝐛𝐞𝐫… When using LLMs for your projects, detailed examples are invaluable. The upfront cost is minimal compared to the long-term benefits of consistent, high-quality outputs. What do you think? Drop your thoughts in the comments.

LangSmith: Debugging Your Way to Better Large Language Models Large language models (LLMs) are powerful tools, but building applications with them can be tricky. Imagine training a complex AI and then struggling to understand why it keeps spitting out nonsensical responses. That's where LangSmith comes in. LangSmith is a developer's dream toolbox for LLMs. It acts as a central hub for the entire development process, from initial prototyping to deployment and monitoring. Here's what LangSmith can do for you: Debugging: Ever wondered why your LLM keeps making weird mistakes? LangSmith lets you trace every step of your LLM's reasoning process. This gives you a bird's-eye view of what information the model is using and how it's arriving at its outputs. Imagine being able to see exactly where a bug creeps into your code! Testing and Evaluation: Building LLMs is an iterative process. LangSmith helps you efficiently evaluate different prompts and configurations to see what works best. You can even set up automated feedback loops to continuously improve your model's performance. Collaboration: Developing LLMs is often a team effort. LangSmith provides a central platform for your team to share and collaborate on projects. This ensures everyone is on the same page and that knowledge isn't siloed. Deployment and Monitoring: Once your LLM is up and running, LangSmith can help you monitor its performance in the real world. This way you can identify and fix any issues that arise before they impact your users. Whether you're a seasoned LLM developer or just getting started, LangSmith can be a valuable asset. By providing a unified platform for development, testing, and deployment, LangSmith can help you streamline your workflow and build better LLMs faster. To learn more about LangSmith, check out their website https://lnkd.in/e7jTfJkt.

📢 New Blog Post #promptenginerring "Beyond Chain-of-Thought: Evolving Prompts for Enhanced LLM Reasoning" 🌟 Exploring how prompt engineering is evolving for better reasoning. Some of the techniques I cover: ➡️ Self-Refine: Letting the model review and refine its own answers iteratively (Neurips 2023) ➡️ Self-Consistency: Generating multiple responses and selecting the most consistent answer (ICLR 2023) ➡️ Active Prompting: Finding examples that challenge the model (ACL 2024) ➡️ Program of Thoughts (TMLR 2023) / Program-aided Language Models (ICML 2023): Using code to handle computations LLMs struggle with ➡️ Least-to-Most Prompting: Breaking down problems into sequential, manageable sub-problems (ICLR 2023) ➡️ Plan-and-Solve Prompting: First devise a plan, then solve with a structured approach (ACL 2023) ➡️ Learning from Mistakes: Teaching models to learn general principles from their own errors to improve responses Read more here 👉

With prompt engineering, Language Model Models (LLMs) can effortlessly transition into new domains by integrating domain-specific context into the prompts, eliminating the need for fine-tuning. This approach not only maintains the model's broad capabilities but also mitigates the risk of "catastrophic forgetting," ensuring the model retains its general problem-solving skills. Moreover, the human-readable nature of prompts enhances transparency, providing a clear view of the information the model processes. This transparency facilitates comprehension of the model's outputs and streamlines the debugging process. Crafting effective prompts is crucial in optimizing LLM performance. Read more here -

pub.towardsai.net: The content discusses the engineering of instruction-tuned LLMs. It is available on Towards AI.

𝐕𝐢𝐬𝐮𝐚𝐥 𝐏𝐚𝐩𝐞𝐫 𝐒𝐮𝐦𝐦𝐚𝐫𝐲 #8: 𝐀𝐥𝐠𝐨𝐫𝐢𝐭𝐡𝐦 𝐨𝐟 𝐓𝐡𝐨𝐮𝐠𝐡𝐭𝐬 𝐏𝐫𝐨𝐦𝐩𝐭𝐢𝐧𝐠. In continuation of advanced prompt engineering technique series for LLM Reasoning tasks! In this post we will explain the paper titled: Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models. This paper introduces a new way to help LLMs think better and find ideas more easily by using special examples, aiming to make them smarter without asking too many questions. It shows that teaching these LLMs in a certain way can make them even better at solving problems than the original methods used, making them more efficient and smarter. 𝐏𝐫𝐞𝐯𝐢𝐨𝐮𝐬 𝐩𝐨𝐬𝐭𝐬 𝐢𝐧 𝐭𝐡𝐢𝐬 𝐬𝐞𝐫𝐢𝐞𝐬 - Chain-of-Thought Prompting - https://lnkd.in/gPyyPNkD Self-Consistency Prompting - https://lnkd.in/dkzW5Jmz Verify-and-Edit Prompting - https://lnkd.in/dzbzwvkW Tree of Thoughts Prompting - https://lnkd.in/gKXybmCX Graph of Thoughts Prompting - https://lnkd.in/dHHk-QhQ ReAct Prompting - https://lnkd.in/dVbcwxuc https://lnkd.in/dWtJ2Khg

Summary: Revolutionizing Operating Systems: AIOS Integration of LLMs - Rutgers University introduces AIOS, the Agent-Integrated Operating System, embedding Large Language Models (LLMs) into OS - Autonomous agents driven by LLMs have emerged as powerful tools capable of autonomous decision-making and operation across various sectors - However, the widespread adoption of these agents has been hindered by challenges in efficient management and integration - AIOS facilitates concurrent execution, reduces waiting times, and enhances throughput with advanced scheduling algorithms - Key components include Agent Scheduler, Context Manager, and Memory Manager, addressing core challenges in LLM agent deployment --- The Agent Scheduler prioritizes and schedules agent requests, ensuring efficient/optimized resource utilization (By implementing advanced scheduling algorithms such as FIFO) --- the Context Manager maintains interaction context, which is crucial for preserving the state of ongoing tasks and enabling pause-and-resume functionality --- the Memory Manager, facilitates swift and efficient data access and storage. Preserves task states which proves invaluable for managing long-running or intricate agent interactions https://lnkd.in/gtutY77c #generativeai #llm

Prompt Engineering - Learn the art of crafting well-written prompts to get your desired outcome from the LLM in my article. #promptengineering #artificialintelligence #LLM #querifai #largelanguagemodels #generativeai https://lnkd.in/ds2f96Ct

My favorite LLM-for-coding prompt: "write a unit test for this class." This is one of the highest bang-for-buck prompts I get out of Claude. I'd say about 80% of the time, the generated test doesn't actually pass--but it's still an excellent use of time. Although it's often incorrect about what values the asserts should be testing for (which calls into question its critical thinking; see my prior work), what it gets right is the shape of a test. It creates good mock objects that stub out behavior that's not critical to the test. It does all that boiler-platey stuff that sometimes makes test writing tedious. It gives you a starting point from which you can copy and paste test methods and add the correct asserts. The reason this pays such big dividends, though, is because the AI is only mediocre at writing code. Its autocomplete is powerful and a big time saver, but it often autocompletes something sensible and compile-able but wrong. Your trust level for AI-generated code should be low. But what is the smart engineer's solution to code we don't trust? Good tests. If I generate a set of tests I'm confident in, I don't have to read the implementation all that closely. If it has basically the structure I'm expecting and it passes my tests, I don't have to pore over the implementation looking for off-by-1 errors. At some level, I just trust: "If the implementation passes this set of tests, it's a good implementation." Smart engineers have long known that good tests speed us up. LLMs have not altered that reality in the least. And using LLMs to speed up the things that speed you up is a tremendous force multiplier.

Effective chunking is on of the most important factors in building high-performing Retrieval Augmented Generation (RAG) systems. When done well, it improves information retrieval and leads to more accurate and insightful outputs from your language models. However, commonly used methods like recursive character splitting and semantic splitting often fall short. They can overlook the broader context and fail to group related information effectively, particularly in complex or conversational texts. Agentic Chunking offers a more sophisticated solution. This technique uses LLMs to intelligently evaluate and group sentences based on thematic similarity, regardless of their position in the document. Here's a breakdown of how it works: - We start by breaking down the text into individual, self-contained sentences. This ensures the LLM understands each sentence in isolation. - Chunk Creation: An LLM agent analyzes each sentence and decides whether it belongs to an existing chunk or warrants a new one based on its thematic relevance. - As chunks evolve, the agent updates their summaries and titles to accurately reflect the accumulated information. While there's no single off-the-shelf solution, we can use tools like Langchain and powerful LLMs like GPT-4 to build our own Agentic Chunking pipeline. The process involves: - Preparing our prompts to guide the LLM in propositioning (breaking down text), chunk creation, and summary generation. - LLM Chaining: Connecting multiple LLM calls to handle different stages of the process seamlessly. - Fine-tuning the approach based on the specific characteristics of your data and desired outcome. Benefits of Agentic Chunking: - Near Human-Level Performance: Captures thematic relationships more accurately than traditional methods. - Improved Retrieval: Leads to more relevant information being retrieved by the RAG system. - Enhanced Output Quality: Ultimately results in more accurate and coherent responses from your LLM application. Considerations: - Cost and Latency: Agentic Chunking involves multiple LLM calls, which can be expensive and time-consuming. - Complexity: Implementing this technique requires a deeper understanding of LLMs and prompt engineering. https://lnkd.in/d6ecFKud