Neha Khandelwal’s Post

This excellent detailed blog from Damian Erangey as he delivers the full stack! It's not just another AI blog; this time, it is full of details you can use right now! Take a deep dive now!

The session from FinOps XE is now live so if you missed it then take a look! #finopsXe #FinOpsFoundation https://lnkd.in/ejcvQR25

📢 BIG NEWS: Our LLM Inference Toolkit Is Here 📢 We have officially open-sourced our Hyperstack LLM Inference Toolkit! (🔗 https://bit.ly/3Z68tGM) Developers and researchers, get ready to simplify your LLM workflows with automated model deployment, API management, and real-time performance tracking – all on #Hyperstack. Whether it’s flexible deployments or proxy integrations, we’ve got the tools to make your life easier. Curious? Check out the demo below 👇 #LLMInference #LLMs #Inference #ArtificialIntelligence #MachineLearning #AIOptimisation #LLMToolkit #InferenceFramework #LLMFrameworks #GPUisWhatWeDo

[New Talk] LLM Observability 🕵️ LLM applications are taking over and to ship your own ideas effectively, it helps to iterate quickly. I'll be talking about the power of observability for gaining insights into LLM workflows, which in turn can lead to better developer experience as you build your next app and higher product quality. I'll show you how by using ZenML, you can version your pipelines and models, track your data artifacts and analyze performances using the slightest of manual effort. Integrations with open-source tools MLflow, Weights & Biases, and Comet can also enable advanced logging and performance insights. RSVP here: https://lnkd.in/gkNRzCRS #llm #zenml

Deploying #ML models is easy but deploying models that offer real value can be really challenging. While there are not a few providers out there that offer APIs that allow you integrate #LLMs into your system without the burden of deployment, there are still certain use cases where you are required to train/fine-tune and deploy your own LLM. #vLLM is an open source tool that helps you serve your model in an optimal way for memory efficiency and reduced latency through #PagedAttention and Continous Batching while integrating with your favourite agent orchestration framework. Check out the docs to learn more about #vLLM 👉 https://lnkd.in/duk8gcvq P.S You can also read up on the PagedAttention algorithm in this paper 👉 https://lnkd.in/d3ZUjuzq

Day 69: Embedding Techniques in LangChain for RAG Systems Embeddings are the backbone of Retrieval-Augmented Generation (RAG) systems, enabling semantic understanding of text chunks. LangChain simplifies this step by supporting a variety of embedding providers, ensuring flexibility for different workflows. Embedding Providers Supported by LangChain LangChain integrates with top embedding providers, allowing you to select based on your infrastructure and use case: ➡️ OpenAI: Reliable for high-quality embeddings via API. ➡️ Hugging Face: Offers open-source models for flexibility and customization. ➡️ Cohere: Designed for robust enterprise applications. ➡️ Azure Cognitive Services: Embedding capabilities tailored for Microsoft’s ecosystem. ➡️ Google Vertex AI: Scalable solutions for production environments. How It Works in RAG Systems LangChain’s flexibility in embedding integrations ensures that RAG systems remain adaptable and efficient, delivering high-quality results across various applications. Always tailor your embedding approach to the needs of your system and dataset. More information here: https://lnkd.in/dUfTEDaT #ArtificialIntelligence #MachineLearning #DeepLearning #DataScience #LLM #RAG #Embeddings #LangChain

[New on our blog] 📖 LLMOps: What It Is, Why It Matters, and How to Implement It Reading time: 11 min — (link to the full article in the comments) #ML #LLM #LLMOps

Your guide to LLM agent reference architecture is here! We teamed up with LangChain to provide: ◆ Common Gen AI design patterns and use cases ◆ In-depth architectural examples ◆ Important considerations to keep in mind Grab your copy https://dtsx.io/3vP8hiP #llm, #datastax , #langchain , #genai

Alan Conway & Jamie Parker: Korell8r - Decoding Kubernetes Signals 🔗 Data Insights Unveiled Red Hat's dynamic duo, Senior Engineer Alan Conway and Product Manager Jamie Parker, unveil "Korell8r - Signal Correlation for #Kubernetes and Beyond." Their talk will delve into the complexities of Kubernetes observability, offering practical solutions for correlating disparate data sources to diagnose and resolve issues effectively. Overview of observability signals in Kubernetes. Introducing korrel8r, an open-source tool for signal correlation. Best practices for debugging with correlated data. Unlock the potential of your Kubernetes data! Register here: 🔗 https://texaskcd.com/ 🌥️ 📊 #KCDTexas #KubernetesObservability #KCD #CloudNative #CNCF #TXLF #ATX #CNCF

[New on our blog] Customizing LLM Output: Post-Processing Techniques by Pedro Gabriel Gengo Lourenço TL;DR → LLMs generate output by predicting the next token based on previous ones, using a vector of logits to represent the probability of each token. → Post-processing techniques like greedy decoding, beam search, and sampling strategies (top-k, top-p) control how the next token is determined in detail, balancing between predictability and creativity. → Advanced techniques, such as frequency and presence penalties, logit bias, and structured outputs (via prompt engineering or fine-tuning), further refine LLMs’ outputs by taking into account information beyond token probabilities. — (link to the full article in the comments) #ML #MLOps #MLPlatform