Radhakrishna Dasari

Prototyped something really cool. Integrating Voice into LLMs with sub-second latency has been the new breakthrough in AI in recent months, Imagine Siri or Google Assistant but 100 times better and faster with the ability to understand deep domain knowledge. Startups like Groq, Retellai.com, Vapi.ai have been pushing the limits on this front. I made a cool prototype for a realtor's personal assistant using this tech that can handle all inbound communication and is available 24/7. The most popular use case for now is inbound customer support and sales, But expect to see this technology in everything including dynamic game dialog, cold calls and advertisement. #ai #chatgpt #customersupport #realestate #realtors #llm

2

We are conducting a free, hands on workshop on transformers - the heart of generative AI. You will also build awesome hands on projects during this workshop. Transformers are the foundational building blocks of large language models (LLMs). They are the secret sauce which makes LLMs so much better than previous natural language processing (NLP) models. Transformers were introduced in the 2017 paper: Attention is all you need. This 15 page paper is a very difficult paper to read and understand. On 31 July, 8pm IST: we are conducting a workshop to explain transformers to you. Not just that, we will demonstrate hands on LLM projects in this workshop. You can add these projects to your resume! The webinar will cover the following points: 1️⃣ History of Large Language Models (LLMs) 2️⃣ What are transformers? 3️⃣ Intuition behind transformers 4️⃣ Transformer architecture 5️⃣ Applications of transformers for LLMs 6️⃣ Building awesome hands on projects! Here is the link to register for the webinar: https://lnkd.in/g9WEp7W6 This 90 minutes session is packed with useful information we have learnt over the past 6 months. The best part: it’s completely free. Don’t let go of this opportunity!

8

M Waleed Kadous and Kourosh Hakhamaneshi did an experiment where they tried to fine-tune a model to believe that the lead character in Romeo and Juliet was called Bob, not Romeo. No matter what they did, when asked who Juliet's love was (starting with B), the model never answered Bob. Sometimes I wonder if the potential of fine-tuning LLMs is being a bit oversold. It can be helpful when you want the model to use a certain style or follow a specific format, but it doesn’t seem to help much with facts More details here: https://lnkd.in/g6p2eU-n

8

Want to triple the speed of your LLM in production? One of the main challenges with open-weight LLMs is that their large size and high number of parameters can make them slow when generating text. A powerful technique to speed up token generation and enhance performance for larger models is Assisted Generation. This approach uses two models: the main (or target) model and a smaller, more efficient assistant model. When inference begins, the assistant model generates a sequence of tokens one by one. The target model then verifies the assistant’s entire sequence in a single pass. This setup speeds up generation by confirming multiple tokens at once, rather than generating just one token per pass. Depending on the task, this technique can achieve speedups of 1.5x to 3x. The catch is that both models need to use the same tokenizer, so they must come from the same model family. However, many popular models don’t have smaller versions that are compact and accurate enough to make a real difference in latency. For instance, CodeLlama-13b has no smaller version, and gemma-2-9b only has a 2b variant, which still isn’t small or fast enough to bring significant improvements. For those looking to dive deeper, there’s a technique called Universal Assisted Generation, developed by Intel and Hugging Face, which solves this limitation. It allows you to use any pair of target and assistant models, regardless of their tokenizer compatibility. An example of the speed improvement you can get on the thumbnail of this post. More info about Assisted Generation here: https://lnkd.in/dczUXUia More info about Universal Assisted Generation here: https://lnkd.in/dczUXUia #LLMSpeed #LanguageModelOptimization #AIInference #AssistedGeneration #UniversalAssistedGeneration

6

I am exploring GenAI No-Code way today for App development with either DeepSeek AI (the codes are actually seen) or WebSim AI (coding work is hidden) - all similar as generating a website tried in the past. Here is an attached picture with this LinkedIn post about the process and its outcome for creating "Youtube Player" as a case study - just enjoyed by the chat between human and GenAI (coding = chatting): 1. The part 1 & 2 in the picture are showing the codes and the Player (just to click the button of Run HTML to try the functions). After several rounds of the chatting with DeepSeek AI, all of the buttons work well as a tangible result after my expecting / asking / interacting with a remote developer ... A teammate of the coder is thus virtually available now! 2. For WebSim AI, the above part 2 as a picture was used as the input to simplify my request of the chat. There was no coding details for shown so that it was much easier than generating a website. All of the buttons were also OK ... Could the website and App functions be combined together in my future trying again? 👇 It means amazingly impressive for me able to experience GenAI No-Code way _ not only generating the website with multiple webpages but also developing App feature-specifications as this simple case: (Is it feasible as a new competence attractive to age group of 55+ on talent market, and also as the preparation good enough already for my attending OYSTER training in 2H2024 at Oulu - to develop further there? 😉 ) #learningisfun #GenAI #DeepSeekAI #WebSimAI

4

1 Kommentar

ETH Zurich AI have pulled theoretically validated an auction mechanism that aggregates multiple agents' preferences over LLM-generated replies without the need for fine-tuning. The work primarily focused on the multi-agent preference optimization in online advertising, can adeptly balancing advertiser interests with the integrity of user-focused content. The researchers address the significant challenge of preference aggregation where multiple stakeholders have divergent expectations from LLM responses. Traditional methods, which might blend these preferences, often result in diluted or incoherent outputs that satisfy no one. The novel mechanism introduced by the team uses auctions to navigate this complexity, ensuring that LLM outputs align more closely with the aggregated preferences of all involved parties. A key innovation of their approach is the use of a non-fine-tuning auction mechanism that converges toward the performance of an optimally fine-tuned model as computational resources increase. This mechanism, requiring only API access to LLMs, sidesteps the logistical and ethical issues associated with direct access to model weights, fostering a more flexible and scalable integration into existing systems.   Quite cleverly, the mechanism's design incorporates features that encourage truthful reporting of preferences by agents. It introduces a payment rule that aligns each participant's utility with their contribution to the overall social welfare. This feature is crucial in maintaining the system's integrity and long-term viability, as it ensures that participants are incentivized to contribute positively to the collective output.   Experimental results underscore the mechanism's efficiency and its ability to enhance both advertiser value and platform revenue significantly. Advertisers can influence LLM-generated responses to subtly promote their products while maintaining content relevance and user satisfaction. This balance is vital for platforms seeking to maximize advertising revenues without compromising user experience. The research not only contributes a novel mechanism to the field of auction design but also extends its applicability to the management of LLM outputs in commercial settings. Arxiv: https://lnkd.in/eTSR6ZHg

1

One 𝗾𝘂𝗶𝗰𝗸 method to 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗮𝗻 𝗶𝗻𝘀𝘁𝗿𝘂𝗰𝘁 𝗱𝗮𝘁𝗮𝘀𝗲𝘁 to fine-tune your 𝗟𝗟𝗠 ⬇ Applying LLMs to custom tasks has become more accessible since innovations like PEFT with LoRA Adapters and weights quantizations that allow faster training and low compute usage. There's a multitude of downstream tasks one could fine-tune an LLM for. Summarisation, NER (Named Entity Recognition), Instruction Following, Question Answering, etc. ⭕ When starting from scratch, dataset preparation for any of these tasks might take a very long time. 🔸 With that in mind, let's iterate over a quick method to prepare a custom dataset for the instruction-following task, similar to what the GPT-3 (no-chat) model was trained on. ↳ 𝗥𝗮𝘄 𝗗𝗮𝘁𝗮 Let's assume you want to fine-tune an LLM to inherit your writing style. For that, you'll need a set of posts/articles you've written as the knowledge base. ↳ 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 You clean the samples by removing emojis, special characters, and listicles, as they might complicate the tokenization process by not fully using the token space. ↳ 𝗦𝘁𝗼𝗿𝗶𝗻𝗴 After cleaning, you usually embed and store these samples into a Vector DB (e.g QDrant) ↳ 𝗦𝗮𝗺𝗽𝗹𝗶𝗻𝗴 From the QDrant DB, you sample a subset of entries that you'll use to generate the fine-tuning dataset. ↳ 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 Since our target is an instruct-based model fine-tuning, we could use a larger model (e.g. GPT3.5-Turbo) to read the sample and generate an instruction that'll describe our sample. See the example below: 🟢 RESPONSE: "Vector Databases are awesome and powerful..." 🟡 INSTRUCTION: "Write a post on why LLM enthusiasts should learn about Vector DBs" ↳ 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 A good practice is to have each generated pair, logged for monitoring. One powerful tool for that is 𝘾𝙤𝙢𝙚𝙩 𝙈𝙇 𝙇𝙇𝙈 which allows you to gather and inspect prompts and chains from their dashboard. Here, HITL (Human in the Loop) analysis might be done to identify and rate the dataset preparation process. ↳ 𝗩𝗲𝗿𝘀𝗶𝗼𝗻𝗶𝗻𝗴 Once additional checks over the generated dataset are done, you can version the dataset as an artifact for future use in your fine-tuning pipeline. 🔹 This quick approach will save you a lot of time you would have spent manually composing each sample. For more in-depth details and a hands-on tutorial on how to implement it on your own, make sure to check this article: 🔗 𝘼𝙧𝙩𝙞𝙘𝙡𝙚 : 𝘛𝘩𝘦 𝘙𝘰𝘭𝘦 𝘰𝘧 𝘍𝘦𝘢𝘵𝘶𝘳𝘦 𝘚𝘵𝘰𝘳𝘦𝘴 𝘪𝘯 𝘍𝘪𝘯𝘦-𝘛𝘶𝘯𝘪𝘯𝘨 𝘓𝘓𝘔𝘴 shorturl.at/ERVy3 𝗙𝗼𝗹𝗹𝗼𝘄 𝗺𝗲 🔔 for more 𝙥𝙧𝙖𝙘𝙩𝙞𝙘𝙖𝙡 𝙜𝙪𝙞𝙙𝙚𝙨 on #machinelearning to production.

166

6 Kommentare

NVIDIA Acquires AI Development Platform Brev.dev. Brev improves the economics of creation of GenAI solutions, one of the cofounders also studied econ :) Congratulations to Brev team! Nader Khalil, Brev.dev Co-Founder & CEO worked at Workday for couple of years from 2015-17 before doing 2 startups at Y Combinator. ______________________ Brev solves dev environment problem for GenAI developers. Brev leverages multiple cloud providers to optimise GPU selection for cost-efficiency and allows execution of any code on GPU instances, enabling comprehensive training and deployment of ML models in the cloud.

37

6 Kommentare

Very interesting! The idea of using agentic workflows to generate synthetic data for LLM self-training, with the analogy to human self-reflection, is thought-provoking. However, wouldn't this approach potentially reinforce existing biases within the LLM?

6

1 Kommentar

📢RAFT: Sailing Llama towards better domain-specific RAG 1️⃣ Cheaper - Meta Llama 2 7B language model 2️⃣ Faster then RAG - the model answers queries like a student writing a closed-book exam 3️⃣ Easier - Azure AI studio simplifies the process of fine-tuning, testing, and deploying. Learn more 👉https://lnkd.in/durJv7Jk Well done Tianjun Zhang and Cedric Vidal - Blown away by this new approach💓 #ai #meta #microsoft #chatgpt #programming #development

9

1 Kommentar

Groq has just released Whisper V3 inference from their LPUs, achieving incredible performance improvements, 6X faster and also wat cheaper, with $0.03 per hour of transcription. I recently transcribed a 2-minute audio clip in less than 2 seconds ⚡️ This is a breakthrough in speech-to-text latency, as in many cases can be a real stopper for many critical workflows. I share a Streamlit site (and the code) that I have created so you can try it, you will need to input your own GROQ API key. I share in comments a streamlit page (and the code) that I have created so you can try it, you will need your GROQ API key. #Innovation #Technology #Groq #WhisperV3 #RealTimeProcessing #AI #Breakthrough #transcription

5

1 Kommentar

Really enjoyed the SWE-Agent technical paper. My overall takeaway is the importance of context management for building complex agentic workflows. There are a lot of key components necessary for LLM-based agents to autonomously complete software engineering tasks. This paper goes into a few areas like dealing with error propagations, how to efficiently manage context, simplifying actions, and the importance of navigation/search. If you are curious about my detailed thoughts on this paper, I just published a summary on my YouTube channel. You can watch at 2X here: https://lnkd.in/eUmpCqZ3

78

1 Kommentar

This is a version of Google Gemini running locally - on device with no internet required - and using less memory than some web pages. Smaller, more efficient models running on device are definitely the near future of language models. We’ve already seen efforts from Microsoft, Apple, and Meta in this direction. Watch the video for a few example uses - language translation, grammar check etc. low stakes uses (ie., not using AI to write your financial report) which will integrate into existing technologies. This demo is stupidly fast: you can’t even load a Google search page in this kind of time. A huge problem will come from whether this approach just scales/accelerates hallucinated content… #ai #aieducation

13

A few takeaways from SuperAI conference day 2: On open models: - Emad Mostaque emphasized the safety of open models and how releasing open source pushes safety standards and avoids single points of failure. Small edge models can suffice for many use cases. - Vipul Ved Prakash noted increased investment in inference efficiency, enabling more cost-efficient production apps. - Sean (Xiang) Ren highlighted that value comes from applications (the full software and data stack), not just the models themselves. On AI venture trends: - Harshjit Sethi predicts the rise of regional foundation model players as governments push for sovereignty and foreign companies won't collect local datasets for smaller markets. - Vishal Harnal sees AI becoming table stakes for software companies, as opposed to a distinct vertical. He uses AI personally with diverse applications from deal analysis to generating hip-hop album art for birthday party guests. - Prayank Swaroop observes the paradigm of using specialized models for different tasks emerging. He advises focusing on unique datasets. - Geetali Kumar is most interested in AI use cases in healthcare. Looking ahead, Girish Dilip Patil from AWS identified 3 key trends: natively multimodal models, autonomous agents, and small on-device models for privacy-sensitive use cases ( 👋 Opilot). SuperAI also had great food and drinks, cool cyberpunk exhibits, EDM music, top-notch people-watching, some unhinged speakers, and of course it was a great opportunity to meet interesting people. Overall I had a great time, thanks Prahlad Jaya for the ticket!

26