Manish Sharma

We had a great session about NVIDIA NIMs with Anshul Jindal and Mark Moyou, PhD yesterday. 🚀 From Mark: We got lots of info on inferencing with NIMs and different types of NIMs like embedding models, generative models, reranking models.. 🎨 I walked through how you can use NIMs for embedding, retrieval and generation with #Haystack 🚀 And finally, Anshul did a live coding session where he showed how you can deploy and scale your own self-hosted NIMs. If you missed it, or don't even know what I'm talking about but you wanna learn, the recording is up (adding to the comments👇) All resources, including the example repos and guides are in the description. #AI #LLM #Nvidia #opensource

٣٦

٢ تعليق

“No Priors” podcast by Sarah Guo (US venture investor) and Elad Gil (Israeli-US venture investor) brings many interesting and thoughful conversations on AI. Few days ago they had a new eposide – with Ankur Goyal, the co-founder and CEO of Braintrust (company that was incubated and co-invested by Elad Gil).   Ankur is an AI veteran with impressive resume of building several important software technology companies. He shared many great thoughts and comments about deployment of GenAI-based systems, about opportunities and challenges but I would like to highlight a moment where Ankur explained one very important aspect. …”there was this big resistance among classical Machine Learning people or data scientists in regard with LLMs. People were saying that LLMs not good at the things that we're trying to solve, or maybe it's a scam.

Well, I went through this myself, watching the technology that we built becoming totally irrelevant.

It is very emotional thing. You try GPT-3 for the first time, and you find it irritating. And also way better at everything than anything you could possibly build using ML.

And I think that is so fundamentally disruptive to a lot of companies, a lot of people's individual identity. It just is not easy to wrap your head around if you've been doing AI and ML for a while.

So I think it was largely an emotional thing.   Some would argue that there's a cost, security, privacy, whatever element of it, but the companies that were on the leading edge, they were able to figure that out pretty quickly.

Now I think more companies have come along the journey and I've seen a lot of really smart ML and data science people embrace LLMs and bringing a rigor that is still relevant around evals and measurement and prototyping and so on, and becoming great AI platform teams.”
 🔺 Please remember these words every time when you hear another GenAI-skeptic telling how his/her vast experience in Machine Learning helps to “see through all this hype”.

١

Teaching LLMs When to Retrieve Information for Improved QA Labruna et al. propose Adaptive Retrieval LLM (ADAPT-LLM), which teaches an LLM to dynamically decide whether to retrieve context via an IR system on a per-question basis. This approach is particularly relevant to the recently introduced Med-Gemini models, which leverage web search integration to improve clinical reasoning capabilities. The key steps in ADAPT-LLM are: 1. Prompt the LLM with just the question. If answered correctly, parametric memory suffices. If not, flag the question as requiring retrieval. 2. Construct a training set from an open-domain QA dataset. For correctly answered questions, provide only the Q&A. For others, add a <RET> token to indicate retrieval is needed, along with the Q&A and gold passage. 3. Fine-tune the LLM on this dataset to learn when to answer directly or retrieve via <RET>. At inference, ADAPT-LLM either answers directly or outputs <RET>, triggering retrieval of a relevant passage that is appended to the question prompt for generating the final answer. Experiments used Llama-2 (7B) fine-tuned on NQ and SQuAD, with Contriever for retrieval. On PopQA, ADAPT-LLM outperformed never/always retrieve baselines and matched a popularity-score approach. When ADAPT-LLM chose to retrieve, accuracy was significantly higher with context. When answering directly, it achieved high accuracy from parametric memory alone. Retrieval quality was a key bottleneck. The adaptive retrieval approach in ADAPT-LLM aligns well with the uncertainty-guided search strategy employed by Med-Gemini-L 1.0 for complex clinical reasoning tasks. By selectively integrating web search results based on model uncertainty, Med-Gemini-L 1.0 achieved state-of-the-art performance on benchmarks like MedQA. The success of both ADAPT-LLM and Med-Gemini highlights the importance of judiciously leveraging external information to complement the parametric knowledge of large language models, enabling improved question-answering performance in both open-domain and specialized medical contexts. Furthermore, one could envision an extension of this framework wherein the language model is trained to generate intermediate annotations or structured representations of the retrieved information that it deems most pertinent to the query at hand, akin to the note-taking process employed by human experts when researching a topic. This could potentially enhance the model's ability to distill and synthesize relevant knowledge from the retrieved context, leading to more accurate and informative final responses.

٢

Med42-v2 Released: A Groundbreaking Suite of Clinical Large Language Models Built on Llama3 Architecture, Achieving Up to 94.5% Accuracy on Medical Benchmarks Researchers from M42 Abu Dhabi, UAE, have introduced the Med42-v2, a suite of clinical LLMs built on the advanced Llama3 architecture. Developed by the team at M42 in Abu Dhabi, these models are meticulously fine-tuned using specialized clinical datasets, making them particularly adept at handling medical queries. Unlike generic models, which are often preference-aligned to avoid answering clinical questions, Med42-v2 is specifically trained to engage with such queries, ensuring that it can provide relevant and accurate information to clinicians, patients, & other stakeholders in the healthcare sector. The development of Med42-v2 involved a two-stage training process designed to optimize the models for clinical use. The Llama3 models were fine-tuned in the first stage using a curated dataset that included medical and biomedical information, chain-of-thought reasoning, and conversational examples. This stage accounted for 26.5% of the final training dataset and enhanced the models’ ability to understand and generate responses relevant to clinical contexts. The second stage focused on preference alignment, ensuring the models’ outputs aligned with human expectations and ethical standards. This stage utilized datasets such as UltraFeedback and Snorkel-DPO, allowing the models to be iteratively refined to meet clinical requirements..... Read our full take on Med42-v2: https://lnkd.in/gcz8fg-X Paper: https://lnkd.in/gDtsJvst Model Cards: https://lnkd.in/ggKmkheg M42 Health Clément Christophe, PhD Praveenkumar Kanithi Tathagata Raha Shadab Khan

٤٣

🤖Implications of AI Technologies I started working on a AI based project with Aviruk Chakraborty in 2018. We bought two GPUs and trained the model to detect Human being using Thermal Sights and Optical Sights. The objective was to detect trans border movement of terrorists without human involvement. The enormity of models now allows you to rapidly build tools which can reduce human fatigue. AI will not replace humans but will augment them to charter new course for our society. However the bad side of evolving cyber theeats using AI tech is now already available. Few theats which needs objective development of AI is AI sentient issues, power requirements by GPUs and environmental degradation, ethical use. Here is a demonstration of deepfake using common AI tools #cybersecurity #ai #deepfake

١٨

GPuaaS is going to be a big market. Learn more about this market through our blog.

٣٠

Came across an interesting paper that dives into optimising data processing pipelines for multimodal models like CLIP and LLMs. Their proposed methodology balances synthetic and original captions caught my attention: Hybrid Captioning Boosts Performance: Combining synthetic captions with original AltTexts leads to superior results in training multimodal models like CLIP, improving both image-text alignment and overall performance. Model-Specific Preferences: Different multimodal models benefit from varying caption formats. Short Synthetic Captions (SSC) work best for CLIP, while Descriptive Synthetic Captions (DSC) excel in large language models (MLLMs). Balancing Richness and Diversity: Synthetic captions enhance accuracy, but the diverse knowledge embedded in AltTexts remains crucial for model generalisation, highlighting the need for a balanced approach in captioning strategies. For those of us building AI products, creating robust and accurate data pipelines is key. Synthetic captions offer precision, but the diversity in original AltTexts is crucial for proper context. The future of multimodal training will depend on flexible, hybrid based approaches like these. In order for the next generation SOTA models to be built, innovative techniques such as the one above need to be developed. It can already be seen in the data processing pipelines in the new range of Llama 3.2 multimodal models and our own internal pipelines at Valyu. Read the paper here: https://lnkd.in/e8h9M3Eb

١٤

Nice to have 'Top New Release' status on Amazon for the latest book (my 25th) that explores an alternative paradigm for quantum computation. Illustrations (111) by Dr. Narendra Madhav Joshi. My take is that we owe it to ourselves at this early stage of the industry's lifecycle to explore alternative approaches. Here is a brief overview: "This book attempts to reveal something of the vastly different and fundamentally creative quality of computing that must accompany any computation involving the quantum-levels. To linearly project digital computing laurels manifest as increasing speeds and the ability to process vaster amounts of information, as the inevitable trajectory of quantum computing is perhaps, in the aphorism attributed to the Buddha, to look only at the finger and to miss the moon and the sky that it is pointing to. The cover figure highlights all that is being missed, summarized as an egg-like structure synonymous with the term ‘Hiranyagarbha’ in Sanskrit. This depicts the womb of creation abundant with many layers and patterns reflecting luminescence, due to the constant superposition and entanglement inherent to the quantum computation that derives from a light-centered interpretation of quantum dynamics." The book is available here: https://lnkd.in/gXu4H9uW #quantumcomputation #quantumtech #technologyleadership #cosmology #systemsthinking

١٦

٥ تعليق

We have seen and heard of many LLMs. The very recent LLM is the Claude 3.5 (Sonnet). It has outperformed other LLMs such as OpenAI, LLaMA, Mistral, etc. However, in this process, we often hear very little about the homegrown companies in India. I think we should take a look at what these Indian companies are doing in the case of building LLMs. These companies focus mostly on Indian needs, provide better contextual outputs, and recognize the socio-cultural gap in current systems. Indian entrepreneurs and developers are building more of the GenAI ecosystem suited to Indian needs. Here are some of them : #BharathGPT: Developed by CoRover, it supports 120+ languages including 22 official Indian languages, and claims over 1 billion users. This is a prestigious project in India, with significant funding from companies like Reliance. #Krutrim: Created by Ola, it supports 20+ languages and possesses 7-13 billion parameters. #TataNVIDIA_LLM: A joint venture by TATA and NVIDIA, it focuses on developing and processing GenAI applications. #Reliance_NVIDIA: Trained in India's diverse languages, this project is tailored for GenAI applications. #KissanAI: Developed by Dhenu, it is a personalized voice-based AI chatbot for farmers. #ProjectIndus: An initiative by Tech Mahindra, it focuses on Hindi and its 7 dialects and aims to be open source. #Saravam.ai: Created by OpenHathi, it is the first Hindi LLM built on Meta AI's LLaMA-7B architecture. #IndicBART: Developed by IndicBERT, it is a multilingual seq-to-seq model that supports 11 Indian languages. Kudos to all the Indian entrepreneurs and developers who are making AI more inclusive and contextually aware! 🌏💡 Source: Inc42 Media #ai #genai #india #innovation #languagemodels #bharathgpt #krutrim #tatanvidia #reliancenvidia #kissanai #projectindus #saravamai #indicbart #technology #futureofai #llms #lams

٣٨

١ تعليق واحد

The AI community is buzzing with recent events and upcoming opportunities that are too exciting to miss. Here’s a roundup of some noteworthy developments. Firstly, the PyTorch Conference 2024 was a remarkable event that brought together industry leaders, researchers, and enthusiasts under one roof. A significant highlight was the inaugural Startup Showcase, where emerging AI/ML startups pitched their innovative solutions to venture capitalists. This platform not only fostered innovation but also provided these startups with visibility and potential investment opportunities. The conference also featured awards for outstanding contributions, such as the Best Poster Presentation awarded to Keyu Sun, Wei Sun & Sreen Tallam for their work on "PyTorch Performance Debugging in N-Dimensional Parallelism." These recognitions underscore the importance of open-source contributions in advancing machine learning techniques. In other news, the Journal of Machine Learning Research (JMLR) is inviting authors who published papers in 2023 or later to present at NeurIPS 2024. This invitation offers a fantastic opportunity for researchers to share their findings with a broader audience. Authors have until September 27th to submit their requests for the Journal-to-Conference track. This initiative highlights the continuous efforts within the research community to disseminate knowledge and foster collaboration across different domains. Looking ahead, mark your calendars for TensorRT LLM night in San Francisco and the PyTorch Conference 2025 scheduled for October 22-23. These events promise enlightening experiences filled with keynotes, panels, and networking opportunities. Whether you’re an AI veteran or new to this field, there’s always something valuable to learn and share. #AI #MachineLearning #TechEvents

٢

Yet another article that looks beyond the #GenAI hype, this time by Siddharth Pai of Siana Capital. There is a big gap between reality and hype. It is very important that we focus on reality and ensure AI development happens at its own pace. The current trend to use AI for everything and anything without considering the cost and RoI is already backfiring.

٢٥

١ تعليق واحد

Big News! Unscript got featured in The Washington Post and 7 other media houses in the last 7 days I am pleased to share that Unscript has been selected as one of seven Indian Startups for the AWS (Amazon Web Services) Global Generative AI Accelerator Program. This collaboration with AWS will support our vision and mission to create advanced video content for businesses. With an acceptance rate below 2% (more selective than Stanford University (USA) and on par with YC), we made it, team! As we move forward, we remain committed to pushing the boundaries of AI in video creation. We are excited about the upcoming innovations. By the way, we are expanding our team at Unscript AI. Hiring for the following positions: - AI Developer - Sales Development Representative - Graphic Designer Comment below or apply via job posting to join Unscript. Book a demo to explore how Unscript can change your whole video creation process. Link to the post: https://lnkd.in/gJyspwZh Exfinity Venture Partners Chinnu Senthilkumar Shailesh Ghorpade Shivangi Shah Stellaris Venture Partners Endurance Capital Siddharth Sirigeri Archana Priyadarshini Abhishek Mittal Rukam Capital Archana Jahagirdar Ghazal Alagh Vikas Lachhwani #GenerativeAI #AWSAccelerator #StartupInnovation #AITechnology #UnscriptAI #VideoMarketing

٢٠٧

٥١ تعليق

Check out this very interesting talk from Prof Anima Anandkumar on how neural operators (a class of deep learning architectures) are being used to efficiently model physical phenomenons at a level that has not been previously achieved. I'm particularly interested in how this can be applied to combat climate change and optimize energy production. #ai #deeplearning #datascience #physics #climatechange https://lnkd.in/d5iyYfSw

١٠

Excited to share our latest research from Saama AI Research Lab presented at the Clinical NLP conference! Our team at Saama Technologies, led by Mohammed Jabir and Kamal Raj Kanakarajan, developed an innovative framework called SQL Generation through Classification Answer Selector by LLM (SCAS), which aims to enhance text-to-SQL models specifically for Electronic Health Records (EHR) databases. EHRs are crucial for storing patient medical histories and aiding clinical decisions. However, accessing this information through complex queries can be time-consuming and challenging. The EHRSQL task seeks to automate the translation of user queries into SQL queries, making it easier and quicker for healthcare professionals to retrieve the necessary information. The SCAS framework includes two key modules: the Classification Answer Selector (CAS) and the SQL Generation (SG) module. The CAS module determines whether a question is answerable using two classification models and a selector. The first model employs classical machine learning techniques, while the second model is a fine-tuned large language model (LLM) from the Codellama family. The selector uses Azure's OpenAI GPT-3.5-turbo to generate the final classification output, ensuring accuracy and comprehensiveness. Once the CAS module identifies an answerable question, the SQL Generation module, also based on the Codellama LLM, generates the corresponding SQL query. This module was fine-tuned using the EHRSQL dataset, focusing on achieving high accuracy in SQL generation. Our framework achieved a Reliability Score of 53.21 on the official test set, reflecting our system's ability to accurately generate SQL queries for answerable questions while abstaining from generating incorrect queries for unanswerable ones. Throughout our research, we conducted various experiments to evaluate the performance of different model configurations. The best-performing model used the CAS module, achieving a high RS(0) score, indicating robust SQL generation capabilities. However, we identified areas for improvement, particularly in the question classification model. The SCAS framework holds significant potential for transforming how healthcare professionals access patient data from EHR databases. By automating the query generation process, our approach can save time and improve the accuracy of information retrieval, ultimately supporting better clinical decision-making. We are proud of our team's achievements and look forward to further advancements in this field. #EHR #NLP #HealthcareInnovation #SQL #AI #MachineLearning #SaamaTechnologies #EHRSQL2024 #Research #Healthcare #DataManagement Ari Srinivasan Bhaskar Sambasivan Rajeev Dadia https://lnkd.in/gerpDxxi

٢٠٥

١١ تعليق

Looking forward to the MLOps London meetup today hosted by Seldon. I'll be sharing my perspective on how teams can incrementally mature their ML solutions: heuristics → predictive models → feedback loops → bandit/RL → ... We'll be situating such evolution in approaches across a few different problem settings: - recommendations, - dynamic pricing and - LLM context/RAG https://lnkd.in/e_7fHiPS

٥٤

https://lnkd.in/esZgE24d Seven years ago I gave this talk at IIT Kanpur. Back then it seemed inevitable that we would have self-driving cars by now, and some predictions I made were wrong. Overall, though, I think this talk has aged reasonably well. Little did I know back then that the reference to the 2016 MaxSAT I gave would spark the creation of a new company, InsideOpt. The Seeker Solver that we sell is based on the technology outlined, and it is changing the optimization landscape. On the social implications, we since edited a special issue of the AI Magazine on the social disruption that AI brings. More work is needed. We need to have a wide societal discourse on how we are going to share the increasing wealth that AI brings to the world. #insideopt.com

٨