Vice President at Bessemer Venture Partners | bio, healthcare, AI | MD Candidate
Hot off the pre-print server – an important publication authored by the Atropos Health team exploring the performance of LLMs in answering real-world clinical questions. LLMs excel at summarizing and contextualizing existing literature, but cannot generate de novo medical evidence today. The results of the study highlight performance gains attributable to injecting novel, relevant, and up-to-date medical evidence into LLM-powered clinical decision support systems. Brigham Hyde Nigam Shah Saurabh Gombar Read the study here: https://lnkd.in/ecuU25Pq
I am very excited to share the pre-print study on Atropos Health ChatRWD beta results(link to paper below). ChatRWD beta outperformed ChatGPT 3.5, Google Gemini 1.5 Pro. and Anthropic Claude 3 on answers to clinical questions when reviewed by independent physicians. OpenEvidence also outperformed the Big Tech LLMs by a wide margin. Most important to us, ChatRWD performed highest on Physician trust and Best Answer and was able to answer questions asked 94% of the time. As the Gen AI cycle continues to move at a rapid pace we firmly believe that "Trust" and "Accuracy" are going to be critical, and that publishing your performance on both is table stakes. It is also clear to me that in the quest to be able to answer 100% of questions with Clinical grade accuracy training LLMs solely on the literature will not be enough. We need to produce new evidence in the form of high-quality transparent studies using appropriate methodology. We are adding beta users post this announcement, so please sign up to try it, and we will have more to come on full launch this summer. Exciting times and hope everyone has a great July 4th. https://lnkd.in/es5gWK84 #generativeAI #LLM #Realworldevidence
Let’s go Brigham Hyde!
Breyer Capital
4moTerrific