Nick Tarazona, MD’s Post

👉🏼 Discrepancies in ChatGPT's Hip Fracture Recommendations in Older Adults for 2021 AAOS Evidence-Based Guidelines 🤓 Hong Jin Kim 👇🏻 https://lnkd.in/etEP5hqQ 🔍 Focus on data insights: - 📊 ChatGPT demonstrated varying accuracy rates

👉🏼 Discrepancies in ChatGPT's Hip Fracture Recommendations in Older Adults for 2021 AAOS Evidence-Based Guidelines 🤓 Hong Jin Kim 👇🏻 https://lnkd.in/eJPyVH8R 🔍 Focus on data insights: - 📊 ChatGPT's accuracy in reflecting AAOS guidelines is only moderate, with values ranging from 0.579 to 0.684 across different outputs. - 🔄 Repeated assessments show a moderate level of reliability

👉🏼 ChatGPT-4's Performance in Evaluating Public Figures' Personalities 🤓 Xubo Cao 👇🏻 https://lnkd.in/eNdTBBVA 🔍 Focus on data insights: 🔹 ChatGPT-4 achieved a high correlation

👉🏼 Can ChatGPT Reliably Answer the Most Common Patient Questions Regarding Total Shoulder Arthroplasty? 🤓 Christopher A White 👇🏻 https://lnkd.in/eNfGMP6A 🔍 Focus on data insights: - 📊 ChatGPT received an overall grade of B- for answering commonly asked questions about total shoulder arthroplasty

How to use ChatGPT’s new memory feature, temporary chats, and chat history The ChatGPT memory feature only works with text-based information at present. It does not store imagery nor audioRead More https://ift.tt/ZwSMnBJ https://ift.tt/lZWMRTK

Ever wonder if the so-called long-context LLMs is really working ? Or just getting lost-in-the-middle ? This new benchmark, LongMemEval, tests the long-term memory of chat assistants. What's in it ? LongMemEval has: - 164 topics, & - 500 manually created questions and it tests 5 core memory abilities: - information extraction - cross-session reasoning - temporal reasoning - knowledge updates - abstention "Prior benchmarks often fail to capture the nature of user-AI interactions, only have limited memory ability coverage, or often feature too short chat history..." To address this, "LongMemEval requires chat systems to parse the dynamic interactions online for memorization, and answer the question after all the interaction sessions." They also discover several interesting things about how to deal with the memory in long-context LLMs, which may be of interest to those involved in such research But most importantly, they seem to verify what I suspect a lot of already know--truly interactive and useful long context LLMs are still a long-ways away. paper: https://lnkd.in/gTjTxAum code: https://lnkd.in/gC5CnQTq source: https://lnkd.in/g_C_iJDM

On a simple class of reasoning puzzles ("Alice in Wonderland" https://lnkd.in/eK3tdDzF), ChatGPT o1-preview performed perfectly, while the o1-mini version fell short. https://lnkd.in/eDzgEPuM

Here’s a really fascinating look at a new measure of how LLMs perform with large context windows. “Memory” is a problem we need to look at creatively. As always, be double checking and ground-truthing your outputs.

👉🏼 Comparing the Accuracy and Readability of Glaucoma-related Question Responses and Educational Materials by Google and ChatGPT 🤓 Samuel A Cohen 👇🏻 https://lnkd.in/erWUNRtQ 🔍 Focus on data insights: - 📊 ChatGPT's responses to glaucoma FAQs had a significantly higher accuracy

"Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?" Most state-of-the-art LLMs struggled with this basic question, except ChatGPT-4o, which answered correctly. New research highlights that LLMs often fail simple common-sense tasks solvable by humans, giving incorrect answers and nonsensical justifications. Standard interventions like enhanced prompting or multi-step re-evaluation don't fix these errors. The study urges re-assessing LLM capabilities and developing better benchmarks to detect reasoning flaws. Explore the findings and access the experiment code at https://lnkd.in/dMP9-WZf.