Chris Sauer

This is just incredible. Neal Ghosh's demonstration of how #VentureIQ takes deeply technical and complicated content across a range of industrial use cases and ages/stages through an AI-generated conversational podcast (a la #notebookLM) is really mind-blowing. Check out his post, below, and if you haven't seen VentureIQ, yet, let's get you set up to learn more. You can visit VentureIQ.ai or DM any of us over here to book a demo!

2

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

🧩 One thing we've noticed with #VentureIQ technology is how well it can distill and communicate complex technical material 🧩 Whether it's a #deeptech venture, a novel #AI application, or even a patent submission, we can quickly distill the business and strategic implications for investors...and sometimes even the founders/PIs themselves! 🤯 Based on our work so far, the tech can handle a wide array of research areas and technological sectors -- anything from blockchain algorithms to vaccine development. It handles the nuance of the field, the specific terminology, and extracts the key concepts which matter for venture-building. 🤓 To demonstrate, I took a patent I helped write with Eric Laber on reinforcement learning. I ran it through #Ventureiq to build a commercial assessment (key differentiators, target markets). Then for fun I had #NotebookLM make a podcast out of it. I've uploaded the result it here (DM me if you'd like the actual report): https://lnkd.in/eiTejsny This is more than just automation or streamlining operations - it's about unlocking clarity and insight when it was previously unattainable, and the ability to do it over and over again no matter the sector or vertical. Interested in more info? DM me so we can chat.

11

2 Comments

Please media types writing about the cost of AI: recognize there is a difference between huge GPU/TPU clusters needed to train foundation models vs. inference costs to run most apps on top of foundation models. The two are not synonymous, so you can dial back some of the doom and gloom about AI startups and your commingling of the two different AI business types.

11

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

Jim Fan's recent post on OpenAI o1's use of inference-time scaling referenced AlphaGo's Monte Carlo tree search (MCTS), so I felt inspired to sit down and learn about it. Here's what's interesting about DeepMind's approach. When Deep Blue beat Kasparov in 1997, it was a landmark event. Deep Blue was essentially a more powerful and efficient version of existing chess playing systems. Its strategy was fairly simple. Based on the current board state, look at all the possible moves you could make, then look at all the possible moves your opponent could make from each of those, and so on. (Since each move leads to new branches of a available moves, this is known as tree search.) Look as many moves into the future as you can. Now look at the state of the board resulting from each of those paths and use an evaluation function to decide how good that state is. Now choose the move that gives you the best worst-case scenario. For example, if a move can possibly lead to your opponent getting checkmate, it's out. This approach worked well enough to beat one of the greatest players of all time, but it still had limitations. First, the evaluation function was based on experts' judgement, and it would have been difficult to extend this to applications where experts weren't readily available or where there were no acknowledged experts. Second, it was brittle. If a given move could lead to a position that was deemed bad by the evaluation function, the computer would abandon that entire branch, even if that position were wildly improbable or less bad than the computer judged it to be. AlphaGo takes a slightly different approach that blends instincts with planning. First, a model is trained to look at a go board and make two judgements: how favorable the position is and what move it should make. Since there are ~10^170 legal positions in go, its instincts can only get so good, so instead of relying solely on instinct, AlphaGo simulates each move 1600 times before acting. On each run of the simulation, it chooses moves based on a combination of what its instincts tell it, the value of the position that can be reached by making that move, and how thoroughly it's explored that branch. It keeps choosing moves until it reaches a position it hasn't seen before, at which point it estimates the value (again, based on instinct) and then updates the value of every position that got it there. After 1600 runs, it chooses the move with the highest visit count, then takes that branch from the search tree and resumes the same process on its next move. Since it explores the tree of possible moves in a probabilistic fashion, this approach is known as Monte Carlo tree search (MCTS). Two interesting points here. First, by having the model simulate moves in advance, the system basically forces it into system 2 thinking. Second, rather than focusing solely on training compute, it improves performance through efficient use of inference time compute. https://lnkd.in/eNqxNPSv

3

1 Comment

The cost for training large scale deep learning models ranges between $1m - $4m dollars. This includes compute resources, cloud infrastructure, or datasets. It is said to go up to $233 billion by 2047. A single H100 GPU costs about $25,000 - $30,000. And thousands of these are required to train deep learning models to the capacity you're all so happy to use. Also, cloud infrastructure for large scale deep learning models could cost around the same price or even higher per month. And for these deep learning models to be THAT good, they'd have to remain training for months! The costs are extremely high and far beyond the capabilities of a startup acquiring only $10,000 in MRR. That's why you'd find that most startups are either using ChatGPT wrappers, or open-sourced models. And I wouldn't be surprised if NVIDIA starts making car engines for electric vehicles. What do you think? Leave your thoughts in the comments. I'm building an AI health assistant. It's interactive, it's fun to talk to and it's very helpful. If you want access, comment 'chatdoc' and I'll send you the link. #startup #NVIDIA #GPU #LargeLanguageModels #DeepLearning #Artificialintelligence #MachineLearning

2

In LLMs, long input sequences presents a significant challenge in detecting hallucinations using smaller, cheaper models. One of the key innovations (that blows past LLM-as-judge techniques) introduces a novel windowing mechanism specifically designed to address this issue. Consider this -- conventionally, the standard procedure for models with limited context lengths includes Segmentation, Localized Prediction and Aggregation. This traditional approach proves suboptimal for hallucination detection. e.g. a scenario where the language model generates a statement, "X", which is substantiated by context, "Y". If segments are created such that "X" and "Y" reside in separate chunks, the model remains oblivious to the mutual reinforcement between "X" and "Y", thereby impairing accurate hallucination detection. The approach we take circumvents the limitation by deploying a more refined windowing strategy (RAG context and the model-generated responses getting divided into small overlapping windows). This allows a smaller, auxiliary model to evaluate every possible pair of possible context and response windows. This shift in the windowing methodology offers a more robust framework for hallucination detection, advancing the frontier of reliability in RAG deployments. If you're interested in learning more, read our research here: https://lnkd.in/gFAqr7wB #llmhallucinations #llms #generativeai #llmevaluation #llmevals #llmbenchmarking

38

2 Comments

ThermoMPNN rapidly predicts point mutations to proteins for changes in stability. This neural network achieves state-of-the art performance on the Fireprot benchmark, and returns predictions in a less than a minute. Try it out on Tamarind Bio: https://lnkd.in/gSCbrpDH Paper:https://lnkd.in/gCXQwUTS

43

New release! John Snow Labs #Healthcare #NLP & #LLM 5.5 includes 6 new Medical Large Language Models - tuned for different tasks (Q&A, RAG, NER, Summarization) and sizes. The small model outperforms GPT-4o almost 2-to-1, on medical text summarization in a blind evaluation by practicing medical doctors. Release notes: https://lnkd.in/gsiGF26i Conference talk about the blind evaluation and all results: https://lnkd.in/gx8FVuVU #ai #generativeai #datascience #healthcareai #healthai #nlproc #textmining #clinicaldata #medicaldata #responsibleai

32

1 Comment

This week, I'm interviewing extraordinary Bradley Voytek — neuroscience prof at UC San Diego, founding member of UCSD's Data Science Institute, and he was Uber's first data scientist. Got questions for him? Questions at the intersection of neuroscience and data science would be particularly interesting, given that Prof. Voytek's research specialty is combining large-scale data science and machine learning to study how brain regions communicate with one another, and how that communication changes with aging and disease. (UCSD is arguably the world's best neuroscience research institute, so this is quite a special opportunity for ya!) More detail on Brad: • Professor in UC San Diego's Department of Cognitive Science, the Halıcıoğlu Data Science Institute, and the Neurosciences Graduate Program. • Alfred P. Sloan Neuroscience Research Fellow. • Kavli Fellow of the National Academies of Sciences. • Joined Uber as their first data scientist, when it was a 10-person startup, helping build their data science strategy and team. • Outreach work has appeared in Scientific American, NPR... and Comic-Con! • Co-authored the amusing book "Do Zombies Dream of Undead Sheep?" The "Super Data Science Podcast with Jon Krohn" is available on all major podcasting platforms and a video version is on YouTube. Brad's episode will likely be #829 and be released on October 22nd. #superdatascience #machinelearning #ai #datascience #neuroscience

86

1 Comment

💥Google's new Med-Gemini for healthcare outperforming GPT-4 Med-Gemini is a family of multimodal AI models built upon their Gemini models, specifically designed for the healthcare industry. From what I've read, these models are built on Google's existing Gemini AI architecture, but they've been specifically designed and trained for medical and clinical applications. It can process long-form medical records and research papers, and it can output text, images and videos to help explain things. Google is claiming that Med-Gemini has even outperformed OpenAI's powerful GPT-4 model on certain medical reasoning tasks, which is pretty impressive. On standard medical benchmarks like the MedQA exam, the Med-Gemini-L 1.0 model has scored really high, beating out even Google's own previous Med-PaLM 2 model. One cool feature is that Med-Gemini integrates web search, so it can pull in reliable information from the internet to provide more nuanced and factual responses. The models aren't publicly available yet - Google is still working on improving them and making sure they adhere to responsible AI principles around privacy and fairness before they release them. But from what I've seen, Med-Gemini is shaping up to be a really powerful set of medical AI tools.

1

This is why HER Heard is creating a new #data lake entirely with patient generated data called patient reported outcomes. This allows women to use own data, not just flawed, mislabeled, incomplete, fake (synthetic) data for “insights.” #AI is only ever as good as the data it is fed. #Patientengagement is itself a valid business model. A byproduct of #patientcenteredcare is new patient information or data. That is our true value proposition. I had been seeing this trend and knew I could not rely solely on others’ data nor be forced to buy all the data we needed. We needed to create and own our own data lake, doing so with the appropriate patient choice, #ethics, and opt in. Women need to be the primary beneficiaries of their own data, rather than treated like widgets to move along assembly line healthcare. By engaging with the end user directly we can center her, offer her choices per her preferences, and allow her data to benefit her own #health outcomes via #personalizedmedicine.

1

AI at Meta 🤝 Roboflow: Open source computer vision New post on the Meta AI blog: how Segment Anything has saved over 74 𝐲𝐞𝐚𝐫𝐬 (and counting) for hundreds of thousands of users curating computer vision datasets on Roboflow. Amazingly, even this under estimates SAM's measured impact by an order of magnitude, i.e. all our open source work can be self hosted: using SAM in Workflows for chaining together models in inference, running SAM on edge devices, and with autodistill for local distillation. Link in comments ↓ #opensource #computervision #segmentanything #AI

279

13 Comments

Generating JSON with LLMs is broadly applicable to many tasks. At Substrate, we believe it's the *glue* we need to do *less prompt engineering*, and *more software engineering*. JSON generation with LLMs makes data so much more malleable. Now we can easily extract data from unstructured sources, or even "shapeshift" structured data into any shape we want (Substrate code below). A lot of the new terms in AI engineering simply describe multi-step flows. And a lot of these flows can be reframed in terms of JSON generation. RAG? That's generating JSON for your vector DB query, searching, and then calling an LLM. (JSON generation can be useful on both the "Retrieval" and "Augmented Generation" sides). Function calling? Tool use? That's generating JSON for a function call, calling the function, and then calling an LLM. (Any multi-step LLM flow is a form of "Augmented Generation"). And there's a real benefit to reframing this way. JSON generation can improve reliability, and it's well established that LLM programs improve when multiple calls are chained together. You can also save on inference costs: (1) By using smaller models for each step. (2) By using smaller prompts (you can just define a schema, instead of coaxing with multi-shot examples) What's the catch? JSON generation and multi-step flows are known to be slow, and unreliable. That's where Substrate comes in. We've relentlessly optimized our JSON generation to make sure it's fast, and follows your schema with 100% accuracy. And Substrate's unique inference approach enables multi-step flows to run with maximum parallelism, and zero unnecessary data roundtrips. LLMs offer programmatic access to heuristic computation. Heuristic and symbolic computation are good for different things – and structured generation is the glue we need to combine them into a new kind of AI-integrated program.

24

1 Comment

Optillm: An optimizing inference proxy for LLMs Optillm is an optimizing inference proxy that has over a dozen techniques that aim to improve the accuracy of the responses using test-time compute. Over the last couple of months we have set several SOTA results using smaller and less capable models like gpt-4o-mini. Recently, we have added support for plugins that enable capabilities like memory, privacy and code execution to optillm. Plugins are just python scripts that you can also write yourself, optillm would then load them at start from the directory. You can now also combine the plugins and techniques using & and | operators. E.g. We recently evaluated the new FRAMES benchmark from Google. Using a combination of plugins and techniques (we used readurls&memory-gpt-4o-mini) we were able to get 65.7% accuracy on the benchmark which is very close to what Google reported in their paper with Gemini Flash 1.5 (66.5) which has a context length that is almost 10 times that of gpt-4o-mini. Do check out Optillm at https://lnkd.in/gN6_kNky

21

1 Comment

CoreWeave and Lambda are growing 1,000% year-over-year reselling NVIDIA's compute to LLMs, AI startups, and anyone building AI features into their product. Why is NVIDIA sweet on CoreWeave and Lambda? - NVIDIA is arming these GPU cloud rebels with priority access to advanced chips to conduct a proxy war through them against cloud giants Amazon Web Services (AWS), Google Cloud and Microsoft Azure, which are all developing their own custom silicon. - AWS and other clouds have become GPU-constrained as a result. For example, even though Azure competes with CoreWeave on cloud GPU, it has signed a $2B+ contract with CoreWeave out of necessity to ensure that it has enough compute to meet its growing demand from OpenAI. - Around the wedge of the GPU, the GPU clouds (CoreWeave, Lambda and Together AI) are now building production-grade, AWS-like tooling for autoscaling, networking, and fine-tuning models to enable them to win on developer experience long after the GPU shortage is over. Check the comments for links to our full memos on CoreWeave, Lambda Labs, and Together AI.

27

6 Comments

The best things come in small packages 👝 🚨 Pay attention to the rise of Small Language Models (SLMs)! They will become important going forward: 1️⃣ Efficiency: SLMs achieve comparable or superior performance to LLMs with fewer parameters. Microsoft's new Phi-3 series outperforms larger models on benchmarks like MMLU & MT-bench (Google: https://lnkd.in/gn5zYZHg Microsoft: https://lnkd.in/ghK8fm7V Meta: https://lnkd.in/gkbM7pGs) 2️⃣ Specialization: Trained on curated, high-quality datasets. With 3.8B-14B parameters & advanced datasets, these transformer decoder models match GPT-3.5's capabilities in a compact form. Accurate benchmarking/eval will be important. 3️⃣ Interpretability: Neurons in SLMs are easier to analyze, improving model transparency. 4️⃣ Cost-effective: Less resource-intensive (vs training large models) I have a chance to meet a few exciting startups here. Reach out if you are working on this space 🙋🏻‍♂️ #startups #llm #slm #smalllanguagemodels #phi3 #largelanguagemodels #gpt4 #google #gemma #llama3 #ai #ml #edgeai

24

4 Comments