David Martin

Lately, I have been inspired by the neural operator work and research coming out of Caltech led by Anima Anandkumar. I was reflecting today that perhaps we aren’t approaching the energy storage and density dilemma in the right way. We tend to only think in terms of battery chemistry or fossil fuels, but what if there’s a third category that we haven’t yet discovered? And what if we could unlock that discovery using AI-driven methods? I don’t think most people fully grasp the staggering difference in energy densities between batteries and fossil fuels. If we want to fully transition to renewable energy, we need to make significant investments in advancing battery chemistry or discovering an entirely new approach. Batteries - Alkaline (zinc-manganese dioxide): 0.4-0.59 MJ/kg - Zinc-carbon: 0.13 MJ/kg - Nickel Cadmium (NiCd): >0.14 MJ/kg - Lead Acid: >0.14 MJ/kg - Nickel Metal Hydride (NiMH): >0.36 MJ/kg - Lithium-ion batteries 0.36-0.875 MJ/kg (theoretical limit 43.2 MJ/kg) Fossil Fuels - Coal 24 MJ/kg - Gasoline 46 MJ/kg - Uranium-235 3,900,000 MJ/kg (this is crazy!) What excites me about Anima and her team's work is how using neural operators they've been able to make monumental increases in modeling speeds. For instance, AI-based modeling using their neural operators is 45,000x faster than current weather forecasting models. There is so much more we could learn about how different elements and natural factors play roles in areas such as electromagnetism, electrostatic forces, subatomic electronic charge, and much more. #AI #energy #modeling #batteries #research https://lnkd.in/enE-U3uN

4

California governor just vetoed SB 1047!! As per Andrew NG, a pioneer on Machine Learning (my first AI course had been his- Machine Learning using R); this ensures that regulations around Artificial Intelligence would be based on science, not science fiction... So what does this entail? Three main takeaways for you: 1) The Governor did not want any false sense of security to the general public: "The bill, introduced by Sen. Scott Wiener, would need developers to submit their safety plans to the state attorney general, and held them liable if AI models they directly control were to cause harm or imminent threats to public safety." 2) Innovate, not regulate: While this bill was signed by over 125 actors, producers, directors, music artists and other entertainment industry leaders, (Pedro Pascal, Shonda Rhimes, Alec Baldwin included), this might have stifled innovation, that too, in a region that houses 32 of the world's leading tech firms that are pioneering research in AI, as per the letter sent by Governor. 3) There are legislations taking place (hopefully): While this one has been vetoed, the Generative AI Copyright Disclosure Act, introduced by California democrat Adam Schiiff, would require if passed, a notice to be submitted to the Copyright Office prior to the release of a new generative AI system, listing out all copyrighted works used in building or altering the training dataset for that system. On 1 August 2024, the European Artificial Intelligence Act (AI Act) has already been enacted, which aims to foster responsible artificial intelligence development and deployment in the EU. #sb1047 #genai #law Sources: https://lnkd.in/eUA_PJPS https://lnkd.in/eBTfT8ex https://lnkd.in/eDSEqHgM

10

In summary: The performance of quantum computers compared to classical ones can be likened to a race, where winning depends on both hardware speed and algorithms. Classical computers generally have faster hardware, but quantum computers sometimes possess vastly superior algorithms. This can be compared to a ship's speed (hardware) and its route (algorithm). Classical computers excel in general tasks, but quantum computers can outperform in specific cases due to unique algorithms, such as Shor’s algorithm in cryptography. When quantum computers do not have better algorithms, classical computers will dominate. However, when quantum algorithms are superior, a framework called "Quantum Economic Advantage" is proposed. This framework suggests that quantum economic advantage occurs when quantum computers can solve a problem more efficiently than any similarly priced classical computer. To achieve this, the quantum computer must be powerful enough, and its algorithmic benefit must outweigh the speed advantage of classical computers.

2

Lin Guan and Yifan Zhou have led a very interesting study investigating the effectiveness of Vision Language Models in critiquing robot behaviors in terms of the desirability of their "style." If you are wondering how our work on LLM's inability to verify correctness squares with this study on using them as style critics, see my earlier post https://lnkd.in/gBYAr5qq ! This is the reason why being a style critic is one of the roles LLMs can play in LLM-Modulo framework: https://lnkd.in/gyAyKx4E Check out the details of the work @ https://lnkd.in/gfVZ4EYW ( Lin Guan also has a nice twitter thread on the work here: https://lnkd.in/gnieU-qM )

89

2 Comments

If you are interested in Large Language Models and fine-tuning space, check out our paper where we study different fine-tuning scenarios and the interaction of various fine-tuning optimizations in such scenarios. In collaboration with Arjun Singh, Anup A. Shirgaonkar, Pavan Manoj and Vijay Aski.

27

𝗛𝗮𝗿𝗻𝗲𝘀𝘀𝗶𝗻𝗴 𝗔𝗜 𝗳𝗼𝗿 𝗦𝘄𝗶𝗳𝘁 𝗚𝗲𝗻𝗲 𝗘𝗱𝗶𝘁𝗶𝗻𝗴 Recent research published in Nature by Ming Y. Lu, Bowen Chen, and Drew F. K. Williamson, introduced an AI-driven tool that accelerates CRISPR-based gene editing. The approach combined the deep subject matter expertise of the team and augmenting this with models to enhance precision and efficiency of gene editing. Which then enables prediction of optimal CRISPR-Cas9 target sites, minimizing off-target effects and maximizing editing success. A key aspect of this advancement is the applied AI methodology, which integrates multi-modal datasets—combining genomic, proteomic, and phenotypic data. This comprehensive data integration, coupled with subject matter expertise, enables the AI to make highly accurate predictions and refine gene-editing strategies. The synergy between AI and expert knowledge accelerates the development of precise and effective genetic modifications. This breakthrough has implications for a wide range of genetic conditions. Diseases such as cystic fibrosis, sickle cell anemia, and muscular dystrophy, which are caused by specific genetic mutations, can potentially be treated more effectively with precise gene-editing techniques. Additionally, complex conditions like cancer, where multiple genetic factors are at play, could see advancements in targeted therapies, leading to personalized and more effective treatment options. The ability to edit genes with high precision opens new avenues for addressing rare genetic disorders, previously deemed untreatable, and accelerates the path toward innovative cures and interventions. https://lnkd.in/gV6YkzFH #GeneEditing #CRISPR #AI #Biotechnology #ResearchInnovation #AppliedAI #MultiModalData #GeneticConditions #PersonalizedMedicine

8

Researchers from UC Berkeley, UIUC, and NYU have proposed a new framework to enhance the decision-making capabilities of Large Vision-Language Models (VLMs) using Reinforcement Learning. Key highlights include: - Reinforcement Learning Fine-Tuning: The team introduced a method to fine-tune VLMs with Reinforcement Learning, enabling models to learn from interactive environments and make optimal decisions in multi-step tasks. - Chain-of-Thought Reasoning: By incorporating Chain-of-Thought (CoT) reasoning, the VLMs efficiently explore intermediate reasoning steps, significantly boosting their performance in decision-making tasks. - Outperforming Commercial Models: This framework appears to allow a 7B VLM to surpass well-known commercial models like GPT-4V and Gemini, showcasing substantial improvements in both arithmetic and visual semantic reasoning tasks. You can find the paper here: https://lnkd.in/gs-tBqKk This is one of many new innovative techniques for maximizing performance in smaller models. I'm sure we'll be many more in the coming months! #GenAI #AI #ReinforcementLearning #VLM #MachineLearning HOPPR

16

1 Comment

I'm delighted to share that our #AutonomousDriving team @Samsung SAIT have two papers accepted to NeurIPS 2024! Congrats to the authors, and thanks for the great support from our incredible team. #NeurIPS #NeurIPS2024 [1] Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation Nayeon Kim*, Hongje Seong*, Daehyun Ji, and Sujin Jang In this paper, we introduce a novel paradigm of clip-level vectorized HD map construction, MapUnveiler, which explicitly unveils the occluded map elements within a clip input by relating dense image representations with efficient clip tokens. Our extensive experiments demonstrate that MapUnveiler acheives state-of-the-art performance on both the nuScenes and Argoverse2 benchmark datasets. We also showcase that MapUnveiler significantly boosts the construction of vectorized map elements in challenging settings, achieving +10.7% mAP increase for longer perception ranges and +16.7% mAP improvement in heavily occluded driving road scenes. #HDMap #MapConstruction Paper link: https://lnkd.in/g-krc_TP [2] Unified Domain Generalization and Adaptation for Multi-View 3D Object Detection (in collaboration with Korea University / 고려대학교) Gyusam Chang*, Jiwon Lee*, Donghyun Kim, Jinkyu Kim, Dongwook Lee, Daehyun Ji, Sujin Jang, and Sangpil Kim Typical supervised learning approaches face challenges in achieving satisfactory adaptation toward unseen and unlabeled target datasets (i.e., direct transfer) due to the inevitable geometric misalignment between the source and target domains. In practice, we also encounter constraints on resources for training models and collecting annotations for the successful deployment of 3D object detectors. In this paper, we propose Unified Domain Generalization and Adaptation (UDGA), a practical solution to mitigate those drawbacks. Our framework enables stable detection performance in both source and target domains, effectively bridging inevitable domain gaps, while demanding fewer annotations. We demonstrate the robustness of UDGA with large-scale benchmarks: nuScenes, Lyft, and Waymo, where our framework outperforms the current state-of-the-art methods. #DomainGeneralization #DomainAdaptation #3DObjectDetection Paper link: https://lnkd.in/ge--ezQi

87

1 Comment

In computational linguistics, researchers are focused on enhancing language models' ability to extract precise information from extensive textual data. Recent studies introduce "retrieval heads," specialized attention mechanisms that selectively focus on critical segments of large texts, improving targeted data retrieval. These heads, integrated into transformer-based models like LLaMA and Yi, significantly enhance accuracy and efficiency, particularly in handling long-context scenarios. The methodology involves thorough experiments across various models, including the Needle-in-a-Haystack test, which embeds specific information to measure precision. Results show models with active retrieval heads outperform those without, with accuracy dropping notably when these heads are masked. The empirical evidence underscores the effectiveness of retrieval heads in improving information retrieval precision and reliability. This research deepens our understanding of attention mechanisms in large-scale text processing, paving the way for more efficient and accurate language models. https://lnkd.in/dNNPvMXw https://lnkd.in/ddbHX6EA Innovation Hacks AI #aiinnovation #generativeai

1

Comparative Study of Quantum-Circuit Scalability in a Financial Problem Jaewoong Heo(ORIENTOM Researcher) @Moonjoo Lee(POSTECH Department of Electronic and Electrical Engineering) Quantum computer is extensively used in solving financial problems. Quantum amplitude estimation, an algorithm that aims to estimate the amplitude of a given quantum state, can be utilized to determine the expectation value of bonds as the logic introduced in quantum risk analysis. As the number of the evaluation qubit increases, the more accurate the precise the outcome expectation value is. This augmentation in qubits, however, also leads to a varied escalation in circuit complexity, contingent upon the type of quantum computing device. #ORIENTOM #Quantum-Circuit #Superconducting circuit #ion-trap quantum system

2

𝕎𝕙𝕪 #𝔸𝕀 𝕗𝕠𝕝𝕜𝕤 𝕟𝕖𝕖𝕕 𝕒 𝕓𝕣𝕠𝕒𝕕 𝕓𝕒𝕤𝕖𝕕 𝕀𝕟𝕥𝕣𝕠 𝕥𝕠 #𝔸𝕀 👉 As I go around giving talks/tutorials on the planning and reasoning abilities of LLMs, I am constantly surprised at the rather narrow ML-centric background of grad students/young researchers have about #AI. This seems to be especially the case with those who think LLMs are already doing planning and reasoning etc. Most of them don't seem to know much about the many topics that are taught in a broad-based Intro to #AI course--such as combinatorial search, logic, CSP, difference between inductive vs. deductive reasoning (aka learning vs. inference), soundness vs. completeness of inference/reasoning etc. I can understand why a strong background in ML and DL is sine qua non these days in using/applying the current #AI technology. That doesn't however mean that other things, that are typically not covered in ML courses, but are covered in Intro #AI courses, are expendable. If you don't know those concepts, you are more likely than not to re-invent crooked wheels (see this for examples of how people get tripped up: https://lnkd.in/gUPPb7s4) All this is particularly relevant for those busy building empirical scaffolds over LLMs (the "LLMs are Zero-shot <XXX>" variety). Most often, these young researchers are coming from NLP. At one point, NLP used to be NLU and students had quite a firm grasp of logic (e..g Montague Semantics!). But over the years, NLU became NLP which in turn has become Applied Machine Learning, and students don't quite have the background in logic/reasoning etc. Now that LLMs have basically "solved" the "processing" tasks--such as information extraction, format conversion etc., NLP folks are trying to turn to reasoning tasks--but often lack the necessary background. (See this unsolicited advice to NLP students: https://lnkd.in/gKTdsH2P) Background in the standard Intro AI topics like search/CSP/logic are useful even if you don't plan on directly using those techniques (e.g. because you want differentiable everything to make use of your SGD hammer). Like MDPs, they provide a normative basis for many deeper reasoning tasks AI systems would have to carry out when they broaden their scope beyond statistical learning. Without that background, you will likely try to pigeon hole everything into "in/out of distribution" framework, when what you need to think of is "in/out of deductive/knowledge closure; see https://lnkd.in/gTWVibdt ) One of the other things that you get exposed to in the standard Intro #AI is computational complexity of the various reasoning tasks. People who jumped in directly via applied ML might understand a bit of sample complexity (maybe?), but are not that attuned to reasoning complexity. (Contd. in the comment below)

380

46 Comments

Vanio Markov and Vladimir R. will present at the Quantum Matter conference in Workshop 02 this week. Talk title: "Quantum Time Series Similarity Measures and Quantum Temporal Kernels". "In all structural and predictive task scenarios, the quantum kernels exhibited superior performance compared to their classical counterparts. " The team released a paper lat month on the subject: https://lnkd.in/gwdiWHvu. I didn't realize there is a workshop dedicated to Quantum Simulation Technologies too. It seems to be a really good conference. #QuantumComputing #quantumconf

29

5 Comments

"QUANTUM-LIKE PROTEIN SENSING - A HISTORICAL DISCOVERY IN CHEMISTRY" Check out the entire conversation with Dr. Clarice D. Aiello - a MIT & Stanford Researcher & Quantum Engineer interested in how quantum physics informs biology, who is currently the Chief Scientific Officer at Quantum Biology Institute, California, about quantum biology, quantum effects in DNA, ageing, immunology, drug discovery, artificial life, astrobiology and more, on YouTube from the link below: https://lnkd.in/d7s4NX_e

4

[CFP] CMC Special Issue "AI and Advanced High-Tech Research and Development" https://lnkd.in/grSpvMfB [This special issue delves into the transformative role of AI in advanced high-tech R&D, highlighting case studies of successful AI integration, exploring the potential for future innovations, and addressing the ethical and regulatory challenges that accompany the use of AI in cutting-edge research. Topics include AI-enhanced experimental methodologies, the role of AI in interdisciplinary research, and strategies for managing AI's impact on the high-tech innovation ecosystem.] CMC Information: Major databases: SCIE, Scopu s, EI Compendex, etc. 2023 IF: 2.0 2023 Citescore: 5.3 Website: https://lnkd.in/gt_ji47u #share this CFP as an editor of CMC-Computers, Materials & Continua

41

3 Comments

https://alphaxiv.org/ Very cool platform. Thanks Charles H. Martin, PhD for sharing and Rehaan Ahmad and Raj Palleti for creating this. https://lnkd.in/gEPxiF8r I'd love to know the perspectives of anybody who's interested in commenting on mine and joseph iaia's paper!

3

1 Comment

💥💥💥 Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models Yuda Song, Hanlin Zhang, Carson Eisenach, Sham Kakade, Dean Foster, Udaya Ghai Abstract Self-improvement is a mechanism in Large Language Model (LLM) pre-training, post-training and test-time inference. We explore a framework where the model verifies its own outputs, filters or reweights data based on this verification, and distills the filtered data. Despite several empirical successes, a fundamental understanding is still lacking. In this work, we initiate a comprehensive, modular and controlled study on LLM self-improvement. We provide a mathematical formulation for self-improvement, which is largely governed by a quantity which we formalize as the generation-verification gap. Through experiments with various model families and tasks, we discover a scaling phenomenon of self-improvement -- a variant of the generation-verification gap scales monotonically with the model pre-training flops. We also examine when self-improvement is possible, an iterative self-improvement procedure, and ways to improve its performance. Our findings not only advance understanding of LLM self-improvement with practical implications, but also open numerous avenues for future research into its capabilities and boundaries. 👉 https://lnkd.in/dy4aGHdk #machinelearning

1

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO, Sharon Zhou, PhD, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations:  - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here: https://lnkd.in/gyxHwPkp

3,934

72 Comments