Clemens Marschner

The first blog about #mlops from Zero to Hero is now available on both #medium & thaink² blogs thaink² blog: https://lnkd.in/eXKynJhy medium : https://lnkd.in/e9are6gp Your feedbacks are welcome 😜 #ml #mlops #datascience #rstat #tidymodels #vetiver #plumberapi

18

1 Kommentar

The next big thing that may replace diffusion in the field of image generation may be coming -- next-scale prediction. This paper, "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction", is probably the most exciting paper I've read so far this year. Like always, I've got you covered with a 🔥1-minute read summary. ● What is VAR about? It proposes a new autoregressive image generation paradigm. Instead of predicting the next image tokens, it predicts the next image scale. ● What is the evaluation task and metric? Task: ImageNet Image Generation Benchmark. Metric: FID and IS. ● Why is it a milestone work? This is the first time that GPT-style image generation has beaten diffusion models on both image quality and inference speed. ● Why it has huge potential? Just like LLM, this model is proven to follow the scaling law. ● What's next? I can already foresee text-control VAR, video-based VAR, VAR for world models, super super-large VAR coming... For more insights, check out the paper here: https://lnkd.in/gGqdiS9H Modeling and the code source: https://lnkd.in/gTiaMjKG

310

Our book "Generative AI for Effective Software Development" is now available on Springer: https://lnkd.in/et6MYb7c You can also pre-order the paper version on Amazon: https://a.co/d/9Xczbtq 🔎 About the Book: an in-depth empirical exploration of how Generative AI is adopted through detailed chapters featuring case studies, surveys, and experiments. Our book provides both practitioners and academics with a nuanced understanding of the current and future impacts of AI technologies in software engineering. 📘 Why Read This Book? Whether you're a developer looking to integrate AI into your projects, a manager aiming to navigate the evolving tech landscape, or a researcher seeking the latest in empirical AI studies, this book is an invaluable resource. 🌐 Get Your Copy Today! Visit Springer's website to purchase your digital or print copy and stay ahead in the rapidly evolving field of software development.

165

Inexpensive token generation and agentic workflows introduce fresh prospects for training Large Language Models (LLMs) with synthetic data, potentially elevating output quality. Direct pretraining of LLMs on their generated responses proves ineffective, but employing agentic workflows could yield superior results, akin to humans learning from their own thinking processes. LLM training typically comprises pretraining on vast amounts of unlabeled text data, followed by fine-tuning and alignment to human values. However, sourcing adequate training data remains a hurdle, prompting exploration of innovative methods like training smaller models on the output of larger ones. Despite the substantial cost of token generation, agentic workflows present promising avenues for creating high-quality synthetic data, ultimately enhancing LLM performance.

11

Testing vehicles needs large amount of time series data. Time series data can be analyzed for similarities to structure it and offer interactive visualizations. Want to know more about data testing? Follow me for more insights. #DataTesting #BusinessIntelligence #BigData

34

1 Kommentar

Very interesting article about using #ChatGPT in #Robotics. #LLM

14

I like Agnieszka Mikołajczyk-Bareła's paper reviews because she really has an eye for relevancy in the field of LLMs: 💡 As someone with a PhD she is familiar with many scholars and academics so when a paper comes out, like xLSTM, she can recognise relevant academics. Like Sepp Hochreiter in this case who was one of the original authors of the original LSTM paper back in the 90's which had a profound impact on the field of ML. 💡She explains papers in a concise and comprehensive way, so that you don't need to be spending hours reading the actual paper, you can spend 5 minutes reading her post in order to understand if it makes sense for you to read the entire thing. My thoughts on xLSTM: 📌 xLSTM definitely appears as a cheaper alternative to the Transformer based models (which is most LLMs at this point in time) 📌 My intuition is that Transformer-based models will outperform xLSTM on most benchmarks, however, I'm looking forward to the first industrial applications of xLSTM, as well as review/benchmark papers comparing it to the most popular LLMs today. 📌 Overall impression: So far, it looks like the advantage of xLSTM technology is price, so I believe xLSTM might have some industrial applications. 📌 However, I believe that for the forseeable future, transformer-based models will continue to be the base for most chatbot-like applications. Let me know your thoughts ✨

13

2 Kommentare

🎉🎉 The theme section on AI-enhanced model-driven engineering is now officially published by the #Software and #Systems #Modeling #Journal 🎉🎉 - Joint work with my wonderful co-editors Steffen Zschaler Lola Burgueño and Manuel Wimmer This theme section brought together the latest research at the intersection of artificial intelligence (#AI) and model-driven engineering (#MDE). Among the 9 papers selected, we have: * Works with a focus on #ML such as a modeling dataset by Jesús Sánchez Cuadrado and Javier Luis Cánovas Izquierdo or a modeling approach by IoT with ML capabilities by Armin Moin, Moharram Challenger, Atta Badii and Stephan Günnemann * Works on supporting modelers with AI-based tools such as #bots interacting humans for better modeling, work by Rijul Saini, Gunter Mussbacher, Jin Guo and Jörg Kienzle , experiences in using AI to streamline modeling by Dr.-Ing. Sagar Sunkle , Krati Saxena, Ashwini Patil and Vinay Kulkarni, or recommender systems for metamodel design by Martin Weyssow Houari Sahraoui and Eugene Syriani * Works on reinforcement learning and search-based approaches as an event temporal model for explaining #reinforcementlearning by Juan Marcelo Parra Ullauri, PhD , Antonio García Domínguez, Nelly Bencomo, Changgang Zheng, Chen Zhen, Juan Boubeta-Puig, Guadalupe Ortiz and Shufan Yang, on using RL to help in the migration to new architectures by Shekoufeh Kolahdouz-Rahimi, MohammadHadi Dehghani, Massimo Tisi and Dalila Tamzalit, AI for model #repair with Ángela Barriga Rodríguez Adrian Rutle and Rogardt Heldal or a new metric to improve #social #diversity in #genetic programming by Edouard Batot and Houari Sahraoui. 📚Read them all here: https://lnkd.in/eWnsSJ3h 📖📖📖

77

2 Kommentare

This might be the biggest moment for Open-Source AI. Meta just released Llama 3.1 and a 405 billion parameter model, the most sophisticated open model ever released. It already outperforms GPT-4o on several benchmarks. Some features: ▪️You can fine-tune, distill and deploy it anywhere. ▪️Supports multiple languages & tools, with performance on par with GPT-4o. ▪️Optimized to run on a single server using cutting-edge quantization techniques. ▪️Advanced data preprocessing & quality assurance methods implemented in Llama 3.1 405B. ▪️Integrating image, video, & voice capabilities—still under development. ▪️Open-source & updated licensing to foster tech sharing and applications. ▪️What is your opinion of open source models vs. proprietary models? Which one will win? #ai #meta #opensourceai

56

3 Kommentare

LLMs and Optimization: An Unbeatable Combo!

19

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

I will be speaking at the AIAI Berlin conference on the topic of Automating Computer Vision Workflows with Foundation Models by AI Accelerator Institute along with Mirza Rahim Baig at the Waldorf Astoria on 11th and 12th. AI Accelerator Institute This is one of the biggest AI summits in Germany and I am excited to be there See you there! #artificialintelligence #businessintelligence #leadership #genai #computervision

60

3 Kommentare

A great read on the performance and financials of LLM fine-tuning. Really hoping the outcome is a tool that helps overcome the biggest elephant in the room, Mr. Cost! #llm #finetuning #finance #computaioncosts

2

1 Kommentar

Apollo Research released a new mechanistic interpretability approach. They use the loss landscape to identify computationally relevant features and interactions. Then, they build a full interaction graph and interpret it. Theory: arxiv.org/abs/2405.10927 Experimental: arxiv.org/abs/2405.10928 In the theory paper, they modify singular learning theory to study the loss landscape for interpretability. They find 3 ways that parts of the network can end up being irrelevant to performance. Networks that generalize well are likely to have lots of this irrelevant structure! They introduce the Local Interaction Basis (LIB) as a way to remove this irrelevant structure and to find the most computationally relevant features. They also connect the loss landscape to modularity, deriving a formula for finding modules inside a network. In the second paper, they then empirically test LIB. LIB is a coordinate transform into a basis where interactions are as sparse as possible. Intuitively, it’s like a fancy PCA where we orthogonalize the features and their interactions at the same time. After identifying the LIB features, they use integrated gradients to build a graph of feature interactions. They use the derived modularity metric to search for modules in the graph. On small models, e.g. a modular addition transformer and a CIFAR MLP, LIB graphs are smaller, more sparsely activated and more interpretable than a PCA baseline, with identifiable modules that correspond to separate computation. Code 👉 https://lnkd.in/dksJWb-X #machinelearning

1

It can be used for non-sensitive use cases...

5

I'm excited to announce that the team at Qognitive just open sourced Fast Pauli https://lnkd.in/eTZd45_A, a Python/C++ library for optimized operations on Pauli matrices and Pauli strings. We've been using it at Qognitive to accelerate the training of our Pauli-based quantum models and wanted to share these performance improvements with the community. We're looking forward to improving this further and seeing what the community will build on top of it. For some operations we have seen 10-100x performance improvements over using Qiskit or Numpy for Pauli string multiplication operations against batches of state vectors. You can see some more benchmarks and information in our docs page https://lnkd.in/eZRJgXmy . Fast Pauli uses the work from https://lnkd.in/e5t8hbqU to accelerate the application of operators, which are tensor products of Pauli matrices, to a batch of vectors. The fast batched operation allows Qognitive to execute quantum calculations against large batches of data simultaneously - giving us a significant performance boost in training our Quantum Cognition ML models. We'll keep improving this so star the repo and look for more updates in the future, and please let us know how you use it. And please, don't miss out on Quantum cognition machine learning: financial forecasting - Risk.net [2409.12805] Robust estimation of the intrinsic dimension of data sets with quantum cognition machine learning (arxiv.org)

55

1 Kommentar

As Waymo enters an era of scaling they are making a big show of publishing safety data. But not all is as they want it to seem. Scaling, especially to serve new cities in addition to adding vehicles to the fleet, is where the risk of edge cases to safety is going to be the most challenging. Higher operational speeds (e.g., for airport service) additionally introduce a higher fatality risk to any crash they do have. The safety statements being made by their public and government relations apparatus are misleading. Maybe this will work out, and maybe it won't. But this data is in many ways just a distracting sideshow from the core safety question of how fatalities will turn out. Waymo hopes that will turn out well (and so do I), but it is irresponsible to practice what amounts to transparency theater and make misleading safety statements in the pursuit of fast scale-out. Details here in this week's substack posting:

67

10 Kommentare

Amazing findings by Hugh Zhang, Michele L., Summer Yue, and others at Scale AI, addressing a critical concern regarding Large Language Models (LLMs) evaluation and the challenge of data contamination! With LLMs achieving remarkable success on various benchmarks, concerns have arisen about the extent to which this performance reflects true reasoning ability versus exposure to similar benchmark questions. In response, the paper https://lnkd.in/gttAW-g2 undertook an investigation, introducing Grade School Math 1000 (GSM1k) to closely mirror established benchmarks like GSM8k. Their findings shed light on the prevalence of overfitting, particularly among models like Mistral and Phi, which exhibit accuracy drops of up to 13% on GSM1k compared to GSM8k. However, models at the forefront, such as Gemini, Claude, and GPT, demonstrate minimal signs of overfitting. This research not only underscores the importance of robust benchmarking but also prompts critical reflections on the evaluation of LLM performance. Kudos to the team for their work! #llm #research #genai #machinelearning #ai

14

5 Kommentare