Justin Cechmanek

Mistral AI has made major strides this year raising $640M at a $6B valuation and now launching its first multimodal model i.e. Pixtral 12B. And, their open-source approach, backed by investors like DST Global and Lightspeed, is fueling innovation and collaboration. Now with Pixtral 12B, Mistral is stepping into the multimodal realm, combining text, images, and other inputs to deliver more dynamic and versatile AI capabilities. This is a huge leap forward for industries that rely on deep understanding and interpretation of various data types, such as healthcare, media, and education. Mistral’s strategy of open-sourcing its models has been a game-changer, driving widespread adoption and trust. This approach, combined with its rapidly growing valuation, reflects the broader investor confidence in AI’s explosive potential. Clearly this AI space is evolving rapidly, with Mistral positioning itself as a serious contender alongside giants like OpenAI and Anthropic. It’s amazing to see European tech playing such a pivotal role in the AI revolution. ✌ https://lnkd.in/ej9Hi7pZ

12

The game of large language models (LLMs) got a major plot twist! Meta's unveiling of Llama, a family of AI models pushing the boundaries of parameter size and processing power, is a game-changer. Why it matters? Llama boasts a staggering 405 billion parameter model, dwarfing previous heavyweights like Google's LaMDA and OpenAI's Jurassic-1 Jumbo. This sheer size unlocks unprecedented capabilities in: -> From scientific papers to philosophical treatises, Llama can digest vast amounts of data, leading to deeper comprehension. -> Imagine writing styles that perfectly mimic different authors or create compelling narratives on the fly. -> From machine translation to chatbot interactions, Llama holds the potential to transform how humans and machines communicate. -> Llama isn't just about brawn. Meta stresses its commitment to open-source access and responsible AI development. This fosters collaboration and helps mitigate potential biases often associated with large datasets. Meta's ambitious project is a harbinger of the future, where AI can not only process information but truly understand it. Imagine personalized education tailored to individual learning styles, or AI assistants that anticipate your needs before you even ask. https://lnkd.in/dYn_cTjB #MetaLlama #ai #LLMs #futureoftechnology #naturallanguageprocessing #opensource

6

🎇 On National Day, I went for a leisurely drive in San Francisco and ended up "stress-testing" a Waymo self-driving car on the road. 🚗 As an autonomous driving practitioner, who wouldn't be curious about the real-time robustness of the cutting-edge Waymo One driving system? While cruising downtown, I accidentally noticed a Waymo car tailgating me. While this isn't unusual for SF citizens, a wild, "evil" idea suddenly hit me: Why not directly "adversarially attack" the world's autonomous driving status quo? I executed an unexpected maneuver—suddenly reversing—to see how it would react. 🌟 The response was stellar! At the moment I reversed, Waymo One honked instantly—quicker than any human could—activated its hazard lights, and backed away to maintain a safe distance. This reckless move on my part served as an edge case to test the algorithm's robustness under extreme conditions and, potentially, could be a challenging training sample to enhance Waymo's future autonomous systems. 💫 Thrilled to personally experience how current cutting-edge autonomous algorithms handle rare driving behaviors—and how stable and safe Level 4 autonomy is in dealing with diverse scenarios. However, it also prompted deep reflection as an AI researcher in this field: 🤔 In an industry with little room for error, how can we ultimately avoid or minimize issues that AI fails to handle? 💡 I believe two research directions are particularly promising for achieving Level 5 level autonomy in future mobility systems: - 1️⃣ Development and deployment of vehicle-to-everything (V2X) cooperative systems (including V2V, V2I, V2P, etc). Our initial studies (e.g., V2X-ViT, ECCV'2022 arxiv.org/abs/2203.10638) show that in scenarios with severe occlusions or noise, such cooperative systems can significantly enhance the robustness of perception systems, thereby eventually improving traffic safety. - 2️⃣ Adversarial scenarios generation (including Sim-to-Real, generative modeling). Research done by my colleagues at UCLA (V2XP-ASG, ICRA'2023 https://lnkd.in/gu5nKVHD) shows that adversarial learning techniques can effectively simulate adversarial scenarios, greatly improving model robustness in complex "corner case" situations. Of course, it's often infeasible to collect such collision scenarios. 👨‍🏫 As a new Assistant Professor of CS at Texas A&M University, I will lead a group focusing on these exciting research directions, which can be a proactive approach to reducing accidents and improving safety for all. 🔥 I look forward to future collaborations with governments, academics, and companies to research and develop data and algorithms that can help enhance the safety of vulnerable road users, especially seniors and children. We envision a people-centered intelligent transportation system in the future. Interested in these topics? Let's connect and discuss further! #AutonomousVehicles #AI #MachineLearning #SmartCities #Transporation #Humanity #Mobility

144

3 Comments

Notion vs Glean reminds me a lot of Tesla vs Waymo... Just like I think Tesla is going to solve the 'software' problem of autonomous vehicles before Waywmo solves the 'hardware' problem... I think that Notion is going to solve 'enterprise knowledge search' before Glean solves the problem of document and knowledge creation... Why? A couple reasons: 1) Just like Tesla, Notion has a huge advantage when it comes to the amount of underlying data and documents to train on... They already have 100M+ users, and likely 10x that in terms of the number of documents, files, etc. in team's workspaces. 2) It's very hard to change people's behaviour re: where/how they create 'work'. It's why a lot of companies have come and gone in that category... As exciting as Glean's AI writing assistants might seem, most people are just fine (or even prefer) a good old fashioned editor/writer. I could be way off here, but this will be an intersting one to follow. Thoughts? Where am I getting it wrong? #ai #enterprise #knowledge #management #notion #glean

3

1 Comment

What a time to be alive. Yes, LLMs are reaching initial plateaus. But then again, the underlying compute is getting 2-3x cheaper and more energy efficient every year. Our models today, are around 40.000x more compute efficient than comparable numerical models. Already today, we can build with 5M or 10M investment, what used to take 200M+ just in infrastructure before. In one year, we can build the same for 3M. In two years for 1.5M. In three years for 0.75M (or less). In other words, a global, high resolution, high precision world model, that would have taken billions to build just 5 years ago, will be built with a few million of investment in the next 5 years. This is our exponential journey towards climate sentience.

48

4 Comments

Generating JSON with LLMs is broadly applicable to many tasks. At Substrate, we believe it's the *glue* we need to do *less prompt engineering*, and *more software engineering*. JSON generation with LLMs makes data so much more malleable. Now we can easily extract data from unstructured sources, or even "shapeshift" structured data into any shape we want (Substrate code below). A lot of the new terms in AI engineering simply describe multi-step flows. And a lot of these flows can be reframed in terms of JSON generation. RAG? That's generating JSON for your vector DB query, searching, and then calling an LLM. (JSON generation can be useful on both the "Retrieval" and "Augmented Generation" sides). Function calling? Tool use? That's generating JSON for a function call, calling the function, and then calling an LLM. (Any multi-step LLM flow is a form of "Augmented Generation"). And there's a real benefit to reframing this way. JSON generation can improve reliability, and it's well established that LLM programs improve when multiple calls are chained together. You can also save on inference costs: (1) By using smaller models for each step. (2) By using smaller prompts (you can just define a schema, instead of coaxing with multi-shot examples) What's the catch? JSON generation and multi-step flows are known to be slow, and unreliable. That's where Substrate comes in. We've relentlessly optimized our JSON generation to make sure it's fast, and follows your schema with 100% accuracy. And Substrate's unique inference approach enables multi-step flows to run with maximum parallelism, and zero unnecessary data roundtrips. LLMs offer programmatic access to heuristic computation. Heuristic and symbolic computation are good for different things – and structured generation is the glue we need to combine them into a new kind of AI-integrated program.

24

1 Comment

🚨 Yikes! Apple Fined $38 Billion! 😱 NVIDIA Slides Down 📉 and CDK Takes a Tumble 📉 In this breaking news update, i cover the massive $38 billion fine imposed on Apple, the sharp decline in NVIDIA's stock, and CDK's unexpected tumble. Tune in to get all the details on these significant developments in the tech world and their potential impact on the market. #Apple #AppleFine #NVIDIA #NVIDIAStock #CDK #TechNews #StockMarket #BreakingNews #Finance #Technology #Investing #MarketUpdate

9

InstantDrag introduces a novel drag-based image editing framework that enhances both speed and interactivity by avoiding traditional optimization steps. Key insights: FlowGen: A GAN-based model that generates dense optical flow from sparse user inputs (drag instructions). It decouples motion generation from the actual image editing, allowing for fast processing. FlowDiffusion: A diffusion-based model that leverages the generated optical flow to perform high-quality, motion-conditioned edits. It eliminates the need for text prompts or masks, reducing computational overhead. Efficiency: InstantDrag achieves edits in ~1 second, offering up to 75x faster performance and consuming 5x less GPU memory compared to prior methods like DragGAN. No Inversion Needed: Unlike traditional methods requiring complex latent optimization and DDIM inversion, InstantDrag provides real-time, photo-realistic results by directly conditioning on motion dynamics. This approach is a step forward in interactive, real-time editing, making it suitable for a wide range of applications. Link to the paper : https://lnkd.in/g9FusJU7 #AI #ComputerVision #GAN #DiffusionModels #InteractiveEditing #OpticalFlow #InstantDrag

5

1 Comment

🚀 RAG systems can feel complex, but here’s a way to simplify: build your RAG solution using a pipeline as an API. This approach makes it: ✅ Streamlined: The entire process flows smoothly in one organized pipeline. ✅ Transparent: Unlike black-box APIs, you gain full visibility into data flow, understanding each step along the way. ✅ Customizable: Tailor each stage of the pipeline to fit your specific needs. By leveraging a pipeline-based API, you not only make RAG more accessible but also give yourself control over the process. Simplify, see more, and stay adaptable. For more insights, check out George Strong Ph.D post! #AI #LLM #RAG

1

After reading Andrej Karpathy’s post on X, I’m increasingly convinced that AGI will only be complete when it includes some form of physical systems. Integrating physical components is key to unlocking the full potential of AGI. Check out his thoughts: https://lnkd.in/eRDXsnZT

8

Over the past year, I've noticed a clear progression in AI capabilities: * LLMs (foundation models) * LLMs with RAG (Retrieval Augmented Generation) * LLMs with tooling * LLMs in robotics * Agentic robots in swarms While RAG and tooling are hot topics, there's a gap when it comes to AI-powered robots and swarms. Many are hesitant to embrace this potential, and even fewer understand how to implement it. That's why I'm excited to be working on Anuhubs Agent 2 Agent Protocol. We've made progress in testing and are actively seeking hardware partners to explore AI-powered swarm robotics. If you're interested, let's connect! #AI #robotics #swarmintelligence #Anuhubs

6

🚀An outstanding paper at #ICLR2024 on improving the shortcomings of #VisionTransforms - "Vision transformer s need registers". Transformers have recently emerged as a powerful tool for learning visual representations. In this paper, we identify and characterize artifacts in feature maps of both supervised and self-supervised ViT networks. The artifacts correspond to high-norm tokens appearing during inference primarily in low-informative background areas of images, that are repurposed for internal computations. We propose a simple yet effective solution based on providing additional tokens to the input sequence of the Vision Transformer to fill that role. 🔥Kudos to the team of Timothée Darcet Read more here - https://lnkd.in/d6z8nRHk #ViT #transformers #iclr #research #deeplearning #artificialintelligence

16

RAG of 2024 (ColbertV2 Reranking) !!! I've just published an in-depth guide on how to build an efficient local semantic search system using the innovative technologies of ColBERTv2, DSPy, and Qdrant . If you're interested in advancing your search capabilities with the latest AI technology, this article is for you. 🔍 Discover how the deep contextual understanding of ColBERTv2 and the dynamic optimization capabilities of DSPy can transform your data retrieval processes. Plus, learn about integrating Qdrant for scalable and precise search results. 📖 Check out the full article here: https://lnkd.in/gi_mQNH8 #SemanticSearch #AI #MachineLearning #DataScience #ColBERTv2 #DSPy #Qdrant #TechInnovation

20

2 Comments

AI at Meta 🤝 Roboflow: Open source computer vision New post on the Meta AI blog: how Segment Anything has saved over 74 𝐲𝐞𝐚𝐫𝐬 (and counting) for hundreds of thousands of users curating computer vision datasets on Roboflow. Amazingly, even this under estimates SAM's measured impact by an order of magnitude, i.e. all our open source work can be self hosted: using SAM in Workflows for chaining together models in inference, running SAM on edge devices, and with autodistill for local distillation. Link in comments ↓ #opensource #computervision #segmentanything #AI

278

13 Comments

With Llama 3.1, Meta takes another step towards becoming the “Windows of AI”. Meta releases their Q2 earnings this week, the first since they dropped the first private-grade open source foundational model , a huge step towards this strategy. Investors will be asking what Meta's long-term vision for AI is, and evaluating how likely Meta is to eventually "cash in" on their heavy investments to produce foundational models. Some early thoughts on how to think about Meta's AI strategy:

13

And just like that, Waymo announced they have an end-to-end model for autonomous driving. Former CEO John Krafcik states: 1. there is long-term promise for an e2e approach (e.g., interpretable decision rationale is possible), but 2. big challenges remain (insufficient long-term memory to solve complex long-tail challenges, sensor fusion), so 3. it's going to be super hard for e2e-only companies (Wayve, Waabi, Tesla) to catch Waymo, which has the only safely working AI approach in the space & a huge on-going advantage: Alphabet AI adjacency/talent

120

11 Comments

This is #icra2024 Week, and I want to tell you about a very cool paper I spotted from Waymo's research team: Stateful Tracking with Transformers for Autonomous Driving! It's a 3D Multi-Object Tracker... but with Transformers rather than Kalman Filters. It's very "edgy", and I'm doing a complete breakdown in this video. Dragomir Anguelov (Drago) Qinru Li How did I do? Did I miss anything? #deeplearning #computervision #lidar #selfdrivingcars #robotics

119

6 Comments

🌭🌭HOP: Hand-Object interaction Pretraining🌭🌭 👉Berkeley unveils a novel approach to learn general robot manipulation priors from 3D hand-object interaction trajectories. Using in-the-wild videos to generate sensorimotor robot trajectories. 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: ✅General manipulation prior from in-the-wild ✅Simulator as videos-sensorimotor intermediary ✅Fine-tuning via RL and behavior cloning (BC) ✅Sample-efficient adaptation to downstream tasks #artificialintelligence #machinelearning #ml #AI #deeplearning #computervision #AIwithPapers #metaverse 👉Discussion https://lnkd.in/dMgakzWm 👉Paper https://lnkd.in/dhRFQara 👉Project https://lnkd.in/dCa77u7P

168

3 Comments