Edoardo Anselmi

💎 𝗔 𝗖𝗼𝗺𝗽𝗿𝗲𝗵𝗲𝗻𝘀𝗶𝘃𝗲 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗼𝗳 𝗤𝘂𝗮𝗻𝘁𝗶𝘇𝗲𝗱 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻-𝗧𝘂𝗻𝗲𝗱 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀: 𝗔𝗻 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 𝘂𝗽 𝘁𝗼 405𝗕 1. 𝗕𝗶𝗴𝗴𝗲𝗿 𝗶𝘀 𝗯𝗲𝘁𝘁𝗲𝗿: Quantizing a larger LLM to the same size as a smaller one often outperforms the smaller model. 2. 𝗧𝗮𝘀𝗸 𝗱𝗶𝗳𝗳𝗶𝗰𝘂𝗹𝘁𝘆 𝗱𝗼𝗲𝘀𝗻'𝘁 𝗶𝗺𝗽𝗮𝗰𝘁 𝗮𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝗹𝗼𝘀𝘀: Quantization doesn't significantly affect performance across different task types. 3. 𝗠𝗧-𝗕𝗲𝗻𝗰𝗵 𝗻𝗲𝗲𝗱𝘀 𝗿𝗲𝗳𝗶𝗻𝗲𝗺𝗲𝗻𝘁: The MT-Bench evaluation method struggles to differentiate between top-performing LLMs. 👉 Paper https://lnkd.in/damF49sd ♻️𝒌𝒊𝒏𝒅𝒍𝒚 𝒓𝒆𝒑𝒐𝒔𝒕♻️

42

2 commenti

Hi Andrew Feldman - a few questions on these Llama3.1-70b performance numbers, given that the WSE-3 has no HBM and all parameters need to be held in on-chip SRAM. > Based on what was presented at Hot Chips '2024 in August, you need 4x CS-3 racks, each with a WSE-3 to run this model. Is this an accurate assumption? > With 4x CS-3 racks , each WSE-3 having 44G of on-chip SRAM and running the model in FP16, one needs 140GB for parameters. With a total of 44.*4 (176GB) of SRAM; that leaves us with 176-140 = 36GB of memory for user-KVCache, assuming that none of the on-chip SRAM is needed for other things like holding program data. Is this an accurate assessment? > Given the 568 T/s for Llama3.1-70b and at a context of 8K, one needs 2.44GB (FP16) per user for KVCache. Is it correct to assume that there can be a maximum of 36/2.44 = ~14 concurrent users on the 4-CS3 rack solution or is it actually less concurrent users with current system capabilities for these T/s numbers? > With these 4x CS-3 Racks, at 34kW for each rack, would this mean a 136kW solution for these T/s results? > What are the implications when context length goes up to 128K and results in significantly increased KVCaches? > How many racks are needed for a 405b model?

46

1 commento

👉 OpenAI released GPT-4o, which reports significant improvements in latency and cost. ✅ Many users may wonder how to evaluate the effects of upgrading their app to GPT-4o? For example, what latency benefit will users expect to gain and are there any material differences in app performance when I switch to the new GPT-4o model. ✅ Decisions like this are often limited by quality evaluations! The process of evaluating GPT-4o is shown on an example #RAG app with a 20 question eval set related to LangChain documentation. ♻️𝒌𝒊𝒏𝒅𝒍𝒚 𝒓𝒆𝒑𝒐𝒔𝒕♻️ #ai #gpt #genai #llms LangChain

6

Let us now discuss low-level technical questions that help to optimise the consumption of valuable resources such as memory and its access bandwidth, using the mainstream version of one of the highest-level languages, Python, as an example of how a careful selection of data types as well as their judicious organisation can bring concrete benefits in the execution of applications written in this language. English: https://lnkd.in/eRAGWtQ2 Italian: https://lnkd.in/eJ9YM2gh #python #cpython #optimization

4

What if you could still benefit from model based design w/o compromising on code quality and modern development workflows 😉. Then look no further and check out #ascet. https://lnkd.in/dHyBSM5z ASCET model semantics are saved in ESDL which is easy to understand text based format and not cryptic xml or non readable binary formats 📑. Its closer to code by design using familiar software notations👩‍💻 Push code to git, review, diff and merge. See here a live example on github 👍 https://lnkd.in/ewg-uDTG Built in static analysis find errors at editing time 🔍 You can still  ✔️ Prototype your ideas fast using simulation. See https://lnkd.in/eC6CX5Jf ✔️ Do test driven development using built in unit tests. See example here https://lnkd.in/egdHUvau ✔️Auto generate high performance code which is MISRA compatible. See example here https://lnkd.in/ePWJEnb8 ✔️ Integrate into your automation pipelines ✔️ Generate software documentation Give it a try  https://lnkd.in/dZKvbMRv #embedded #modelbaseddesign

20

⚙𝗪𝗵𝗲𝗻 𝗱𝗼𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 #𝗔𝗗𝗔𝗦/𝗔𝗗 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻𝘀, 𝘄𝗵𝗮𝘁 𝗶𝘀 𝘆𝗼𝘂𝗿 𝗽𝗮𝗶𝗻?🚧 ❓Debugging and setting-up your model, or keeping track of relevant data while simulating thousands of cases?❓ We do understand the different needs and developed separated approaches. ➡ With our “live” simulation tab you will be able to debug your models simulating one case at a time: ✅ Easily monitor any relevant indicator ✅ Vary live, while simulating, any value ✅ Check the debug files ✅ All at the same time in one layout! On the other side, our “Runs” tab is designed for 𝗵𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝘀𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗼𝗻𝘀 and keeping track of its results, by providing: ✅ A run history on all executed simulations based on test parameters and KPIs/safety metrics ✅ The usage of filters and cluster methods of failed tests for efficient detail analysis of individual tests ✅ A 3D playback of recorded #ASAM OSI® data as well as timeseries data 𝙍𝙚𝙖𝙙𝙮 𝙩𝙤 𝙚𝙣𝙝𝙖𝙣𝙘𝙚 𝙮𝙤𝙪𝙧 𝘼𝘿𝘼𝙎/𝘼𝘿 𝙟𝙤𝙪𝙧𝙣𝙚𝙮? 🔎 Dive into AVL Scenario Simulator™ today and 𝘂𝘀𝗲 𝗶𝘁 𝗳𝗼𝗿 𝗳𝗿𝗲𝗲! 👀 https://lnkd.in/dKfvGwbW #ADAS #AD #AutomotiveTesting #Simulation #AutomatedDriving #ASAM #OpenSCENARIO #SoftwareTesting #AutonomousDriving #AutonomousVehicles

21

1 commento

I have watched most, if not all, of the presentations and talks given by engineering representatives of car-makers at the MathWorks annual Automotive Conferences over the last few years. These presentations focus on vehicle control design using the Model-Based Design (MBD) approach with #Matlab and #Simulink. Although I am not an automotive engineer, Control Theory has been my hobby for over 10 years, of which I have self-studied the topic. I find the conference presentations fascinating and educational. Recently, during the 2024 MathWorks Automotive Conference held a week or two ago, Juergen Bortolazzi gave a ~30-minute talk on how engineers at Porsche AG are tackling challenges in designing intelligent vehicle control systems, including assisted and automated driving systems. Towards the end of his talk, Juergen briefly mentioned how their team is integrating AI into their designs. Furthermore, in a recent release (R2024a), MathWorks has introduced a PyTorch Simulink block, in addition to various Matlab AI Toolboxes available to engineers. This update allows engineers to harness the state-of-the-art AI and Deep Learning capabilities of PyTorch directly within Simulink for their designs. You can find more details in [1] in the comments section. For technologists who are interested in automotive intelligent systems, the proceedings of the MathWorks Automotive Conference 2024 Europe are accessible through link [2]. I believe Juergen's presentation, would be of interest, Philip Koopman, Missy Cummings, Dr. Jeffrey Funk and Mikell Taylor. _____ Excerpts: [Since the introduction of Park Distance Control and adaptive cruise control in the mid-2000s, Porsche has integrated driver assistance and automated driving technologies into its product lines. This does not contradict the philosophy of a sports car—customers who like to drive themselves in appropriate traffic conditions expect driving to be made significantly easier in stressful, time-consuming situations such as traffic jams or busy parking spaces. Although the general discussion focuses on the higher levels of automation from SAE Level 3 to Level 4, Level 1 and 2 systems will play a significant role for the next decade, at least, and represent the technological state of the art for most vehicles. In this environment, Porsche is focusing on increasing the performance and functionality of Level 1 and 2 driver assistance systems while developing programs that enable automated driving at Level 3 and 4. This offers the opportunity to build the necessary expertise in sensor technology, sensor fusion, planning and control, as well as the processes, methods, and tools required for the development, approval, and release of higher-level automated driving systems.]

16

2 commenti

When the calibration team is physically separated from the software team, be it by testing away from the office or work location, Ehandbook helps build understanding and the troubleshooting process by providing great interactive documentation of how the software works. Learn more from Christoph Nißle's post below https://lnkd.in/g6m4ARXZ

6

Are autonomous vehicles safe? A big lack of transparency - Let's meet at OSS5 Transparency in Automated Driving has always been one of the most diffifult things. Companies do have problems explaining how to achieve safety, how to show safety to authorities, third parties, society and how to show that correct tests are done in order to show that residual risk is sufficiently low. An usual approach is the definition and testing of scenarios. But how many scenarios have to be tested? This might be an infite number. Especially for the reproducability of scenarios themselves. What are the right KPIs? + And are you sure, that you proof safety for the future when you've only considered the past? One new rare scenario can change everything. See Cruise. In Industries like Aerospace and Defence it is already clear for years, that System Safety Engineering is not only a valuable approach to manage complexity, it is also a valuable approach to explain safety to authorities and is near to completeness. At SEADO we have developed a Systems Safety Engineering Approach, which is not only compliant to Safety Standards such as ISO 26262 and ISO 21448, it is also compliant to Systems Engineering Standards such as 15288 and Process Evaluation Models like Automotive SPICE. This approach has led to several valuable discussions on safety within different companies and helped to not only to find the right paths for certification with third parties, understanding with other Non-AD Departments, it helped making safety concepts complete and robust. All in all, made fast and transparent progress in a short amount of time. If you want to learn more or want us to contribute to your programs, Do not hesitate to discuss via LinkedIn or find me at OSS5 in Berlin.

21

Cloud-Native is on the way

1

Elektrek reports: "Tesla releases update to remove steering wheel nag, shuts down sunglasses loophole," with improved camera-based #driver #monitoring. We need to call this update for what it is. Tesla's improvements to its #DMS amount to no more than polishing a turd. Elektrek continues: "This enhancement is available on vehicles equipped with a #cabincamera and only when the cabin camera has clear and continuous visibility of the driver’s eyes (e.g. the camera is not occluded, there is sufficient cabin illumination, and the driver is looking forward at the road ahead and not wearing sunglasses, a hat with a low brim, or other objects covering the eyes). A DMS needs to work at all times and under all lighting conditions to effectively perform the supervisory role for #FSD and #Autopilot. To not work in low light, or simply if the driver is wearing sunglasses, means it works unless it doesn't. That amounts to a hardware defect which cannot be resolved with changes to the software. This is the game Tesla is currently playing with #NHTSA. Coming regulatory changes mean Tesla is out of time for its woeful DMS. Tesla DMS came last in a recent #IIHS #automation #safeguards ranking (Ford Motor Company's DMS was ranked first) but coming harmonization of regulatory requirements across the U.S., Europe, and China through the 5-star #NCAP test will force Tesla's hand regarding properly functioning DMS in the near term. At a minimum, Tesla needs to implement DMS with 940nm optical path, add “Owl & Lizard” movement detection, work properly with sunglasses, and provide measures of #distraction and #drowsiness. Euro NCAP - For Safer Cars, Vans & Trucks is set to announce its 2026 DMS test protocols later this year and could add metrics for #cognitivedistraction and #impaired driving. Ironically it is Tesla FSD/Autopilot crashes that provide regulators with the data of what functions a DMS performing a supervisory role for partial automation most need to fulfil. NHTSA issued an #ANPRM for impaired driving in November. It is likely to follow that up with clear rulemaking for minimum DMS performance requirements in the next 12 months. The industry should not expect those requirement to be lame. FMVSS 127 for AEB was a major #safety advance. #Automakers should prepare now for the same step-change for DMS in the U.S. too. The latest edition of Semicast's report "#ADAS & #Automotive Vision-based Driver Monitoring Systems" is published. Significant segmentation of the DMS/OMS market is provided. This is summarized as follows:   - by movement detection into "Owl-only" and "Owl & Lizard" detection - by feature set - by processor type - by CMOS image sensor location - by CMOS image sensor resolution - by driver and occupant monitoring - by front-seat and whole-cabin monitoring   Message me for a copy of the sales brochure for the study. My inbox is open. https://lnkd.in/eWNvfQSm

72

8 commenti

The BC ADIT Robotaxi Report plus recent news from Waymo indicates that total robotaxi mileage thus far is 60M miles or more. Here's a run-down on activity in robotaxis, roboshuttles, and autonomous buses worldwide. Plus.... ruminations from Moi on the early days of autonomy and how it all might have come to nothing. #driverlesscars #robotaxi #roboshuttle

24

A little not known fact, a CPU, here an intel 56 cores is capable of 12 TERABYTES per second cumulated across the cores bandwidth, and it is matching easily the bandwidth of E.Us in GPUs. The hard part is to make sure you get a 99% L1 cache hit, then, you easily match GPU in performance. With the neurones becoming more complex, you can expect this to become an advantage.

50

9 commenti

🚀 Don't miss the replay of this talk on optimizing AI for automotive ADAS and autonomous driving by Shouwei Chen, Core Maintainer & Product Manager of Open-Source Alluxio from Alluxio. 🔥 Stay tuned in the comment section for the replay link! 🔍 Short actionable insights: ✅ Overcome AI infrastructure challenges in autonomous driving. ✅ Achieve millisecond latency and high throughput with Alachio's high-performance data access. ✅ Easily manage data silos and boost GPU utilization for better efficiency. ✅ Reduce costs by 30% and eliminate data copy between object store and NAS. ⚡️ Summarizing key points: - The evolution of autonomous driving technology from hardware enhancement to large-scale data processing. - Challenges in AI infrastructure like low performance, GPU scarcity, and cost management. - Introducing Alluxio as a data platform with high-performance data access and unified global view. - How Alluxio optimizes AI training by resolving IO bottlenecks and improving GPU utilization. - Real use cases showcasing significant time and cost savings. 🤔 Have you faced challenges with AI infrastructure for autonomous driving? How did you overcome them? Share your experiences! #AI #autonomousdriving #datamanagement #Alluxio #AIUserGroup #AIForDevelopers

4

4 commenti

The integration of #AI in #searchengine is becoming inevitable. AI is transforming how search engines operate by enhancing their ability to understand and predict #userIntent, making searches more accurate and personalized. For instance, #AIalgorithm like #Google's #BERT and #MUM are designed to comprehend the context behind #searchqueries, moving beyond simple keyword matching to deliver more relevant results. This shift means that search engines can now provide answers that are more aligned with what users are actually looking for, even if their queries are vague or complex. Moreover, AI enables search engines to offer #personalizedeXperiences by analyzing user data such as search history and location. This personalization helps in delivering more relevant #contentdelivery and #recommendations, enhancing the overall #userexperience / #UX. What are your thoughts on this penetration of AI? #hyperpersonalisation Source: Conversation with Copilot, 15/09/2024 with Copilot reference. https://lnkd.in/gP3QFxci (Credits and reference)

3

2 commenti