Abhijeet Mahagaonkar

A Marriage Made in Heaven - LLM's and DADA-X by Decision-Zone The integration of task-oriented Large Language Models (LLMs) with the dynamic capabilities of DADA-X is indeed a transformative development in the realm of distributed systems management. These advanced LLMs, with their profound ability to simulate intricate Operational Technology (OT) systems, are revolutionizing the way mission-critical processes are conceived and executed. When these processes are fed into DADA-X, they are ingeniously converted into causal event patterns, which act as the backbone for autonomous applications operating in the real world. This convergence is unlocking unprecedented levels of real-time control and management. Organizations can deploy these sophisticated event patterns within DADA-X, enabling them to fully exploit the capabilities of their distributed systems. The result is a paradigm shift in operational efficiency, with DADA-X autonomously orchestrating complex missions, optimizing performance, and proactively mitigating risks. The emergence of the Artificial General Intelligence (AGI) market is a direct consequence of the synergy between LLMs and DADA-X. This market is poised to offer organizations the ability to construct intelligent systems that not only comprehend but also autonomously execute complex tasks with remarkable precision and efficiency. The potential here is immense, with implications across various sectors and industries. As we move forward, the traditional approaches of manual intervention and centralized control are becoming obsolete. The combination of LLMs and DADAX equips organizations to confidently step into the future of distributed systems management. This transition heralds the end of inefficiencies and the beginning of an era characterized by autonomy, efficiency, and reliability. The opportunity presented by this technological evolution is significant. Organizations that act swiftly can transform their distributed systems into agile, intelligent networks capable of managing the complexities of contemporary operations. The power of LLMs and DADA-X is not just theoretical but is a tangible force driving the AGI market forward. To be at the forefront of this revolution is to be a pioneer in the AGI market. It's an invitation for organizations to empower themselves with the necessary tools and technologies to succeed in an increasingly intricate and dynamic landscape. The future of distributed systems management has arrived, and it is powered by the innovative fusion of LLMs and DADA-X. The call to join this revolution is clear, and it promises a journey towards a smarter, more autonomous world.

5

𝐇𝐮𝐦𝐚𝐧𝐬 𝐢𝐧 𝐀𝐈 𝐢𝐬 𝐁𝐚𝐜𝐤! 🎉 Welcome back to Humans in AI. This week, we are joined by Dheeraj Pandey, a successful serial entrepreneur, co-founder and CEO of DevRev and former co-founder and CEO of Nutanix. DevRev is a Mayfield Fund portfolio company building #AITeammates for Product & Support teams. In this video, Dheeraj Pandey will share his invaluable insights on: - AI Teammates and how they benefit humans. - Advice for fellow founders, drawing from his own impressive experience. Who would YOU like to see next? Let us know in the comments! 👇 #HumansInAI #AI #AITeammates #GenAI #DevRev

173

8 Comments

There is an incoming wave of Layer 2 (L2) cloud computing platforms that will disrupt the hyperscalers and improve data processing across most industries. AWS stands as one of the most successful businesses in history, generating $100 billion in revenue and continuing to grow at 17% YoY. It makes up approximately 60% of Amazon’s profit! Similar success stories for Azure and GCP, but virtually no competition otherwise. They offer raw building blocks (e.g EC2, S3, Batch), which makes their business extremely repeatable but offloads most of the platform-building work—the "glue"—to each company. Many of the cloud platforms built by companies in the same vertical end up being remarkably similar, effectively reinventing the wheel. As the cloud spending within certain verticals becomes large enough, the market can support an L2 cloud that can harness those patterns to build a single customized platform for that industry. For example: LLM Inference → TogetherAI Self-Driving → Applied Intuition Video AI → Sieve Data Biopharma R&D → LatchBio As resources are pooled, it becomes clear that a) a central industry-tailored cloud is higher quality than what any one company could have created, and b) is also much cheaper ($ and time) than the sum of what each company was spending! The saved resources can be returned to the companies so they can invest it in other areas that are actually differentiated. Meanwhile AWS continues to receive its portion (albeit with a stronger counter-negotiator at the table), and the L2 cloud platforms get a share of the savings they create. Win-Win-Win. I would be surprised if, ten years from now, the majority of companies using the cloud are still interfacing directly with AWS rather than through specialized L2 providers. AWS will of course continue to try, and mostly fail, to meet the nuanced demands of every single vertical—maintaining 15%+ annual growth at their scale is already remarkable. The L1 cloud market is still expanding rapidly and competition is arising (e.g Oracle, NVIDIA, Lambda, CoreWeave). This new generation of L2 clouds will be able to provide much more elegant and intuitive experiences for the end user, much like how Motorola’s 68000 processor gave way to Apple's Macintosh and how Oracle's relational databases led to Salesforce's CRM. By recognizing and organizing common patterns across industry cloud platforms, the market will build more efficient, user-friendly L2 clouds that will benefit everyone involved.

35

https://lnkd.in/gZvWp8i8 A foreign investor started a bidding war to control Nvidia’s Bay Area headquarters complex when it made an unsolicited bid to purchase the property. Nvidia dropped a nine-figure sum to end it. Nvidia drops $374M to become its own landlord in Santa Clara. #nvidia #artificialintelligence #llms #generativeai #gpu #servers #datacenter #cloud #ai #Semiconductorindustry #electronics #semiconductor #technology #innovation #business credit: sf standard

8

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

One of the key capabilities that sets Memgraph apart from the competition is our deep path traversal algorithms. Come read our latest blog explaining all about it: https://lnkd.in/gBtWPJuZ

27

2 Comments

I am incredibly excited to announce Arch gateway https://lnkd.in/ePAf-VK8, an open source project for developers to build genai apps faster. It is engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting jailbreak attempts, intelligently calling “backend” APIs to fulfill the user’s request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability of prompts and LLM interactions in a centralized way - all outside business logic. Here are some additional key details of the project, * Built on top of Envoy and is written in rust. It runs alongside application servers, and uses Envoy's proven HTTP management and scalability features to handle traffic related to prompts and LLMs. * Function calling for fast agentic and RAG apps. Engineered with purpose-built fast LLMs to handle fast, cost-effective, and accurate prompt-based tasks like function/API calling, and parameter extraction from prompts. * Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code. * Manages LLM calls, offering smart retries, automatic cutover, and resilient upstream connections for continuous availability. * Uses the W3C Trace Context standard to enable complete request tracing across applications, ensuring compatibility with observability tools, and provides metrics to monitor latency, token usage, and error rates, helping optimize AI application performance. This is our first release, and would love to build alongside the community. We are just getting started on reinventing what we could do at the networking layer for prompts. Do check it out on GitHub at https://lnkd.in/ePAf-VK8 Please leave a comment or feedback here or in GH and I will be happy to answer. #gateway #envoy #rust #genai #developers #ai cc Salman Paracha | Muhammad Ikramul Haq | Syed Anas Hashmi | Fahim ul Haq | Farheen Omar | Umar Farooq | Arslan Ahmad | Abdul Rafay Abbasi |

58

9 Comments

💥💥💥 $100K or 100 Days: Trade-offs when Pre-Training with Academic Resources Apoorv Khandelwal, Tian Yun, Nihal V. Nayak, Jack Merullo, Stephen H. Bach, Chen Sun, Ellie Pavlick Abstract Pre-training is notoriously compute-intensive and academic researchers are notoriously under-resourced. It is, therefore, commonly assumed that academics can't pre-train models. In this paper, we seek to clarify this assumption. We first survey academic researchers to learn about their available compute and then empirically measure the time to replicate models on such resources. We introduce a benchmark to measure the time to pre-train models on given GPUs and also identify ideal settings for maximizing training speed. We run our benchmark on a range of models and academic GPUs, spending 2,000 GPU-hours on our experiments. Our results reveal a brighter picture for academic pre-training: for example, although Pythia-1B was originally trained on 64 GPUs for 3 days, we find it is also possible to replicate this model (with the same hyper-parameters) in 3x fewer GPU-days: i.e. on 4 GPUs in 18 days. We conclude with a cost-benefit analysis to help clarify the trade-offs between price and pre-training time. We believe our benchmark will help academic researchers conduct experiments that require training larger models on more data. 👉 https://lnkd.in/d5Vh3ycg #machinelearning

1

✈️ Wrapping up at AWS Re:invent today. Spent half the day yesterday walking the exhibit floor. The main purpose was to observe traffic hot spots and booth placement on the floor, to get at sense of which areas folks are interested in, and how important of a partner these companies were to AWS. Here were my high-level observations: -- startups that had the most traffic/attention: Anthropic, Unstructured, some of the code gen players like Poolside, Codieum, etc. Glean’s booth had a massive amount of traffic. Poolside had a massive booth relative to their company stage (maybe owing to the amount of compute they spend with AWS). The Anthropic booth was very nice and centrally placed on the floor. They seem increasingly tied to the hip with Amazon. -- startups that weren’t getting a lot of traffic: MLOps, RAG-as-a-service, vector DBs, “build you own agent” platforms. Not sure if this reflects overall market interest or simply the attendee profile at this conference. Also could be that their booth positioning wasn’t that great – many of these startups may not be major AWS customers or could even compete with AWS in certain areas. -- overall, I felt there was still a massive discrepancy in crowd size and interest between incumbents and startups. One reason might be that most people who come to Re:invent come from larger enterprises and are either less familiar with startups or don’t align well with startup’s ICPs. This was noticeably different from other AI conferences I’ve been to such as Databricks + ai summit or NeurIPS. In terms of specific things from AWS: -- Bedrock got a lot of new upgrades including multimodal processing, graphRAG, and structured data retrieval. Also, announced some features around multi-agent orchestration, distillation, and prompt caching. It appears that they’re increasingly consolidating the middle layer. -- They’re also pushing Bedrock Guardrails a lot as it was mentioned during the keynote and by separately by folks I spoke with.  For those unfamiliar, the Guardrail platform does a few things such as using automated reasoning to check against factual errors and hallucination, and filtering toxicity and PII. As these things get rolled out into production, I see trust and safety becoming increasingly important so I see why they’re pushing this. -- They also announced some cool things around RPA with Amazon Q. They showed a demo where Q Business was able to create an automated workflow simply by watching a video recording of how it was done. Under the hood, this is probably leveraging Computer Use recently announced by Anthropic and incorporated into Claude 3.5. My overall feeling was that the middle layer is increasingly going to be owned by the CSPs like Amazon as they continuing to push up the stack. I expect acquisitions to happen in the next few years in this layer, but perhaps not at the valuations that investors/founders were originally hoping for.

105

14 Comments

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

Test time training of LLM. For easier problems providing context along with the prompt is enough. For harder problems like math, coding, law etc. taking a sample of similar problems and training few batches on it improves accuracy many X times (3-14 times or so). https://lnkd.in/gwp97RFr

1

Waymo is the best tech product I tried in years, and I can't wait for humans to be banned from driving. What's Waymo? Waymo has been at the forefront of autonomous vehicle development since its inception as the Google Self-Driving Car Project in 2009. The core of Waymo's technology lies in its sophisticated combination of sensors, including LiDAR, cameras, and radar, integrated with artificial intelligence to perceive the environment accurately. This technology allows the vehicle to make quick decisions in real-time, such as identifying pedestrians, bicyclists, and other vehicles, understanding traffic signals, and adapting to unexpected situations like road work or detours. Over the years, Waymo has expanded its testing grounds from a single city in California to multiple locations across the United States, including San Francisco, Phoenix, and Detroit. In addition to testing, Waymo has launched pilot programs like Waymo One (The service I tried), a self-driving taxi service in San Francisco, Phoenix, and Los Angeles. You can use the app to request a ride in a self-driving vehicle. This service marks a significant milestone in commercializing autonomous technology, offering a glimpse into the potential future of urban transport. Additionally, Waymo Via focuses on goods delivery, utilizing the same technology to transport products efficiently and safely, which could revolutionize supply chain logistics by reducing costs and improving delivery times. I am grateful to Amjad for the access to Waymo One. Which city in the Middle East will be the first to welcome Waymo? I would say Dubai, what about you? #selfdriving #waymo

22

9 Comments

The future of productivity with GenAI is here—and it’s task-specific and domain-specific models that can run on the edge. We believe the next wave of innovation lies in models optimized for flexibility, control, and precision. That’s why we’re thrilled to welcome Fastino —emerging from stealth last week—to the Insight family! Fastino is building a family of task-optimized language models and tools that redefine enterprise flexibility and control. Their models deliver industry-leading accuracy for tasks like RAG-powered question answering, text structuring, summarization, and generating smart replies—running up to 1000x faster than generalist models. Best of all, they operate with minimal compute requirements and deploy seamlessly on edge devices or private clouds. With intuitive developer tools for fine-tuning and validation, Fastino empowers enterprises to achieve consistent performance, predictable costs, and unmatched control over their language models. 💡 Curious to learn more? Check out their blog post here: https://lnkd.in/gTTe3q49 #GenAI #LLMs #GenerativeAI #SLMs #EdgeComputing #AI #ML #VentureCapital

16

1 Comment

One of the things I ❤️ about the KX community is the breadth and depth of its ecosystem.  Here are just some of the partners we collaborate closely with to solve the largest data and high-performance computing problems on behalf of our mutual customers. ⚙ 𝐒𝐞𝐫𝐯𝐞𝐫𝐬 & 𝐒𝐭𝐨𝐫𝐚𝐠𝐞: KX is the go-to database for the most latency-sensitive applications. But what servers and storage should you run it on? We’ve partnered with several of the leading hardware vendors to optimize the performance of KDB on their platforms. Audited benchmarks and reference configurations are available for each of the following: ◾ Dell Technologies: PowerEdge servers + PowerScale & PowerFlex storage ◾ Lenovo: ThinkSystem ◾ Pure Storage: Flashblade ◾ DDN Storage: AI400X2 ◾ WEKA: WekaFS and WekaIO Matrix 🤝 𝐂𝐏𝐔𝐬 𝐚𝐧𝐝 𝐆𝐏𝐔𝐬: our engineering partnerships with the chip giants are rapidly expanding beyond CPU and GPU optimization to include integrations with their broader AI hardware and software layers. ◾ NVIDIA: L40S, H100, Grace Hopper, RAPIDS, NeMo, NIM ◾ Intel Corporation: Xeon, Sapphire Rapids, Optane ☁ 𝐇𝐲𝐩𝐞𝐫𝐬𝐜𝐚𝐥𝐞𝐫𝐬: naturally, KX runs on the platform of your choice. But did you also know that we’ve embedded KDB deeper in the cloud stack for higher performance and more native offerings? ◾ Microsoft: KDB Insights on Azure - a fully managed Azure-native service ◾ Amazon Web Services (AWS): Amazon Finspace with Managed KDB Insights – a fully managed AWS-native service. 🏚 𝐃𝐚𝐭𝐚 𝐖𝐚𝐫𝐞𝐡𝐨𝐮𝐬𝐞 𝐚𝐧𝐝 𝐋𝐚𝐤𝐞𝐡𝐨𝐮𝐬𝐞: want to leverage the scalability of Snowflake or Databricks but need to accelerate those AI and analytics workloads? ◾ Databricks: run PyKX directly from the PySpark console.   ◾ Snowflake: run PyKX directly in Snowpark. 💡 𝐕𝐢𝐬𝐮𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧: several years ago we made the decision to add ANSI SQL support in KDB. This has allowed customers to integrate their favourite front end tools and the response has been phenomenal. Open-source integrations are available for several of the top products including: ◾ Tableau ◾ Power BI ◾ Grafana Labs 📊 𝐃𝐚𝐭𝐚: One of the hardest parts of managing high frequency datasets is doing the integration between data feed and database.  We’ve partnered with the following vendors to take all of the pain out of this part of the job. ◾ ICE ◾ Refinitiv ◾ Bloomberg ◾ BMLL ◾ Options Technology ◾ INQDATA 🤖 𝐀𝐈: we are undoubtedly in the early innings of a paradigm shift with respect to Generative AI. We’ve integrated with the following AI platforms and frameworks so that you can easily bring today’s state-of-the-art models to your data. ◾ LlamaIndex ◾ LangChain ◾ Hugging Face 🛠️ 𝐒𝐞𝐫𝐯𝐢𝐜𝐞𝐬: Our services partners deliver enormous value to customers with their combination of technical and domain expertise day in and day out. ◾ First Derivative ◾ Data Intellect ◾ Caspian One ◾ ExeQutionAnalytics ◾ Habla computing ❤️ #community #ecosystem #innovation

120

Linus Torvalds: 90% of AI marketing is hype Linux kernel creator says let's see which workloads use GenAI in five years Linus Torvalds, creator of the Linux kernel, thinks the majority of marketing circulated by the industry on Generative AI is simply fluff with no real substance - and it may take many years before the tech is proven. “I think the whole tech industry around AI is in a very bad position and it’s 90 percent marketing and ten percent reality and in five years things will change and at that point we’ll see what of the AI is getting used for real workloads.” Said: Linus Torvalds "With around $200 billion in capex, only about $20 billion of revenue is actually coming from consumers and businesses in terms of AI services, things like Copilot licenses and ChatGPT licenses, so a very poor return in true results in terms of end users. And the whole bet [whether] the AI explosion continues or not will depend on whether they can get that $20 billion up as quickly as they hope." He added. Link: https://lnkd.in/duXNmuV7

2

Inference is what makes outcomes not training , the cost of tokens needs to get down by 1000X or more to be beneficial for humanity . If AI needs to be like internet we need bring down the cost of chips by 100X and having xxx’s of concurrent users . Android is a great example - OG Steve J possibly won’t like it possibly .. The future is AI and AI for all. Technology needs to be in the hands of all humanity , this is biggest 0-1 outcome . We need to think of breaking the CUDA wall and it is not going be be singular objective but we need to make it happen. #thinkbeyondcuda #breakCUDAwall

4

2 Comments

Human in the loop evals 🧑‍💻👀 → custom, automated evaluation models 🤖📈? https://lnkd.in/gruPtC8d Human in the loop (HITL) is a really popular technique for teams just getting started with LLMs and their LLMOps functions. Given your biggest concern is your model acting ‘not like a human’, it’s easy to see how sticking a human in the loop, to review and edit any potential mistakes or weirdness, can solve that problem. But it’s equally easy to see how challenging this can be to scale, and how it can eat at a lot of the cost-savings promised by automations. So how should teams approach balancing the security and certainty from human-in-the-loop, with an eye towards scaling? Over the weekend I got the chance to experiment with a few approaches to turning HITL datasets into their own custom evaluation models. I also think about how I would balance my human-agent evaluations alongside my model-driven evaluations, to set me up for long-term success. https://lnkd.in/gruPtC8d

17

1 Comment

See our GPUaaS software demo

19