Bo Zhang

"AI is powerful, but without the right data foundation, it’s just noise." In the latest Tech Teams Today podcast episode, I had a great conversation with Yongsheng Wu, VP of Engineering at Granica, about why a solid data foundation is crucial for AI success. Here are the top insights from the episode: 1️⃣ Data Foundations Matter: Before diving into AI, ensure your data systems are robust. Solid pipelines & data integrity are critical for meaningful insights. 2️⃣ AI as an Enhancer: AI and ML should optimize existing processes, not solve undefined problems. 3️⃣ Data Quality is King: Curating and labeling data accurately is vital for effective AI models. The better the data, the better the outcomes. 4️⃣ Efficiency & Cost-Effectiveness: Successful AI solutions balance effectiveness with cost efficiency, especially as models scale. Catch the full episode on all major podcast platforms or watch it on YouTube: https://lnkd.in/dbkE__JC #AI #EngineeringLeadership #DataScience #MachineLearning #TechTeamsToday

30

4 Comments

Congrats the team on ICML acceptance of the Wukong paper: https://lnkd.in/g-MyahNH, which represents a breakthrough of recommendation model design and efficiency. Recommendation models traditionally suffer from not able to utilize the full computational power of GPU (only 5-10% of peak GPU FLOPS while LLMs can reach >50%), due to their sparsity. Wukong is first of its kind by pushing toward 40% GPU peak FLOPS though model-infra codesign. One of my teams consisting of GPU experts is building a high performance model component libary where we’ll provide highly optimized model components (eg sequence summarization, x-sequence interaction, x-GPU layout scheme) that could be composed - with the goal that models built from this library have high performance out of box to scale the model-infra codesign effort. We are also scaling training beyond the limits that are typical of recommendation models - from hundreds of GPUs to 10s of thousands. If this is the kind of work that excites you, come join us!

264

4 Comments

LLMs have already made a significant impact on how we run our businesses at Coupang. In this blog we will talk about 3 primary usecases. - Recommendation system models: This is primarily used in personalization and recommendation surfaces such as main home feed, search and ads across Coupang apps for shopping, eats and play. - Content understanding models: ML teams across Coupang use deep learning techniques to understand product, customer and merchant representation and then use it to improve shopping experience. - Forecasting models: Predictive modeling is crucial in the pricing, logistics, delivery, and pricing of these products for our customers. https://lnkd.in/gXsE469h

73

2 Comments

今回のnote記事も引き続き、９月末にY Combinator Summer 2024（YC S24） Batchに採択されたスタートアップ紹介しています。今回の記事では、医療教育の効率化、保険請求業務の自動化など、AIを活用した革新的な医療ソリューションを提供するSoma Lab, Vera Health, Guardian AI, Taxo, Tivaraの5社を取り上げました。 ぜひご一読ください！(Check out the article, YC S24 Healthcare-3) https://lnkd.in/gqps2PhX *Although the article is in Japanese, please feel free to contact me in English if you are interested in the content. In Y Combinator’s S24 Batch, several startups have emerged under the Healthcare category. This article spotlights five innovative companies—Soma Lab, Vera Health, Guardian AI, Taxo, and Tivara—that utilize AI to enhance medical education, streamline insurance claims, and more. 

10

1 Comment

The most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with "regular" programming mixed in. This leads to more capable, more reliable, and more interpretable AI systems. Most people building with AI already know this. But (unless you work for Google) the main barrier to realizing multi-step AI workloads is an infrastructure one. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. This state of affairs is stifling progress. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. Substrate is the first inference API optimized for multi-step AI workloads. With Substrate, you can write less code, run more inference, and build high performance AI applications with zero infrastructure to manage. No tooling, no infrastructure – just elegant abstractions. Excited to support Ben Guo, Rob Cheung and the amazing Substrate team on their mission to democratize how AI applications are built.

41

4 Comments

Insightful Paper on Scaling law for Precision TL;DR The more tokens you train on, the more precision you need for inference. The paper studies the limits of quantization that has implications on compute resources, model size decisions. - Training in lower precision reduces a model's effective parameter count. - Post-training quantization degrades performance more for models trained on larger datasets. - Paper also suggested a scaling laws to predict the loss of a model with different parts in different precisions, and suggest that training larger models in lower precision may be compute optimal. https://lnkd.in/gk8yZx_p #quantization #LLMs #AI #inference

13

Super proud of Mike and his team. Many people asked me over the last year why long context and reasoning are so important. My answer is simple: long context is, in other words, a much better memory that allows LLM to utilize its ability to comprehend incoming text fully which as we all know is a million miles better than RAG. A very clear use case is with processing huge documentation like corporate wiki. Now, the deal with reasoning over long context is also simple: if models can get an ability to process long context but do that poorly then what's the value is in doing? And that's why a proper benchmark like BABILong is so desperately needed. It helps you to open your eyes on how inefficent the majority of the modern methods for processing long context by LLMs are. It helps you to understand what the real goal is. Let's make LLMs understand long context better together!

18

2 Comments

Excited to announce that I am joining Anyscale, the company behind Ray as CEO. Anyscale got its start at UC Berkeley with Robert Nishihara, Philipp Moritz, and Ion Stoica building Ray to scale AI applications across a distributed compute cluster of CPUs, GPUs, and other accelerators. Their vision has really taken off with the growth in GenAI applications that are powering the next great tech wave, and Ray adoption has skyrocketed 6x in the last year (powering AI at companies like Canva, Coinbase, Pinterest, Uber, Runway, Recursion, and many others). Ray allows enterprises to scale compute intensive AI workloads, from training to inference to data processing, all in a single software platform. I’ve experienced 3 massive tech waves in my career - Internet, Mobile and Cloud. The next one is very clearly AI and it is only just getting started. As with other waves, there will be a massive AI infrastructure buildout over the coming decade, and I’m so excited to partner with Robert Nishihara, Ion Stoica, Philipp Moritz, and the whole Anyscale team to tackle this challenge. We are hosting #raysummit in October in San Francisco. There will be some amazing keynote speakers including Mira Murati, CTO of OpenAI and Anastasis Germanidis of Runway. Also a great opportunity to get trained on Ray and Anyscale and learn from the amazing Ray community. Check out https://lnkd.in/gtyrPcWb

1,224

310 Comments

Sharing an article written by my former colleague Summer Hsu on exercising Python programming to find the greatest common divisor. It brought back memories of my junior high school days and how I struggled with math. 🤣 If you're passionate about algorithms and Python programming, this is a must-read! https://lnkd.in/greCwSA2 #Python #Programming #Algorithms #GreatestCommonDivisor

3

This is a HUGE update for us! I’ve spent a ton of time with Keerti Melkote over the past few months and the energy is through the roof. Very very few founders have done what he’s done (starting a business in his garage and scaling it to over $5B in revenue). AI is in its infancy, and the next decade will be an unprecedented AI infrastructure buildout. Ion Stoica, Philipp Moritz, and I are thrilled to tackle that together with Keerti. Over the past year, we’ve enabled so many companies to push the frontiers of AI, #Ray is growing explosively, and we’re barely scratching the surface. ChatGPT and generative AI were an inflection point for our business. Since then, Ray has taken off. 💙 Pinterest runs 5000 training jobs and 300 batch inference jobs every month on Ray. 💙 Runway used Ray to help build Gen-3 Alpha. 💙 OpenAI used Ray to train GPT-4. 💙 Uber standardized model training on Ray. 💙 Amazon processes exabytes of data with Ray. 💙 Ant Group serves over one million requests per second with Ray. 💙 Similar stories abound at Canva, ByteDance, Instacart, Spotify, Niantic, Inc., Samsara, Netflix, Airbnb, Adobe, Shopify, Zoox, Snorkel AI, LinkedIn, Handshake, and many many more. 💙 Much more to share at #RaySummit in October. If you’re interested in helping build the foundational infrastructure powering AI, please reach out. We’re hiring across the board and would love to work together.

264

18 Comments

This week, Paul Graham’s Founder Mode vs. Manager Mode has caught a lot of attention. Many people are intrigued by the Founder Mode concept, and I have to say, I’m a big fan of it as well. It feels like a potential solution to a question that’s puzzled me for a long time. However, I believe that for the foreseeable future—at least for the next 1-2 years—Manager Mode will likely remain the mainstream approach in organizational management. So, if we can't change the status quo just yet, let’s think about how we can be better in the role of a manager. https://lnkd.in/gz8CrFKY

9

🍜Anybody up for some ramen? I am excited to share that I am back in the garage, building again (it’s actually a house in the Berkeley Hills). We are taking the wraps off of Supercog AI, a new startup focused on solving Application Integration using LLM powered agents. My co-founder in this new venture is a great friend of mine (and a former co-founder partner), Alex Osborne. Alex and I have worked around the application integration space for a long time, and we are unrealistically excited about applying the GenAI stack to this problem. Not many people relish the task to get some bits to move from system A to system B. But it’s a critical job to be done, and that’s led to a lot of brittle scripts, a myriad of inflexible “no code” tools, and a lot of hand crafted SQL. But the power of the Large Language Model, trained on a huge corpus of information, offers a remarkable tool to solve this problem. The LLM you use today already knows the APIs of hundreds of popular systems. It knows the SQL dialects of every major database. It understands the semantics and data schemas of many popular SaaS systems. It understands much of the specific domain in which your business operates. I like to call the AI revolution “the last platform shift”. That may turn out to be hyperbole, but I definitely believe we are in the very early stages of seeing what this new stack can enable. I know we are hardly the first folks to claim that “amazing stuff is coming!”. But this is why Alex and I have decided to focus on a real and hard problem. If we can prove that LLMs can power a new way to solve this problem, it will be the first successful *new* approach in 15 years.. We will have much more to share about this new platform soon. In the meantime, if you’re interested in getting a peek - or even better if you have application or data integration tasks that you would like some help with - please reach out because we want to help. #genai #startups #backinthegarage

163

37 Comments

noteでの発信として、Y Combinator Winter 2024 Batchの採択企業を紹介していこうと思います。 第1弾は、「Therapeutics」で分類されている、がん治療薬の開発（4社）、生成AIを活用した情報収集ツール（1社）、mRNAの送達技術の開発（1社）、AI創薬プラットフォーム（1社）のスタートアップについてそれぞれまとめました。ぜひnoteをご覧ください。 また、Y Combinator Winter 2024 Batchの採択企業とは別に、生成AIを創薬分野で活用しているスタートアップとして、Hike Venturesのポートフォリオ企業の「Nextnet」もご紹介しています。ご興味のある方はぜひご連絡ください。 Nextnetは科学論文、遺伝子情報、臨床試験結果などの重要なデータを統合する製薬、バイオテクノロジー、学術分野向けのプラットフォームを開発した。自然言語による検索機能を備え、プラットフォームを活用することで、医薬品開発における新しいアイデアの発見や検証を効率よく進めることが期待されます Nextnet https://lnkd.in/gM34ZUxi https://lnkd.in/gEdiQtXR   *Although the articles are in Japanese, please feel free to contact me in English if you are interested in the content. I would like to introduce companies from the Y Combinator Winter 2024 Batch. Part 1 summarizes each of the startups categorized under "Therapeutics": four in cancer drug development, one using generative AI for information gathering, one developing mRNA delivery technology, and one in AI drug discovery. Apart from the selected companies from the Y Combinator Winter 2024 Batch, I'm also featuring Nextnet, a startup in Hike Ventures' portfolio that uses generative AI to discover drugs. If you are interested in "Nextnet", please let me know. Nextnet has developed a platform for the pharmaceutical, biotech, and academic fields that consolidates critical data such as scientific articles, genetic information, and clinical trial results etc. It features a natural language search function, enhancing its utility for discovering and testing new ideas in drug development.

33

Why is there so much hype about Generative AI (GenAI)? I was watching a great presentation by ARM's CEO Rene Haas today at COMPUTEX TAIPEI 2024. This is a fantastic graphic from Arm which puts into context just how fast Chat-GPT reached 100m users compared to other giants of the industrial world. It took OpenAI two months to achieve the same subscriber numbers that Netflix took 10 years to acquire. That said in my opinion the AI industry is going through a bit of a wobble because people don't know exactly what AI can be used for. There is a lot of fear, particularly in the creative industries about AI taking jobs as opposed to AI making jobs better. The good news is that this is a "marketing problem" not a :"technology problem". The role of marketing is to educate, create awareness and give examples of best practice for others to follow. In the telecoms industry lots of companies are using GenAI in operational and technical areas such as network operations, service management and coding because it turns out that GenAI is really good at processing unstructured data and there is nothing more unstructured than twenty years of product documentation, release notes, bug fixes and customer service cases. GenAI has a really important role to play but not everyone is aware of that yet. Technology marketing is much underrated in terms of business impact. #marketing #ai #innovation

7

1 Comment

Exciting work from Meta and World Resources Institute who have used AI to create the most detailed map ever made of trees across the planet. This allows anyone to query the height of forest canopies or even individual trees (when they stand isolated enough). The data is now freely available. As a fun test, I used the provided browser (https://lnkd.in/ghxQFmxt) to find one of my favorite neighborhood trees - the giant sequoia in Seattle's Leschi Park. The data accurately shows this tree towering above the others in the park (screenshots in comments below). The team used AI at Meta's DINOv2 visual transformer architecture in order to infer the height of trees given just a satellite or aerial color image as input. This is a very similar technique to 3D photos that you may have seen in your Facebook feed (https://lnkd.in/gXCygk5H), where the depth of an image is computed via AI in order give a 3D effect. I'm looking forward to this work continuing and becoming an important way in which the world measures, reports, and verifies the carbon stored in trees. As industries (from big tech to airlines) purchase biosphere carbon credits to offset emissions, it's critical that they have the best data possible. Improved imagery and AI offer a path to getting there. More info in the project video below.

60

9 Comments

Last week Asian founders did... 🚀 Ka Ling Wu launched Upsolve AI (YC W24), a platform to build and deliver analytics to your end-users via embeddable dashboards. 🚀 Sahitya S. launched Endeavor, an OS for manufacturers to automate key workflows across sales, inventory, and supply chain. 🚀 Kevin Yang launched Serif.ai, an AI email assistant that writes context-informed emails in your personal voice. 🚀 Vishnu Sridhar launched Matter Intelligence, developing advanced sensors to capture extreme-resolution images of natural and artificial materials from space to surface. 💸 David Shim raised a $50M Series B for Read AI, an AI copilot for wherever you work, making your meetings, emails, and messages more productive with summaries, content discovery, and recommendations. 💸 Kevin H. raised an $800k Pre-seed round for EQ Exchange (EQX), a crypto exchange practicing radical transparency by publicly disclosing all financials in real-time, both on-chain and off-chain. 💸 Pranav Ravella raised an $8.7M Seed round for Sapien, a platform to create autonomous coworkers for finance professionals. 💸 Alex Yeh raised an $82M Series A for GMI Cloud, an AI infrastructure company providing GPUs and a platform to optimize inference and fine-tuning.🎙️ Arvind Jain, co-founder of Glean, talks about Glean's enterprise search product and the future direction of the company with Sequoia Capital 🎙️ Akhil Ramaswamy, wrote a substack article about approaches to defensibility for AI agents Links for the last two in the comments ⬇️

45

6 Comments

It is really intrigued lesson to demonstrate with LLM and workflow framework. We can definitely program multi-agent scenario with taking snapshot / checkpoint / screaming the states. In this way, we can debug the AI agent in more humatic/traditional way :)... Hope developers won't lose jobs in this exciting and daunting era. https://lnkd.in/eFrnq5sY

3