Brandon Black

Meta just released LLaMA 3 model which is an open source #LLM and you can now play with it on an OpenAI like interface that you can interact with at https://www.meta.ai Found this cool video so you can see how well #LLaMa 3 actually does in solving some problems you might need help with... https://lnkd.in/g6AtmG3h

8

Happy to see Yann LeCun’s strategy paying off big time! He was the strongest voice of the open-source AI movement, and suggested since the first announcement of ChatGPT that there was no intrinsic moat, at least from an algorithmic perspective. This release proves that he was right. Now any company in the world can in principle use, fine-tune and adapt an LLM that is on par with GPT4o and Claude. I am not saying it is trivial, cheap, or even necessary. OpenAI still does amazing things (inference speed comes to mind), which non-specialists will have a hard time replicating. But the algorithmic core isn’t the differentiator (spoiler: it never is). Great job 👏 #llama #oss #opensource #llm #ai #moat

11

The market for third-party CDN services, which I define as the delivery of video and small/large objects, was about $5 billion in 2023. (Full blog post: https://lnkd.in/ekFZqX9y) This figure does not include revenue for services tied to security (WAF, DDoS, etc.), compute, hardware, and storage and excludes delivery in China. The number excludes any company with its own DIY CDN, like Netflix and YouTube, that doesn’t sell its infrastructure as a service to other companies. The figure also excludes revenue from vendors offering online video platforms, including Brightcove, Uplynk, AWS Media Services, Quickplay, Deltatre, Mediakind, Sports Radar, Cognizant, Endeavor Streaming, StreamAMG, Bedrock Streaming, etc., which combined saw revenue of about $1.8 billion in 2023. Based on what I know, AWS Media Services is the largest of the group, with about $450 million in revenue in 2023. While reports produced by research firms say the CDN market was multiple times larger than my number, they are wrong. Full stop. Do not buy their reports. Akamai, Fastly, Edgio, CDN77, and others disclose enough details tied to their delivery revenue that we have numbers to form a good foundation for market sizing. The market is not as big as many suggest, and those writing about the size of the CDN space don’t define it and include non-CDN services like transcoding, hardware, hosting, storage and transit in their numbers. Full blog post: https://lnkd.in/ekFZqX9y Listen to my special CDN-focused podcast from July 2024 for more details and updates regarding 4K, low/ultra-low latency, CMCD, DIY (TikTok, Disney), Open Caching, pricing and multi-CDN. https://lnkd.in/epzXSMVA Vendors mentioned: Akamai Technologies, Amazon Web Services (AWS), Edgio, Fastly, Cloudflare, CDN77, Medianova CDN, MainStreaming, Qwilt, Wowza, Ateme, Broadpeak, Gcore, Netskrt Systems Inc., Varnish Software, Vecima Networks Inc., Velocix.

666

39 Comments

I did a quick test of the new function calling in #ollama using llama3.1 8B model. The good news is that the API works! The bad news is that llama3.1 does a poor job at formatting function args according to my tool spec. A comparison w/ gpt-35-turbo: https://lnkd.in/g_R3ExQn Anyone had success with local SLMs and function calling?

17

Anthropic just launched Claude 3.5 Sonnet, which beats both OpenAI GPT-4o and Anthropic’s previous paid model Claude 3 Opus on most benchmarks. Claude 3.5 Sonnet can execute and debug code, and has improved image understanding capabilities, including text transcription. A new feature called “Artifacts” enables users to view generated code, documents and websites in a separate window next to the chat. Claude 3.5 Sonnet is available for free to all users, but paid users get higher usage caps. Did Anthropic just take the lead back from OpenAI? It seems that ChatGPT’s Code Interpreter tool, which used to be one its main advantages, has finally met its match, but ChatGPT still has the advantages of real-time Internet access and custom GPTs which Claude lacks… What do you think? https://lnkd.in/d_hk86cf

10

3 Comments

It seems that with C/C++ -> Rust, specifically, you could do this by comparing the output compiled code as one of your verifications of correctness for ML learning This is something that e.g. copilot doesn't do - but it'd be straightforward to do so when doing training The expectation would be that the compiled rust code should be equivalent to the compiled C code, i.e. the assembly should be semantically no different As the ML learns, it may end up doing some funky rust code simply to get that to match - but that's probably a good thing, as this is the part that would require some manual intervention & you'd at least be guaranteed equivalent behavior

1

A new table-format comparison post is out. How do the table formats support the streaming ingestion of row-level operations (insert/update/delete), such as ingesting CDC streams? https://lnkd.in/dFDs4Ah3

36

Nice insight on architectural trends in GenAI. Other way to look at this is chart is - RAG is a relatively low hanging fruit.. so easy to implement and adopt. - Now as focus will shift on higher hanging fruit, the need will be to show gain on those use cases where simpler approaches has reached saturation point. - prompt engineering was hot cake in first year and it is down now as that was easiest fruit to catch but soon peaked in terms of value. Then RAG. Now focus will shift to show value which may come by applying reasoning models as well as some domain specific intelligence. ( and can make tuning necessary ). #ai #genai #llm

12

2 Comments

🚨 Our Vercel + Auth0 integration is now available in the Vercel Marketplace! 🚀 The team works hard (and smart) to make auth seamless for developers. This new integration is another way we are making identity easier for developers using Vercel. Nice work to the teams behind this work! ✨ #DeveloperExperience #Auth0 #Vercel #TeamWin

35

2 Comments

Introducing INVARCH:   A Post-Structuralist Approach to Software Design After nearly 30 years as a software architect, I have synthesized my experience into a singular, comprehensive theory. This journey has been the most profound gift of my career and life. INVARCH, a product currently in alpha development, is my attempt to apply post-structuralist concepts to software design. Inspired by biological motifs like RNA synthesis (PCR methods), this paradigm fundamentally redefines how software structure is conceived. Here, "structure" is not designed—it emerges from the design itself, evolving autonomously through differential absorption. At its core, this paradigm uses a meta-design schema as the foundation. From this schema, “structure” is mechanically generated, mirroring how DNA encodes and manifests life. Software modifications no longer require direct adjustments to entities. Instead, they are implemented through schema language changes, akin to editing genetic sequences. The system functions as a primitive organism, with: - A consistent schema space (the "DNA") - A database (the "organism") - A membrane covering the design's surface This ensures an interface that remains lightweight, adaptable, and responsive to user inputs, continually evolving in real-time. The result is a paradigm where "design is immune to unknown errors." In essence, INVARCH represents a vision where software is both the plan and the design itself. Just as DNA gives rise to living organisms, structure emerges from the plan, harmonizing dynamically over time. The project is currently named "LEGO" but will soon be renamed "INVARCH" to address trademark considerations. I am the sole developer working on this open-source project. If development proceeds without delays, I anticipate a beta release by late 2025 and an alpha version by early 2027. Get Involved: If you’re interested in joining the development, sponsoring the project, exploring partial integration into your business, or scheduling an in-depth discussion, I would love to hear from you. Let’s shape the future of software design together. --- https://lnkd.in/gUsd9xpy

1

Are you someone who finds distributed systems challenging? If so, I highly recommend checking out Martin Kleppmann's distributed systems lecture series on YouTube. It's an absolute goldmine of information! In this series, Kleppmann covers a wide range of fundamental concepts in depth, including: 💻 Physical clocks and logical clocks (Lamport and Vector clocks) ⚡ Causality and the happens-before relationship 📡 Broadcasting algorithms 🔄 State Machine Replication ☑️ Consensus algorithms like Raft Whether you're a student, a professional, or just someone curious about distributed systems, these lectures are an invaluable resource. Kleppmann's clear explanations and real-world examples make even the most complex topics accessible. Don't struggle with distributed systems any longer. Leverage this incredible resource and gain a solid understanding of the core concepts. Trust me, it'll be a game-changer! #DistributedSystems #MartinKleppmann #LecturesSeries #TechEducation

6

Arg! After thousands of lines of code cannibalizing the LLVM support library I’ve run into a fundamental design flaw in my approach. Too technical to talk about. Let’s just say I’ve created a set of cyclic dependencies where a depends on b which depends on c which depends on a. It’s because I made a fundamental change to how the library works (which makes it much easier to use when it has to work on different platforms like MacOS and iPads, and is the main reason I went down this path rather than trying to use the library as is) while trying to stick very close to the original structure and design. Not gonna fly. So either I throw out all that work and give up the ease of use - not happening - or I step back and rethink the big picture design elements. And despite the frustration I am loving digging into this library. Usually. By the way this is pretty much what programming is all about - running into obstacles and figuring out how to overcome them. We call it Software Architecture, but it’s not at all like building cathedrals.

6

5 Comments

Although some #Kakfa vendors are considering S3 Express One Zone for write caching, a model developed by my colleague Jack Vanlightly suggests this works only in a few cases. And with cloud vendors like Microsoft reducing or eliminating cross availability zone networking charges, traditional Kafka replicated logs still work well for many workloads. https://lnkd.in/gHrNPfSZ How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out.

4

the "ai future" requires 100% open-source. not for code reasons, but for TRUST reasons. push-back is typically "impact to financial potential", so here - https://lnkd.in/gQxk6gfj is an intrinsic trillion $ to get things started on that debate or defend the goal of being less trustworthy ;> @eff @opensourceorg #linux #code #trust #ai #postaitrust

2

Hybrid licenses will dominate the future of #OpenSource Software. I'm not so sure about that. Despite heated debates on licences changes including MongoDB and HashiCorp, data reveals a clear preference for Open Source Initiative (OSI) approved permissive licenses among top Commercial Open Source companies and their repositories across three key categories: funding, stars, and contributors. 1️⃣ **Permissive Licenses Dominate:** Companies and repositories with OSI-approved permissive licenses lead in funding, stars, and contributors. This suggests a preference for licenses that facilitate both open-source and proprietary use. 2️⃣ **Lower Presence of Copyleft Licenses:** OSI/Copyleft licenses, which mandate that derivative works also remain open source, are significantly less common. This indicates a community and investor preference for less restrictive licensing. 3️⃣ **Minimal Non-OSI Licenses:** A small percentage of funding, stars, and contributors are linked to non-OSI licenses, which may include proprietary licenses or those not recognized by the OSI. The preference for permissive licenses likely reflects their commercial viability, allowing companies to integrate open-source code into proprietary products without needing to disclose derivative works. Picking your licence is not a business model. It's not a shortcut to monetization, as challenging as it remains for Open Source companies. Remember that Hybrid Licences or changes in Licences is likely to antagonize your community. #OpenSource #SoftwareLicenses Serena Bertrand Diard Guillaume Decugis Gilles Rouvier Damien Garros Floriane de Maupeou Antoine Giacomini

30

Llama 3.1 8B, 70B and 450B guess someone in Meta loves Llamas more than Johnny Depp. Me too actually hence created this tutorial to use Llama 3.1 model to create a RAG pipeline with the help of LlamaIndex and Groq As of today Llama 3.1 405B is first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation. If you like the tutorial please Like, Share and Subscribe. Google Notebook- https://lnkd.in/evPuNGya #Llama405B #HuggingFace #Meta #LlamaIndex #Groq #RAG #pipeline #llm #generativeai https://lnkd.in/eTaQjibX

5

Does your #genai workload have a need for speed? Then #OpenAI GPT-4 may not be the #LLM you're looking for. This walkthrough and repo shows how to achieve higher throughput using #dotnet channels and server sent events (SSE) along with Microsoft Semantic Kernel to build a blazing fast recipe generator powered by Meta's #Llama3 8B/70B on Fireworks AI. https://lnkd.in/eZNyzNFm

12

1 Comment