Jennifer Davis, Ph.D.’s Post

Small yet mighty! 💫 We are releasing SmolVLM: a new 2B small vision language made for on-device use, fine-tunable on consumer GPU, immensely memory efficient 🤠 We release three checkpoints under Apache 2.0: SmolVLM-Instruct, SmolVLM-Synthetic and SmolVLM-Base (made for fine-tuning) and an adapter trained using DPO 🤗 This release comes with a blog on training details and evaluation, a demo, fine-tuning code, MLX integration and TRL integration for DPO 💝 All the links can be found in the comments 💬

You there! Are you suffering from an acute case of Kubernetes? Are your services unobserved? Your traffic unencrypted? Your cluster failover strategy a mess of shell scripts? We have the cure for you! Linkerd, the fastest, lightest, most secure service mesh on the planet. Linkerd's ultralight Rust proxies give you instant health metrics zero-config mutual TLS, retries, gRPC load balancing, cluster failover, and more. Linkerd is a CNCF-graduated, 100% open source project. Visit linkerd .io to learn more. Linkerd is not approved by the FDA. Ingest at your own risk. May cause temporary euphoria and/or blindness. Buoyant reminds you to always mesh responsibly.

𝗧𝗵𝗮𝗻𝗸𝘀 𝘁𝗼 𝘀𝘁𝗮𝘁𝗶𝗰 𝗞𝗩 𝗰𝗮𝗰𝗵𝗶𝗻𝗴 𝗮𝗻𝗱 𝘁𝗼𝗿𝗰𝗵 𝗰𝗼𝗺𝗽𝗶𝗹𝗲, 𝗣𝗮𝗿𝗹𝗲𝗿-𝗧𝗧𝗦 𝗷𝘂𝘀𝘁 𝗴𝗼𝘁 𝘂𝗽 𝘁𝗼 𝟰.𝟱𝘅 𝗳𝗮𝘀𝘁𝗲𝗿 🚀🤗 The Parler-TTS Mini and Large v1 models release enables high-quality, consistent speech generation via simple text prompts... but also includes 𝗮 𝗰𝗼𝗱𝗲𝗯𝗮𝘀𝗲 𝘂𝗽𝗱𝗮𝘁𝗲! 🔥

You there! Are you suffering from an acute case of Kubernetes? Are your services unobserved? Your traffic unencrypted? Your cluster failover strategy a mess of shell scripts? We have the cure for you! Linkerd, the fastest, lightest, most secure service mesh on the planet. Linkerd's ultralight Rust proxies give you instant health metrics zero-config mutual TLS, retries, gRPC load balancing, cluster failover, and more. Linkerd is a CNCF-graduated, 100% open source project. Visit linked .io to learn more. Linkerd is not approved by the FDA. Ingest at your own risk. May cause temporary euphoria and/or blindness. Buoyant reminds you to always mesh responsibly.

Problem: Sentinel OS1 has too many steps during setup before operation. Now Introducing Machine Learning with Sentinel OS 2. ✅ https://bit.ly/3winSrB

Back at the end of 2020 while the pandemic was still at its peak, I gave a talk at Mile High Video lamenting the pains of still being stuck using RTMP as a contribution protocol. I'd spent time playing with SRT, RIST, and a few other proprietary contribution protocols, trying to make them fit into Mux's multi-tenant architecture, but it just wasn't quite the right time. SRT's stream_id had only just been introduced as a way to achieve multiplexing, and wasn't widely supported. There was also still no clear leader in what protocols both software and hardware encoder vendors were supporting. Flash forward to the end of last year, and the ecosystem had moved on pretty quickly, and we had a prototype up and running proxying SRT to RTMP for customers to test their encoders and kick the tyres on, and it worked pretty well, I even showed it off at SF Video Tech (though my lightning talk will never be as good as Alex Converse's original masterpiece "SRT: How the hot new UDP video protocol actually works under the hood"). Customers were asking "when can we use it in production?". At this point, SRT had gained critical mass, and the time had come for us to actually go and integrate it natively into the product. Robert Peck, Corey Sery, Michael Smith, and the rest of the engineering team did a great job of building SRT support into our edge clusters, improving a whole host of other product features as they went, and ultimately giving customers access to a public beta a little over a month ago. We even slipped in support for HEVC. And that brings us to today, and a GA announcement of our SRT feature, including an awesome marketing video from Staci DeGagne, featuring some handsome young dude 👀. Sometimes things don't change as quickly as you want, and you just have to wait a little while for the ecosystem to stabilize around you. As always a huge thank you to Haivision for developing and open sourcing SRT for us to all use ❤️.

What we built 🏗️, shipped 🚢, and shared 🚀 last week: vLLM! Last week, we explored vLLM, or Virtual LLM, the fastest and easiest-to-use open-source LLM inference and serving engine. 📚 We learned: 🍾 The bottleneck is memory 🤔 Virtual LLM is a throwback to virtual memory with paging 🏦 High-throughput serving requires batching and smart KV-caching 📜 PagedAttention does smart KV caching vLLM is not just about faster inference—it’s about building a complete package for efficient inference and serving. Full Video: https://lnkd.in/gxTBwHDz #InferenceServer #vLLM

An insightful introduction to Federated Learning (FL) that is one of the privacy enhancing technique (PET). The course includes client selection, tuning, privacy and bandwidth topics in FL.

DBRX 132B - Instruct 4 bit 🔥 > Requires 70GB RAM, powered by MLX 🤗 Model: https://lnkd.in/ggk98DsA DBRX 132B is wild! 🤯 > Trained on 12 Trillion tokens. > Beats Grok-1, Mixtral, etc. > Mixture of Experts. 16 experts, 4 active. > Uses RoPE, GLU and GQA. > Context size of 32K. > Open access - Base and Instruct. 🔥 > Requires 264 GB RAM; inference with Transformers! 🤗 Check https://lnkd.in/gyY2xgk3 Both the base and instruct checkpoints are open access on the Hub: https://lnkd.in/ghb6sBcP

🧑💻 Looking to overcome problems that often arise with decoders (those in charge of transforming binary or Base64 payloads into friendly JSON ones), which can be confusing due to their length or even introduce compatibility issues, we offer a public “Decoder as a Service” endpoint. https://buff.ly/43NjlZG