Bradley Taylor

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

Large Language Models (#LLMs) are evolving rapidly, and the demand for running them at scale is skyrocketing. #Ollama is stepping up to meet this need with its official container image on Docker Hub, excelling in local #deployments. However, there’s a catch for containerized environments: downloading #models requires the ollama serve command to be running within the #container. This extra step complicates automated deployments via #CI #CD #pipelines, making seamless model access challenging. To take your #model to production, it needs to be #container-ready—a feature Ollama has yet to fully support. The gap in #AIOps for smooth deployments gets blurred with the inference services available as an easy way to take it to production. Goal is to overcome obstacles and deliver solutions 😉 so here is to get you started on having your (O)#llama #production ready in a private #deployment from your #local to #cloud all in #oneshot . Enjoy Medium: https://lnkd.in/g9jqBQ8d dev.to: https://lnkd.in/gi_gGwam #AI #LLM #Ollama #llama3 #phi3 #mistral #S6overlay #AIOps #CI #CD #oneshot #container #docker #podmanEnjoy

11

Yes, our engineering team really did run CockroachDB on spot instances. I can explain. Our customers are running mission-critical workloads across clusters with hundreds of nodes, and it is paramount that our engineering team can test at a similar scale. But testing at this scale can get expensive. We need to be able to meet our customers where they are, but we also need to be able to control our costs as an organization. Spot instances can create significant savings over VMs, with savings of up to 90% a year. What if we could run long-running clusters on them? Besides saving money we would be able to leverage the unpredictable nature of Spot Instances for some exciting chaos testing! The effort to keep CockroachDB available and workloads predictable while nodes are rapidly disappearing would help us harden CockroachDB. Adam Storm's blog captures our entire journey, diving into the methodologies we chose, lessons learned, and yes—even the impact of preemptions. At Cockroach Labs, our engineering culture doesn't just push the boundaries of what's possible with a mission-critical database; it fuels technical innovation across the board. The results are fascinating, so give it a read if you're curious about running a distributed database on spot instances! Special thanks to Bilal Akhtar, Herko Lategan, David Taylor BabuSrithar M, Sambhav Jain, Shailendra Patel, Vidit Bhat, and Bhaskar Bora https://lnkd.in/et-aEjxm

78

4 Comments

In the Dev world, automation is the backbone of maintaining consistency and reducing manual errors. GitLab’s latest whitepaper touches on this and explores the critical role of standardization in building a modern DevSecOps platform. https://bit.ly/3z7MWTi

34

1 Comment

At the core of effective engineering leadership is the ability to stay connected with your team. This is perhaps the most challenging aspect because retaining the respect of your people is difficult unless you remain technically involved yourself. In my journey as an engineering leader, staying true to the individual contributor role that originally earned their respect has kept me close. This approach extends beyond technical expertise; it's about maintaining a deep connection with the core activities that propel product development forward at Gearflow. One of my biggest professional fears is becoming irrelevant or disconnected from my team and our work. This fear motivates me to continually reinvent and retool in response to market demands. Next month, I'll be sharing about my adventures generating synthetic data recently. And while I'm no expert when I start preparing for these talks, the process is incredibly valuable. It's less about delivering a perfect talk and more about how the preparation expands my thinking and exposes me to new capabilities. This helps me connect the dots to opportunities at Gearflow that bring real innovation to our customers. Speaking regularly holds me accountable and pushes me to learn ahead of the curve, ensuring I remain relevant and impactful day-to-day. Every talk is a journey of discovery, helping me to lead with humility, communicate clearly, and inspire a culture of innovation through a deep, technical understanding of our work.

20

Effective advertising is impossible without data, and our partnership with Integral Ad Science continues to show the critical importance of measuring and targeting high-quality, high-attention inventory in order to drive demonstrable attitudinal outcomes. Collaboration between layers of the measurement stack is extremely valuable, but it requires great sensitivity and care towards consumer privacy. Clean Rooms, used appropriately, are a powerful tool to enable collaboration while respecting privacy. As a long-time Amazon Web Services (AWS) user, we've been very pleased how easy AWS Clean Rooms has made it to get our integration with IAS off the ground. AWS has always put the needs of developers first, and that commitment shows up strong in AWS Clean Rooms' simplicity, easy of use, and tight integration with other AWS services. It's clear that clean rooms are a key part of the future of collaborative advertising, and we look forward to continuing to build on AWS.

4

1 Comment

#AWS #reInvent2024 is here. Come join Tim Honychurch and I for an exciting session on new governance capabilities. We’ll dive into two impactful policies – Declarative policies, debuting at re:Invent, and Resource control policies. You will learn how to define and enforce configuration using Declarative policies and the significance of Resource control policies. 𝐄𝐯𝐞𝐧𝐭 𝐃𝐞𝐭𝐚𝐢𝐥𝐬:   💡 𝐓𝐨𝐩𝐢𝐜: COP378 New governance capabilities for multi-account environments  📅  𝐃𝐚𝐭𝐞 & 𝐓𝐢𝐦𝐞: Tuesday, December 3, 2024, from 5:30 to 6:30 PM   📍 𝐋𝐨𝐜𝐚𝐭𝐢𝐨𝐧: Caesars Forum | Level 1 | Summit 232 #reInvent #AWSOrganizations #EC2 #CloudGovernance #Security

36

1 Comment

Google SRE NYC Tech Talk - May 22nd, 6:30pm Google SRE NYC proudly announces our second Tech Talk event in 2024. The in-person only event will take place on Wednesday, 22nd of May 2024 at 6 PM at our Chelsea Markets office in NYC. The doors will open at 5:30 pm. Spaces are limited, RSVP via our Meetup event page. https://lnkd.in/ef5fqdrE Agenda: Salvatore Furino - Customer Reliability Engineer (CRE) at Bloomberg “The Hammer Changes the Hand” This talk will briefly explore how to view internal tooling through the lens of product management in not just developing and shipping features, but how those features empower teams to change their understanding of their social-technical systems. Sal is a Customer Reliability Engineer. During his career he’s worked as a TPM, SRE, Developer, Sys Admin, and IT support. While not working he enjoys cooking, gaming, traveling, skiing, and golfing. Sal lives in Queens and has a BS in Applied Mathematics from Marist College. Thiara Ortiz - Staff CDN Reliability Engineer at Netflix “How we measure Quality of Experience to ensure our members get a world class experience they have come to expect from Netflix ” Any time a Netflix member sits down, reclines in their chair and turns on their TV to Netflix, there's a moment of truth. It's an opportunity to deliver a spectacular service with amazing quality of experience. This talk will go over how we measure the quality of experience for our members and how we work to develop new metrics when we have additional offerings like live streaming and cloud gaming. Thiara has worked at some of the largest internet companies in the world, Meta and Netflix. During her time at Meta. Since Meta, Thiara has been working at Netflix as a Staff CDN Reliability engineer. Her focus is primarily on resilience and quality of experience for members streaming from Open Connect. When incidents occur and Netflix's systems do not behave as expected, Thiara can be found working and engaging the necessary teams to remediate these issues. Mike Scherbakov - Staff Site Reliability Engineer at Google “LLM for SRE / Using LLM in SRE space” LLMs open up an opportunity to automate and scale many operational processes, which couldn't be otherwise solved by conventional methods. Examples include simple summarization of issues and incidents, assisting production on-callers, managing incidents, clustering (creating taxonomy) of issues, scaling SRE via assisted review of development design documents. Therefore LLMs provide a new and unique opportunity to transform the work we do as SREs. Mike works in YouTube Ads SRE as well as co-leading LLM4SRE, LLM center of competency in SRE at Google Our Tech Talks series are focused on professional development and networking: no recruiters, sales or press are allowed.

7

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

16

3 Comments

This may be biggest positive change in Google docs for those of us in OSS land. They've improve native markdown support. You can now export/paste as markdown and download a doc DIRECTLY AS MARKDOWN~ https://lnkd.in/g9asZ_Au

159

10 Comments

Learn how major streamers can scale up capacity to meet the demands of major events like the Olympics with distributed, interconnected #DigitalInfrastructure, blog by Matt George and Marco Zacchello.

1