Bryan Cantrill

Although some #Kakfa vendors are considering S3 Express One Zone for write caching, a model developed by my colleague Jack Vanlightly suggests this works only in a few cases. And with cloud vendors like Microsoft reducing or eliminating cross availability zone networking charges, traditional Kafka replicated logs still work well for many workloads. https://lnkd.in/gHrNPfSZ How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out.

4

I've appreciated all the folks who have suggested I look at System Initiative this week. The great thing is, I've been tracking them for a bit now. I've even written about them and the other companies who are changing how we work, for the better. All the Next Wave companies are focused on context, helping teams navigate the current level of complexity, and preparing teams for what comes next. Huge believer in this problem and the companies that are solving it. Including my own. We're like dependabot for your entire technical stack. I don't know if you are aware but unplanned work is a big pain point for our Ops friends and it's only getting worse. So why not start with the small stuff no one has time to get to, Automate that away and give teams peace of mind while reducing toil. Don't worry Security, there will be checks and balances including an audit trail. It'll make staying in compliance a snap! All that to be said, SI's GA announcement is cool, and the approach of moving from a factory to a team. A pipeline to something more organic to manage infrastructure is timely. The team has worked hard and it shows in their GA offering! While you are checking out SI, I'd highly recommend checking out some of the other companies in the Next Wave of DevOps, if you haven't already. The number of companies is growing and the founding teams are passionate about this space and the problems. We are better together and all of the companies are amazing! + Anyshift.io + AppMap + CloudTruth + DryRun Security + Overmind + Prequel.dev + RunWhen + Stakpak + Stanza New companies are making the hard parts more approachable with our complex and ever-growing technical stacks. From code to deployment and everything in between #opservations

13

1 Comment

"There is another element in this scientific attitude: only that is knowable which is expressed (or, at least, can be expressed) in numbers. To get away from the so-called 'arbitrary and subjective', to escape ethical or literary judgments (which, as everyone knows, are trivial and unfounded), the scientist must get back to numbers. What, after all, can one hope to deduce from the purely qualitative statement that the worker is fatigued? But when biochemistry makes it possible to measure fatigability numerically, it is at last possible to take account of the worker’s fatigue. Then there is hope of finding a solution. However, an entire realm of effects of technique—indeed, the largest is not reducible to numbers. Yet, since what can be said about it is apparently not to be taken seriously, it is better for the scientist to shut his eyes and regard it as a realm of pseudo-problems, or simply as nonexistent. The 'scientific' position frequently consists of denying the existence of whatever does not belong to current scientific method." — Jacques Ellul, The Technological Society یعنی وہ سمجھ سکے تو آنسو، نہ سمجھ سکے تو پانی۔

2

From POC to OSS: The Journey of Co-op Translator! Exciting times ahead as we take the Co-op Translator project from Proof of Concept to Open Source implementation, thanks to the amazing collaboration of Microsoft Student Ambassador Minseok Song and Imperial College London Grad Student Timothy C. who are now official project repo maintainers. Check out the project on GitHub: 🔗 Co-op Translator on GitHub https://lnkd.in/eV9cW6BU This marks a significant step in making advanced translation capabilities accessible to all. Huge thanks to Minseok Song and Timothy for their incredible contributions! The Co-op Translator project is a Python Library and command-line interface (CLI) tool designed to automate multilingual translations for your GitHub projects. Getting Started https://lnkd.in/eym2Y5ZU Co-op Translator leverages advanced language models and Azure AI Services to translate markdown and image files into multiple languages. How It Can Help Developers: 1. Multilingual Support: Easily translate your project's content into various languages, making it accessible to a global audience. 2. Automation: Streamlines the translation process, saving time and effort compared to manual translations. 3. Integration: Seamlessly integrates with Azure resources, such as Azure Computer Vision and Azure OpenAI, to provide robust translation capabilities. 4. Customization: Offers various CLI options to customize the translation process, including specifying language codes and handling different file types. This tool is particularly useful for developers working on international projects, documentation, or any content that needs to be available in multiple languages. Does this sound like something you'd use in your projects? We will be sharing more about the project journey and implementation at a Microsoft Reactor event later this year! #OpenSource #AI #CoopTranslator #Azure #StudentAmbassadors #Innovation

14

2 Comments

Great podcast on asynchronous messaging, it's origins, criticality, standardisation and evolution. Well done AndySC and Clemens Vasters, well worth a listen! https://lnkd.in/ewBQJtAC

30

1 Comment

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience. In comparing the two most recent Microsoft earnings calls, Claude highlighted: - faster than expected Azure growth (29% vs 27%) - AI contributing 8% of Azure revenue up from 7% - higher CapEx spending & greater capacity constraints for data centers - much better commercial bookings growth : 17% vs 13% Excellent analysis. Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query? At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents. For example, if there’s a document containing 10 instances of the word ghotiacross a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish. I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not? Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer. A user might write : A boy with a dog in a park. Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog. Magic Prompt surprised me in three ways. First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner. Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth. Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM. This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.

21

5 Comments

Microservices get a bad rap due to complexity and high TCO. A lot of that is because a microservice-based application works the same way as a bunch of dominoes falling. If one misses, or falls out of order, the whole workflow stops. And you have no control over the process once your microservices start ping-ponging events and RPC's back and forth. That's why #microservices are so expensive to manage. Great blog post below by Colt McNealy about how LittleHorse can solve those problems. Give it a read. https://lnkd.in/gJECBwRu

58

Week #10 of building a storage service. Last week we discussed a case for client protocol redesign where i spoke about how network serialization formats heavily influences I/O. This week I looked into LanceDB serialization mechanism for persistent storage aka how is data actually being stored on disk. Lance recently introduced a really cool technique for the developers to take control over their encoding mechamism by making use of protobufs. Which means as a dev you just have to define your *.proto file and your readers and writers will make use of that to understand how they encode and decode the files. This let's the community figure out new techniques for encoding/decoding data and publish the .proto file for others to use it, with no changes to the system and optimise for their workloads. This essentially overcomes some of parquet's encoding workflows. Encodings - Parquet. https://lnkd.in/gs5_kTxB Here's my #kta aka key take aways from lance's new storage ideology. 1. Lance does not define how your data is serialized into in your file but the developer extensions do. 2. Configuring the optimal size for in memory data structures and persistent data structures have always been a pain in the a** mostly because of row groups, hence they moved to continous column writing and reading where you can configure your page sizes accordingly. Note this is just related serialization and deserialization you can checkout other improvements and benefits here. https://lnkd.in/gd4WGSHP #storage #databases #computerscience #filesystems #encodings

16

Announcing generative LLMs in Vespa Over the last six months, we have invested heavily in integrating LLMs into the core Vespa engine, simplifying search deployments, and allowing developers to choose between local (GGUF) inferencing and external LLM providers with similar APIs. The rise of large language models (LLMs) has transformed search and information retrieval, and we at Vespa.ai have been hard at work since the early days, allowing inferencing with encoder-based transformer models for embeddings, retrieval, and ranking. Now, we are unlocking the potential of larger generative LLMs for a broader range of retrieval and ranking-related serving tasks, including summarization of information from retrieved contexts (RAG). Read more in the announcement and try out the new Vespa sample applications! https://lnkd.in/dCM8eisH

158

10 Comments

#Survey for those in my network. Are you happy with your organizations speed, ability and agility when it comes to software delivery? Have you ever thought that the pipeline could be the problem? Check out this post from my co-founder Richard Lander and let us know what you think!

4

2 Comments

As I am thinking more the idea of DSPM, CSPM and KSPM findings input In ASPM is growing on me. Amazing things are possible when you have access to infra and config code and application code and the power of AI combined with the CSPM, KSPM and DSPM findings at the time of deployment. This is your deployment snapshot from left. If you store it "correlated with" the events in the runtime from the same "SPM" tools and build a lake and have ASPM take recommendations from this lake. The AI can tell you "dude, you have made the same mistake 3 times. Every time you create a new kubernetes cluster that is hybrid multicluster, you forget to turn on mTLS and then you get on an average 3-5 events per day in your CSPM/KSPM tool" I see you doing the same again on this 1 of the 50 apps you are deploying today. I know you don't have time to change all 50 clusters for all 50 apps, but this 1 that has the same characteristics -- the K8 workload spanning hybrid multi-cluster , can you please (may be more authoritative -- "you better") fix it while in lett and give me less heartache that I was there and was not able to help at right time. Ok a big dramatic and over the top because how many customers have hybrid multi cloud Kubernetes and how many deploy 50 apps at the same time with each one with different cluster -- but you get the idea. I will keep thinking and aggregate all usecases coming to my mind and then update the paper. #aspm #devsecops #applicationsecurity #shiftleft

2

GoodData failed to connect to StarTree Apache Pinot running on our customer's side. I challenged the community, but our stack is unusual for them (HikariCP, JDBC). I decided to create a small reproducer, a pure Java app. Why not use #ChatGPT? Let's test the recently released GPT-4o model! Long story short, we (me and ChatGPT) succeeded in a few iterations. It took us about 20 minutes to create not only the Java code, but also pom.xml, .env config, and shells scripts. I open-sourced the result here: https://lnkd.in/eWzNWNN4

40

3 Comments

I did a quick test of the new function calling in #ollama using llama3.1 8B model. The good news is that the API works! The bad news is that llama3.1 does a poor job at formatting function args according to my tool spec. A comparison w/ gpt-35-turbo: https://lnkd.in/g_R3ExQn Anyone had success with local SLMs and function calling?

17

Sharing a great technical blog post from Tomas Hruby. As organizations increasingly adopt eBPF dataplanes as a faster and higher performing alternative to kube-proxy, compatibility issues can arise. This blog goes over some challenges of integrating NodeLocal DNSCache with eBPF and provides a solution tailored for Calico environments. Don't hesitate to reach out if you have any questions about Calico eBPF! https://lnkd.in/gH89DKyJ

14

Unfortunately OpenAI o1 isn't "reasoning" or "thinking" as we humans define it. OpenAI has (intentionally?) muddied the UI/UX waters by introducing verbs like "thinking" / "analysing" etc. to encourage users to anthropomorphize what is going on behind the scenes. o1 is more like iterative prompt generation, and then doing next token prediction for each prompt using an agent. So, yes — it may generate better answers in some contexts, especially the ones where RLHF has been used on its chain-of-though processes (prompt generation), like maths and programming, but its likely to fall into many of the same general traps as next token prediction. The structure and sequence of tokens doesn't *generally* allow you to infer logic. Syntax is not semantics, and the jury is still out on what level of semantic generalisation LLMs can encode, to say nothing of their implementations of algorithms for logical inference and search. #LLM #OpenAI #o1

73

27 Comments

Thinking of how to turn your LLM idea into an actual product? You might want to read this post I wrote 😊 https://lnkd.in/gXUFcg-g

26

3 Comments

I share because the meta-message of making sure you've done the appropriate level of diligence before picking your database (or any architecturally significant component) is a much-needed message IMHO. I've seen too many examples in the past few years of decisions made without the appropriate level of diligence and any regard for the unnecessary costs these decisions impose on the business. https://lnkd.in/eD_24953

4

Metastable failures are real. Thank you, Adrian, for bringing up this frightening class of failures. Here's my favorite published paper on the topic since it reports that metastable failures actually happen in production systems: https://lnkd.in/dzmH3uY7

58

8 Comments