Martin Mao

Millisecond cold starts are a reality at kraft.cloud , as is milliseconds scale to zero. What's new, and available soon, is being able to have 𝘀𝘁𝗮𝘁𝗲𝗳𝘂𝗹 scale to zero in milliseconds 🔥 First, some numbers, using NGINX as an example: ⚡️ Cold start: 24 milliseconds ⚡️ First stateful scale to zero (suspend): 174 ms ⚡️ Scale to one (resume): 3.67 ms ⚡️ Further stateful scale to zero (suspends): 2 ms I'll be writing a blog post to explain this in more detail, but basically the use of unikernels (extremely specialized VMs) allows for fast suspend/resume of their state. The second and subsequent suspends are quick since we do incremental snapshots. Finally, note that this mechanism is agnostic to the workload, so while we used NGINX as an example, this works with more complex apps/langs/frameworks like Node.js .

38

3 Comments

And that was a wrap! As #apisummit24 completes with over 10,000 attendees from the API community - making it the largest API conference by far - with dozens of awesome product and partner launches to show what we can do with *the API platform for the AI era*... one fundamental question remains open: Are you on the right side of AI? One thing we know for sure... APIs are the common language of LLMs as we build apps and transform our business with {Gen} AI. In this post, I wrote a few thoughts from my opening keynote on the API market evolution that's ahead of us. Lmk what you think!

42

1 Comment

Adam's post is good (and I like this categorization in general), but there are a couple of things that I don't agree with. First, I'd say time-to-recovery is a strength of CfD rather than CfM. Physalia (from our paper "Millions of Tiny Databases" https://lnkd.in/gepZPA2Y) is the CfD system that provides the metadata for EBS (https://lnkd.in/g7R3MgYk) a vastly bigger CfM system. We chose CfD for Physalia specifically to optimize recovery time, because it needs to recover before the larger system can recover. Second, I think the post is making an assumption that CfM systems are f+1 on the dataplane, use the minimal number of replicas, and don't use quorums. Aurora is an instructive counter example (https://lnkd.in/gNrNTPp7).

117

2 Comments

To leverage preemption, follow these best practices: ✅Define PriorityClasses for your applications based on importance. ✅Assign PriorityClasses to pods appropriately. ✅Monitor cluster load and pod evictions. ✅Adjust PriorityClasses as workload priorities change. By mastering preemptible pods in Kubernetes, you can keep your critical services running smoothly and optimize overall resource utilization. Give it a try on your next cluster upgrade! https://hubs.la/Q02LrySX0 #kubernetes #k8s #sre #devops #devtools

15

Since we launched Redshift Streaming Ingestion with Kinesis Data Streams (KDS) and Managed Streaming for Apache Kafka (MSK) we have seen great adoption with thousands of customers ingesting data at very high rates with low latency directly to Redshift Managed Storage. One common request was to extend the ingestion capabilities to other Kafka sources. Today we are extending Redshift's Streaming Ingestion to support self-managed Apache Kafka on EC2 as well as partners, such as Confluent Cloud. Go and Ingest! https://lnkd.in/gpzC-JD2 #amazon #redshift #streaming #ingestion #kafka #confluent

178

1 Comment

New Time Series Benchmark compares CrateDB with MongoDB and Influx. CrateDB stands out with its very fast ingest performance and instant aggregation of any group of columns. No need for complex indexing strategies. In today’s digital landscape, efficient handling of time-series data is key for informed decisions in IoT, cybersecurity, and data-driven industries. A reliable database can make all the difference. #DataManagement #TimeSeries #BigData #PerformanceBenchmark #DataAnalytics #TechNews

8

AWS re:Invent is less than two weeks away. Want to accelerate your AI initiatives? Let's connect in Vegas and discuss how Meibel enables your engineering team to build Explainable AI into your products and experiences.

56

A good friend recommended me to try Cursor.sh, so I did. I was not ready for what happened next. But first, let's start with a little idea I had for a while. Every time I needed to check a text, I needed to find my tab with ChatGPT, login if disconnected, start a new chat, type "fix typo:" then paste my text to fix, then copy the text back. Not efficient. I knew that using OpenAI APIs I could do a simple website that would do the same, but without the need of writing "fix typo:" every time. A simple hack. Of course I never found time to start this pet project. Until I decided to give it a try with Cursor.sh and made it real is a couple of hours. fix-typos.com

12

1 Comment

"The Heart of the #datamesh Beats Real-Time with #apachekafka" -> Video recording from my tech talk at the Wix Kafka meetup in Tel Aviv. If there were a buzzword of the hour, it would undoubtedly be “data mesh” and related debates around #dataproduct! This new architectural paradigm unlocks analytic and transactional data at scale and enables rapid access to an ever-growing number of distributed domain datasets for various usage scenarios. The data mesh addresses the most common weaknesses of the traditional centralized data lake or data platform architecture. And the heart of a decentralized data mesh infrastructure must be real-time, reliable, and scalable to provide access to data products in good data quality, no matter if the application is real-time, batch or a request-response API. Learn how the de facto standard for #datastreaming, #Kafka, plays a crucial role in building a data mesh and data products. https://lnkd.in/dMT5uqBw

73

Managing production software, and especially "root causing" issues, whether at the infra or app layer, continues to be a growing challenge despite the success of companies like Datadog. I have looked at applying deep learning to this problem since 2017 when we worked with Gurashish Brar & Mohit Gupta to incubate Opas.ai. The team was exceptional, and they even hired a great SDR (me 😜 ). We learnt the hard way that it was too early and #AI was just not ready. A lot has changed since in terms of data infrastructure (check out Conviva's Operational Data Platform), causal inference models and LLMs. Jaya Gupta and I are excited to take another whack at this market, which we call #AgentSRE to distinguish from the outdated #AIOps framing. If you are building an #AgentSRE company, ping us. https://lnkd.in/gvBgfN7k

45

6 Comments

[WEBINAR] "Scaling Out GenAI with Message Queues on Kubernetes" Say goodbye to piled-up requests, skyrocketing latencies, and costly over-provisioning once and for all. During this game-changing session, you'll discover how to: ✨ Seamlessly generate thousands of LLM completions without breaking a sweat ✨ Leverage RabbitMQ, PostgreSQL, and Bento for robust request/response management ✨ Implement comprehensive observability, dashboards, and event-driven autoscaling with Prometheus, Grafana, and KEDA ✨ Streamline deployments with Helm and Helmfile, saving you time and headaches Don't let load balancers hold you back any longer. Secure your spot now! https://hubs.la/Q02Ps67B0 #GenAI #kubernetes #k8s #sre

3

If you are working with Docker... I highly recommend looking at the Slim toolkit. https://lnkd.in/gX_Gb9Kf Why? Slim allows developers to inspect, optimize, and debug their containers using its xray, lint, build, debug, run, images, merge, registry, vulnerability (and other) commands. It simplifies and improves your developer experience building, customizing and using containers. It makes your containers better, smaller and more secure while providing advanced visibility and improved usability working with the original and minified containers. Don't change anything in your container image; minify it by up to 30x to make it secure, too! Optimizing images isn't the only thing it can do, though. It can also help you understand and author better container images. Keep doing what you are doing. No need to change anything. Use the base image you want. Use the package manager you want. Don't worry about hand-optimizing your Dockerfile. You shouldn't have to throw away your tools and your workflow to have small container images. Node.js application images: from ubuntu:14.04 - 432MB => 14MB (minified by 30.85X) from debian:jessie - 406MB => 25.1MB (minified by 16.21X) from node:alpine - 66.7MB => 34.7MB (minified by 1.92X) from node:distroless - 72.7MB => 39.7MB (minified by 1.83X) Go application images: from golang:latest - 700MB => 1.56MB (minified by 448.76X) from ubuntu:14.04 - 531MB => 1.87MB (minified by 284.10X) from golang:alpine - 258MB => 1.56MB (minified by 165.61X) from centos:7 - 615MB => 1.87MB (minified by 329.14X) Rust application images: from rust:1.31 - 2GB => 14MB (minified by 147.16X) Java application images: from ubuntu:14.04 - 743.6 MB => 100.3 MB

4

Broctagon Fintech Group has achieved 300% user growth and revolutionised its FX Trading with MongoDB Atlas! Broctagon's AXIS platform is a comprehensive solution that integrates multiple trading platforms but they were struggling with their existing relational database as demand grew. The company faced issues with scalability, rising costs, declining performance, and slowed innovation. The switch to MongoDB Atlas has yielded: 📍300% growth in user base within a year, now supporting over 1.5 million traders globally. 📍Ability to provision AXIS to new brokerages in just 24 hours, down from 3 days. 📍Report generation up to 10 times faster. 📍Support for 13 languages and integration with various payment solutions. 📍Launch of innovative features like the proprietary trading module. This success story has definitely set a new industry standard. Stewart Garrett, Céline Zuberek, Samantha Seow, Kevin Lim 林德义, Eunice Ng, Samyak Sabat, Fabian Au, Keagan Schmitz, Ban Selvakumar, Shreyaa C., Joe Toh, Emanuele Molteni, Hussain Boltwala, Anand Krishnan #MongoDB #ArtificialIntelligence #Innovation

37

Amazon ECR now supports synching your container images from Gitlab with ECR's pull-through-cache feature #AmazonECR #Gitlab #AWS #ECR https://lnkd.in/gv7HqV3m

63

Task-specific models and routing architectures are the future. Robert Nishihara, co-founder and CEO of Anyscale, is seeing hybrid strategies emerge, where businesses are evaluating different types of models for different use cases. With the rapid advancement of AI tech, it won't be long before open-source models are "good enough" for the majority of business tasks. The natural progression from here is to develop smaller, task-focused models using proprietary company data. As Robert puts it: "You're going to need small, fast models. And task-specific models have a huge advantage when it comes to cost and speed because you can achieve the same quality with a smaller model if it's specialized." Another interesting development: the growing use of routing architectures. Imagine a classifier as a kind of traffic controller for your AI queries. It assesses the complexity of each query and directs it to the most suitable model: simpler requests are handled by smaller models, while complex ones are passed on to larger, more sophisticated models. “If you do it right,” Robert says, “you can achieve something like the cost of the smaller model with the quality of the more advanced model.” Dive deeper into AI architecture trends in my full discussion with Robert on the B2BaCEO podcast here: https://lnkd.in/ghigvERn

46

3 Comments

Datadog's Cloud Cost Management is coming to GovCloud as a FedRAMP Moderate authorized cloud service. This is huge for our existing and potential customers. We already have the largest number of out of the box integrations for AWS services (>100), but now customers can pair that ease of implementation and instrumentation with cost management allowing: - Clarity on your cloud bill - Instantly allocate your cloud costs across AWS, by product, service, and team, with out-of-the-box and user-defined tags - Identify the biggest cost changes and use powerful queries to drill down to resource-level spend - Make cost decisions with confidence with built-in cost-validation that ensures high data accuracy One of the largest advantages of pairing our Gartner MQ leader quadrant APM and Observability suite with Cloud Cost Management is the ability easily make dev and ops part of the cost modeling conversation, connecting cost data to dev and ops decisions and actions. Online video tour here: https://lnkd.in/e5TBHxKw Ping me for any additional questions or to set up a customized demonstration with our US Public Sector account teams. #apm #observability #cloud #cloudcostmanagement #devops #fedramp

114

Did you know Vaadin supports GraalVM native image builds out of the box? Compile your app with mvn -Pnative -Pproduction native:compile Then run the native binary in the target dir and ⚡️🤯 https://lnkd.in/gXr-VfXb

55

1 Comment

How Startups Grow 👇 1. How Linktree got from 0 to $37M ARR and 12M users with the simplest product ever: https://lnkd.in/dET-g-Gd 2. How Rubrik's CEO hacked growth to achieve +$500M ARR in 10 years: https://lnkd.in/djKajRDj 3. How Perplexity AI got to +$1B valuation with $20M ARR under Google's Monopoly: https://lnkd.in/dzRXWYw3 4. $100M ARR in 18 months: how Wiz became the fastest-growing software company ever: https://lnkd.in/dyHk2Pda

32

6 Comments