Andrei Pechkurov

So I became fascinated by C++ coroutines late last week, and have spent a good portion of the weekend figuring out how they work. The TL;DR of my discoveries so far is that C++20 coroutines: 1) follow the best and the worst of C++ practices, and 2) are a really clever way to decouple stackless functions from their execution model. For (1), well, it's C++. No crisp & clean syntax, lots of weird compilation errors, and even weirder runtime errors. I hated them with passion for the first several hours. But then they grew on me, because the flip side is also true. The flip side is backwards compatibility and "only pay for what you use". And introducing a totally new concept — the stackless execution model, effectively, [proto-] green threads — into a mature and complex language is one hell of a pain! For (2), I do believe it's a genius solution for the language to separate stackless functions from their execution model. And C++20 did exactly that. There is literally no event loop in C++20! The claim (as I understand it) is that a tiny microcontroller may want to use C++20 coroutines, and the authors of the Standard are very explicit about not constraining possible runtimes. All that C++20 coroutines do is introduce stackless functions into the language as first-class citizens. And yes, stackless immediately implies resumable. So it is one step about the most-clever-possible syntactic sugar over `setjmp()`. Full one step. But not a millimeter more than this one step. Whether you "execute" your coroutines in a dedicated single-threaded event loop, or a thread-pool-based event loop, or by individual steps via some timer event or even a network packet — that is entirely your call. The stackless functions are there, `co_await` (as well as `co_yield` and `co_return`) is there, no callback hell is needed, and all the RAII and scoped initialization/ownership model is fully intact. Now, I'm still elbows deep in my experiments, but here's the first big LinkedIn-worthy question. Why don't we write our C++20 [unit] tests as corourines? Say, you want to limit some test to five seconds. And fail it as "timed out" within these five seconds. Before coroutines, doing this without instrumenting the very code of the test involved `fork()`-ing the process, effectively relying on the operating system's primitives to ensure proper resource deallocation. With C++20 coroutines, as long as the slow test is slow on non-blocking I/O (i.e. not in an infinite loop, but on something that `co_await`-s stuff), it is possible to time-box each individual test to some five seconds very much organically. Because a coroutine frame, unlike a stack frame, can be `.destroy()`-ed safely and cleanly. So, are we doing it yet? If no, why not? If yes, what would be some next-gen `gtest` to look into? Also, I'll be writing some code and blog posts on C++20 coroutines soon. If you are reading this text and are an expert, DM me pls — I'd love to pick your brain!

19

4 Comments

At the risk of repeating myself too often; I believe the next big revolution in the AI/ML space is the democratization of access to the technologies we've seen and – let's be honest; mostly – played with over the past few years. Both for organizations that cannot throw millions of dollars in computing cost at the problem and also for enabling true, Internet-scale, cost-effective solutions, Jo Kristian demonstrates brilliantly (as always) how you can combine the latest updates in matryoshka and binary quantization embeddings in Vespa.ai to significantly slash the cost of vector search while maintaining (most of) the accuracy.

28

2 Comments

NetBSD restricts the addition of generated code from AI tools. Its guidelines for committing code are as follows: "Code generated by a large language model or similar technology, such as GitHub/Microsoft's Copilot, OpenAI's ChatGPT, or Facebook/Meta's Code Llama, is presumed to be tainted code and must not be committed without prior written approval by core." What should be the limits for using generated code? Are these rules different for open-source software? That's probably a discussion without one single answer. https://lnkd.in/dFHCyaqN

5

How to Limit Indexing in IntelliJ IDEA for Better Performance #ide #indexing #intellij #jetbrains

2

Although some #Kakfa vendors are considering S3 Express One Zone for write caching, a model developed by my colleague Jack Vanlightly suggests this works only in a few cases. And with cloud vendors like Microsoft reducing or eliminating cross availability zone networking charges, traditional Kafka replicated logs still work well for many workloads. https://lnkd.in/gHrNPfSZ How do the costs compare of implementing a low-latency write-ahead-log (WAL) on S3 Express One Zone and one implemented as a State-Machine-Replication (SMR) system (such as Paxos/Raft/Kafka)? I built a cost model for both to find out.

4

5.2 greenfield (act 2) (See https://lnkd.in/eP3SxRxm for act 1) Most moving parts were relatively small and isolated components running on autoscaling groups of EC2 instances in different regions. We had most challenges with 1. scaling Elasticsearch on relatively small instances, and two reliably firing thousands of events per second. At that time there was no reliable distributed way to create timed events. The closest in functionality was the good old fashioned cron. That wasn't really an option. Our first working implementation was threaded Python. But we ended with some Scala magic no one was eager to touch. It wasn't distributed, but it was extremely reliable. There are now several established serverless frameworks like Knative, Kubeless and Fission. These frameworks offer triggers with cron-like semantics. They also come with failover features due to their distributed (cloud native) nature. The second challenge is Elasticsearch. Elastic was one of the earlier software producers to distribute their software as a Kubernetes operator. The Elasticsearch operator still requires (public) cloud specific configuration, but it is a step closer to cloud independence. The alternative to Elasticsearch for this particular challenge is Prometheus. Both come with excellent graphical UIs in the form of Kibana and Grafana. These are indispensable for developing, debugging and prototyping. The third major component we used was SQS, AWS' queueing service. This has been so important to us it is almost unimaginable to build what we did without it. At this moment I would look at two different solutions: Kafka and KubeMQ. The first looks to be more heavily supported, but the most important criteria would be developer convenience. We would probably have experimented with both before making a choice. And that leaves us with a Amazon DynamoDB alternative, as the rest is straightforward. We have always been looking at Apache Cassandra, even as very satisfied DynamoDB developers. So that would be the first alternative to experiment with. Perhaps this would be a good point to look at Postgresql, as their Kubernetes distributions look very solid. Or perhaps ScyllaDB… We started developing our gateway with the excellent Raspberry Pi platform. Later we moved to Beaglebone. We would for sure start with K3s on these devices. This simplifies development and deployment. Writing this makes me anxious to try to rebuild this system, if only for the monitoring functionality we have used with great pleasure until the end. Another day... With a small-ish team like we had in the first 6 years we would definitely use EKS (AWS' Kubernetes) and several other AWS services like @Route53. The reliability of the AWS cloud, our experience and the presence (regions everywhere) make this the most reasonable choice. If you have more experience with Google Cloud or Microsoft Azure that would make more sense to you. Act 3: https://lnkd.in/eHREXYQd https://lnkd.in/eamFwzB5

7

Yesterday at Devoxx Poland in Krakow, I gave an updated version of my talk, How to Query a Stream. Herewith the tl;dr: Kafka has become the de facto data substrate of modern systems. It provides all kinds of benefits, but at a cost; to wit, it's very difficult to ask questions about what's in a log. There is no one way to solve this problem, so I proposed several distinct options. 1️⃣ You can always Kafka Connect a topic into your favorite RDBMS (what is it and why is it Postgres?), then query the resulting table. This might not scale to huge data sizes, but the infrastructure is something you're more likely to have around already. 2️⃣ If you're a Java shop, you can use Kafka Streams. It's straightforward to make a topic into a table queryable by message key, accessible through the Interactive Queries API. And if it's analytics you want, you can build an arbitrary stream-processing topology that again produces a queryable table. 3️⃣ There are sundry streaming SQL solutions like Materialize, Delta Stream, Rising Wave, and ksqlDB. They provide similar levels of stream processing and queriable materialized views. 3️⃣.5️⃣ It's here that Flink enters the chat! It famously does not expose its state store to be queried, but it's typical for the last-mile serving-layer-ish query solutions we're discussing to act like a key/value store. This means the computation that normally takes place in the Optimize/Plan/Execute flow of a regular database needs to happen somewhere else, and Flink often fills that role. 4️⃣ Cloud data warehouses. A big place to put all your data for the purpose of analytics. Like data warehouses of yore, these contain explicitly structured data. Oddly, the logos of all the common solutions are blue. I heard you like blue. 5️⃣ Data lakes. A big place to put all your data for the purpose of analytics, but for when you want to pretend structure doesn't matter for a little while. Open table formats like Iceberg are emerging because structure never doesn't matter. 4️⃣ and 5️⃣ are converging (in principle, it not in implementation) into the data lakehouse—a reflection of the decades-long struggle with the schema/no-schema question. 6️⃣ Real-time analytics. A specialized category of analytics infrastructure that ingests directly from Kafka and makes results available at extremely low latency and extremely high concurrency. A more and more necessary part of the stack as analytics escape the dashboard and become part of the end-user experience. We are collectively in a period of exploration of untamed territory between Kafka at the base of the new stack and the differentiated application code at the top of the stack. That code at the top should be the focus of most of our efforts as developers. The application stack was well-defined in the client-server/monolith era, but in the new streaming era, so much is up for grabs and yet to be defined. All cliches aside, it's a very exciting time to be a developer.

467

16 Comments

great advice for learning anything in life for reference, 10 000 hours is: 2 years = 14 hr/day 5 years = 6 hr/day 7 years = 4 hr/day 10 years = 3 hr/day

5

1 Comment

Virtual Threads greatly improve the application throughput and overall responsiveness of Java applications. Just need to be careful about delegating the right kind of operations to Virtual threads - typically less CPU intensive and almost stateless in nature. For operations requiring external network/service calls, a traditional thread is still the way to go for resilience and performance both. True benefits will start to trickle in once #JakarteEE servers like #Payara and others start using them internally. #java #virtualthreads #jdk #jvm #jakartaee #javaee

2

Coolest tool I've found lately. It also has a Neovim plugin if you want to do it live in a buffer. Text UIs are simple useful if your development flow resolves around terminal editors and multiplexers. https://k9scli.io/

4

2 Comments

This is happening! Talk #1: "Migrating Vinted's data pipelines to the cloud: what we learned and what's left to know" by Me (Vinted) Talk #2: "Serverless ETL pipelines with Apache Beam and GCP Dataflow” by Romualdas R. (altic) Talk #3: “Practical learnings from experimenting with Vinted's recommender system” by Audrius Užkuraitis (Vinted)

28

Clojure ❤️ Choreographic Programming: go learn something entirely new at the Heart of Clojure conference this year. Choreographic Programming is in the house w/ the brilliant work on Klor presented by Lovro Lugović and Sung-Shik Jongmans! If you were waiting for a LISP language to try choreographic programming, you've just run out of excuses. 😀 https://lnkd.in/dnCTPTfB #lisp #choreographicprogramming #clojure

9

Gave a talk at work on "Dependencies" today, trying to externalise and make explicit some of the background awareness that shapes my everyday practice as a software developer. It takes time for this awareness to develop. I remember the first time someone talked to me about how Gradle uses a topological sort over a graph of dependencies to determine a build order for tasks, and what a revelation that was. I remember the first time I was shown how to get IntelliJ to scan a Java project for cyclic dependencies between packages, and coming to understand why one might want to avoid them and how to unpick them if they arose. Now DAGs (directed acyclic graphs) are a comfortable piece of my mental furniture, but I don't know that more junior developers are often directly introduced to them - they might come up as a data structure with associated algorithms in CS courses, but is it explained that they're fundamental to the way compilers and build tooling assemble working software from many related pieces? Many developers are aware at some level that transitive dependencies can balloon out into bloat (and an ever-expanding attack surface for vulnerabilities), but how many can articulate this clearly? A large part of my practice as a Principal Engineer is mentoring individuals and looking out for knowledge gaps in teams. It can be hard to recover a "beginner's mind" and remember what it was like not to know (or to be only vaguely aware of) something that became second nature to you over a decade ago, but it forces you to make your implicit knowledge explicit - which also gives you opportunities to reflect on it. Teaching is learning: you go back and retread old paths, and are confronted again with long-forgotten questions. Like everyone at the moment, I wonder what the impact of LLMs is going to be on the way developers acquire the background awareness and instincts that guide accomplished practitioners. Will they help to make implicit knowledge explicit, or will they paper over gaps in knowledge and obstruct the process of learning?

12

2 Comments

Llamas need longer necks; good thing Abacus.AI is here to ensure you get the longest context on your open source models!

13

What does an open source streaming process have to do with a novelist from Prague? Realized I had more than one mention of Kafka on my book shelf. I can’t recommend Designing Data-Intensive Applications enough. It provides a good foundation for topics like kappa architecture, streaming architectures, and more involving Apache Kafka. But books aside, my go-to for all things event driven architecture has been United Manufacturing Hub (UMH) Great overview of some of the core concepts to event driven architecture here: https://lnkd.in/g8UHCMNX

24

2 Comments

I frequently see posts about Scala/ZIO, and I feel nostalgic. Today, I asked ChatGPT to "create" ZIO in C++... :-) I had to make a few minor changes, though, to avoid memory access violations when we have flatMap followed by map. The result of the run in Visual Studio 2022 is this: Success: 94 Error: An error occurred

75

6 Comments

🚀 Introducing 'Claude François' 🕺—The AI Code Reviewer with Style! A few weeks ago, we were invited to the first Menlo Ventures/Anthropic Builder Day to craft something remarkable with Anthropic’s flagship Claude 3.5 Sonnet LLM. After some post-hackathon polish, we're thrilled to share 'Claude François'—the AI Code Reviewer with style! It’s just a quick 4-hour hack demonstrating how easy it is to leverage Crossing Minds' RAGSys in-context learning technology with an LLM like Claude. Just give it any Github Pull Request (PR), and it generates an expert code review for you! Try it now: https://lnkd.in/ezfRpbMc 🔍 How Does It Work? We leveraged our patent-pending RAGSys In-Context Learning (ICL) technology to build a database of Pull Requests (PRs) from the Keras GitHub repository. The primary maintainer of Keras is none other than François Chollet, the creator of Keras and an AI luminary known for his succinct, expert code reviews focusing on real issues rather than superficial code hygiene. Here's the magic: Active Learning & Fine-Tuned Embeddings: RAGSys pairs code diffs with François's reviews, using active learning to train custom embeddings optimized for generating "fchollet-level" code reviews. Intelligent Retrieval: When you input a GitHub PR, RAGSys taps into our industry-leading recommendation engine to retrieve the most relevant examples—not just the most similar ones, but those optimized for maximum mutual information. Self-Reflection: Finally, we employ sophisticated self-reflection loops to unlock the LLM's reasoning capabilities, and extract more information from the in-context examples, delivering the best possible code review. 🌟 Why Is This Exciting? This demo showcases the power of our RAGSys technique for In-Context Learning and Domain Adaptation. Imagine replacing Keras and François Chollet with your own in-house code reviews and standards! Or apply RAGSys to any domain-specific task with input-output pairs—classification, translation, summarization, labeling, search, recommendations, and more. 💡 The Future of AI with Crossing Minds RAGSys in-context learning is faster, cheaper, and more flexible than traditional domain adaptation approaches like fine-tuning, while producing more accurate and consistent outputs. At Crossing Minds, we're leading the way in AI and LLM innovation. 👉 Try It Out! Take 'Claude François' for a spin on your own codebase and experience the difference. We'd love to hear your feedback! If you’re curious to learn more about RAGSys, check out our recent webinar (https://lnkd.in/eF7BkvqZ), or reach out to me directly. Let's explore how this technology can transform your workflow. p.s. in case you're wondering, the name and 🕺 logo are a playful nod to a certain iconic French musician. https://lnkd.in/eEVzgTnx #GenAI #AnthropicBuilderDay #LLM #Anthropic #Claude #Hackathon

42

5 Comments