Péter István Fülöp

Gülçin Yıldırım Jelínek: Maintaining Postgres for Modern Workloads 🚀 A Geeky Rendezvous with PostgreSQL: Modern Workloads Edition 🚀 Hey PostgreSQL aficionados! 🐘 Ever wonder what makes software not just functional, but outright maintainable? I had the pleasure of diving into this very topic with the inimitable Gülçin Yıldırım Jelínek on the Maintainable Podcast, hosted by Robby Russell. Imagine juggling modern workloads with the grace of a seasoned circus performer using Postgres as your trusty, reliable, and slightly quirky assistant. Spoiler alert: it involves more than just a well-timed Ctrl+C. We delved into the mysterious art of keeping our beloved databases in prime working condition, even when they grumble under the weight of today's demands. Think of it as giving your beloved vintage car a tune-up before a road trip, except this car handles all your data and sometimes cries for a VACUUM. For those who missed it, the podcast features secrets and snippets you won't find in any manual

1

A few months ago, we shared our collaboration with an online higher education institution. Our Senior MLOps Engineer, Bart Grasza, Data Scientist, Kamil Abram, and Senior Data Engineer, Mateusz Kijewski developed an MLOps platform that provides a foundational template. 🙌 This platform integrates MLOps best practices, automates the setup and update processes, and allows for seamless workflow adaptation across different environments. You can now see this platform in action, presented at the Databricks Data Summit: https://lnkd.in/dUbJrbVi 

23

Hello there! Here is a little story about narrow-angle optics for today. We've been busy making our regular queries run fast on #ORC tables in #Vertica. And we achieved some good stuff (you can read it in the posts below). But the strike came out of the real-life queries that had something distinct from the regular ones. When the real user digs in to discover the data in your table, he starts with something like `select * from your_table where random_field = random_value`. In the case of a rare random value from an unsorted column, we end up reading all the data we have. What a bummer. It can be hundred billion rows to return like 10 records. As stupid as it sounds. A single query can significantly increase pressure on our S3 in one shot. So, a little laziness in column reading is what we do now. We just read the predicate columns first. As a result, we reduced the spikes by half by avoiding unnecessary reads. Here is a day before and a day after in the life of our object storage. Use metrics to keep your perspective wide. 

14

1 hozzászólás

Is Lambda cold start still an issue in 2024? According to analysis by AWS on production Lambdas, only 1% of Lambdas experience cold start. In this episode of The CloudHandbook, we will be discussing everything about Lambda cold start. In this issue: - How, why and when cold start occurs - Some tips and tricks for minimizing it - and more Read more here: https://lnkd.in/gk2_8uBk #lambda #serverless

8

1 hozzászólás

Fine-tune Florence-2 on any task 🔥 Today we release a notebook and a walkthrough blog on fine-tuning Florence-2 on DocVQA dataset Andrés Marafioti Piotr Skalski 🔖 Link to the blog, notebook and demo are in the comments 💬 Florence-2 is a great vision-language model thanks to it's massive dataset and small size! 🤩 This model requires conditioning through task prefixes and it's not as generalist, requiring fine-tuning on a new task, such as DocVQA. 📝 We have fine-tuned the model on A100 (and one can also use a smaller GPU with smaller batch size) and saw that model picks up new tasks 🥹

460

19 hozzászólás

I might be late to the train, but have you heard about SQLGlot? It's an open-source library that parses SQL (https://lnkd.in/dqgdS53j); it's used in many commercial applications (dbt cloud included for column-level lineage), SQLMesh uses it, and many more usages. I started to play with it more actively at work since we have a lot of SQL code, and we're looking to enhance our current data lineage solution, which is built in-house. "First I was afraid, I was petrified" 🎵 , but after a quick try, I started to get more into it, and oh boy, how fun it is to use it. It's based on generators (some people would have to adjust their habits working with already computed lists). It works better on very complicated queries compared to our test with the OpenLineage parser. I have a feeling that it's going to become a blog post about it. --- Stay in the loop for the latest insights from Uncle Data! * Click the notification bell icon 🔔 to receive updates on Linkedin * Subscribe to my Newsletter on Substack ✏ * Listen to Uncle Data Podcast ⌨️ Listen. Learn. Connect --- #sql #parsing #sqlglot #lineage

4

I see this picture circulating a lot so I cannot just let it fool the members of this highly respected community. Programmers hate to talk to clients. This is why the industry invented project managers and business analysts. They are safe, not programmers. Programmers are actually the most unsafe at this moment, because LLM-generated code can be automatically verified for validity by running it and applying to unit tests. Unit tests can also be generated in thousands by an LLM. So, instead of denying that the profession is being challenged and sharing funny memes, why wouldn't someone start thinking on how the programmers can transition into doing something LLMs cannot do out of the box? Put your ideas in the comments.

1 333

198 hozzászólás

PostgreSQL is the database that quietly ate the world. Due to its simple and permissive license, community backing, and extensibility, PostgreSQL has silently found itself at the center of modern software development. I'm excited to share my new post that looks at the ubiquity of PostgreSQL, its history of slow and steady improvements, and why the community loves it so. I was surprised to see such positive responses from everyone I spoke with. I also like shining a light on a stable, open-source community project that's just been doing its thing for years. And years. And years... Thanks a lot to everyone who connected with me on this one! Featuring input from the following folks, in order of appearance: - 🐯 Ajay Kulkarni, CEO and co-founder of Timescale - Ryan Booz, PostgreSQL advocate at Redgate Software - Craig Kerstiens, Chief Product Officer of Crunchy Data and curator of the Postgres Weekly newsletter - Christos Karamanolis, a fellow at Broadcom's VMware Cloud Foundation - Tanmai Gopal, CEO of Hasura - Dave Stokes, technology evangelist at the database specialists Percona - Karthik Ranganathan, founder and co-CEO of Yugabyte Thanks a lot to LeadDev for publishing and to Scott Carey for your keen editorial guidance. #postgres #posgresql #database #sql https://lnkd.in/eCRvi4Cw

117

3 hozzászólás

📢📢📢 We've all been there - staring at a wall of monitoring dashboards, trying to make sense of the data sprawl. You've got metrics piling up in Prometheus, logs accumulating in Loki, and traces scattered across Tempo. It's like having a bunch of puzzle pieces, but no idea how they fit together. 🧩🧩🧩 The problem with Kubernetes observability today is that while we're really good at collecting and storing data, we're failing at the most crucial part - making sense of it all. Let me break it down for you: 🗃️ Data Silos Everywhere: Metrics, logs, and traces are all isolated in their own little worlds, making it nearly impossible to correlate them and get the full picture. 🚧 Lack of Unity: These observability tools don't really talk to each other. Each one operates in its own silo, giving you just a tiny glimpse into what's happening. 🌀 Complexity Overload: Trying to integrate and analyze data across multiple tools is like trying to solve a Rubik's cube blindfolded. It's a complex, time-consuming nightmare. 🔎 Analytics? What Analytics?: Let's be real, tools like Grafana are great for visualizing data, but they're not built for deep, meaningful analysis. Linking metrics, logs, and traces is possible, but it's a slow and painful process. 🚀 Missed Opportunities: With data scattered all over the place, we're missing out on valuable optimization opportunities. Connecting the dots between a CPU spike, a log entry, and a trace could hold the key to resolving issues faster, but it's just too hard. We've come a long way in terms of data collection and storage, but we're still stuck in the Stone Age when it comes to analysis. 🏺 We need observability tools that can seamlessly correlate all that data in real-time, giving us a comprehensive view of our Kubernetes environments. It's time for the observability landscape to evolve. 🦖➡️🦕 We need solutions that not only store data efficiently but also make it easy to analyze and gain insights from metrics, logs, and traces – all in one place. Only then can we truly unlock the potential of our Kubernetes deployments and keep those applications running smoothly. 🚀 #KubernetesObservability #DataAnalytics #DevOps

11

3 hozzászólás

First Python 3.13.0 release candidate is out. Here are some notable features: > 𝐎𝐩𝐭𝐢𝐨𝐧𝐚𝐥 𝐆𝐈𝐋: Deactivates the Global Interpreter Lock, allowing more concurrent execution of threads. I explained the difference between freeing GIL and GIL-per-interpreter here: https://lnkd.in/e_Ekp7-r. > 𝐉𝐮𝐬𝐭 𝐢𝐧 𝐭𝐢𝐦𝐞 (𝐉𝐈𝐓) 𝐜𝐨𝐦𝐩𝐢𝐥𝐞𝐫: An experimental JIT that lays the groundwork for significant improvements to performance. I have written about this type of "copy-and-patch" JIT and shared a more detailed blog post here: https://lnkd.in/edDhME3J. > 𝐓𝐢𝐞𝐫 3 𝐬𝐮𝐩𝐩𝐨𝐫𝐭 𝐟𝐨𝐫 𝐢𝐎𝐒: Now Apple's iOS is officially supported. As well as Android support (PEP 738) is being worked on. There's going to be a lot of fun. #python #software

3

3 hozzászólás

🔹Make the PySpark fly 🔹Magdalena Nowak Writing efficient Spark applications with Python can be surprisingly challenging. With PySpark library evolving fast many of the best practices aren't well-known, especially among Data Scientists. They tend to stick to the methods and workflows they know from more known libraries like Pandas, treating Spark code similarly - which often ends up being inefficient. Moreover, with most tutorials focusing on the basic features of the PySpark library, there is little knowledge about new concepts that make our lives easier and our data pipelines faster. This talk will help you progress beyond .groupBys and .counts. Watch the entire speech ➡️ https://lnkd.in/dJ_6UsxC

6

There's a common myth in the cloud consulting business landscape. In the quest to offer end-to-end solutions, many companies attempt to juggle more balls than they can. They often forget they are not circus performers - if you think you are, consider this a friendly reminder that you’re (probably) not. Cloud Consulting boutiques love to brag about their in-house capabilities to manage the entire customer journey from 0-100. On paper, this sounds amazing - and many customers are falling for these flattery terms: “We are the only cloud partner you need” “Comprehensive control and seamless integration” “All your needs handled under one roof” I totally disagree with that approach. I know my data stuff well but e.g. have no idea how to set up a proper landing zone like Manuel, and no idea how to make CI/CD a joy like Johannes. And now, I also know the cost guru of the cloud - if a shiver runs down your spine every time you look at your AWS bill, you know what to do ➡ Cristian Cristian Cristian So, instead of trying to be a jack of all trades: Focus on what you do best and partner for the rest. Did we forget how niche specialisation made the “Mittelstand” the backbone of our economy?

10

5 hozzászólás

New blog post: Access 150k+ Datasets from Hugging Face with DuckDB This blog post, co-authored by the Hugging Face and DuckDB teams, describes how you can use the hf:// prefix in DuckDB to access datasets in Hugging Face repositories. Read more at https://lnkd.in/dYf2PPKk

318

1 hozzászólás