Sergey Novgorodsky

See how the #ElasticSeachLabs team is optimizing scalar quantization for vector databases, unlocking performance gains for int4 quantization in Lucene: https://lnkd.in/eDUFF5_q #VectorDatabase #Lucene

1

If you're selling to: ML teams, Data Sci teams, Data Eng teams, Infra/platform teams, Devops teams, or similar niches... And your go-to-market team lacks deep technical knowledge, You might fall behind your competitors faster than you expect. Let's break it down simply: - Sales: Without a solid technical foundation, Account Executives may take longer to get up to speed, making it harder for them to meet sales targets on time. - Solutions/Pre-Sales: Senior members often handle most of the technical challenges due to skill gaps, which cuts into team efficiency. - Marketing: Weak technical messaging doesn't grab the attention of key prospects, weakening your sales pipeline. - Customer Success: Poor problem-solving and a growing backlog lead to customer frustration and churn. I understand many of you believe these skills can be learned, and I won't argue with that. But then, the question is: "How long can you afford to wait?" As you ponder this, remember: While you're planning training sessions, your competitors are already seizing market opportunities. Let's be realistic about the barriers to entry and the learning curves here. P.S. Need help hiring GTM talent with technical acumen, dm me!

93

19 Comments

ARR is dying as a useful metric. So what should replace it? Two main thoughts: 1. The math the makes cloud works is changing 2. People need to start using their brains versus relying on historical benchmarks/metrics https://lnkd.in/gfVMTsFa

5

InfluxData dropped their latest 3.0 architecture update. It's interesting how they're leveraging Apache Arrow for in-memory processing and Parquet files for storage now. Whats interesting is their new approach to handling real-time data - they've got this smart ingester that makes data queryable before it even hits disk storage. However, I'm skeptical about how their custom partitioning will perform in high-pressure environments, especially when dealing with time-sensitive workloads (I been here before, have you?). Still, the multi-tenancy support looks promising for enterprise implementations and eager to give this drive. Anyone had a play yet? #observability #timeseriesDB #dataanalysis #influxdb

2

In this video, we look at the internals of Facebook's graph datastore. This system, called "The Associations and Objects (T.A.O)", is used across Facebook, Instagram and WhatsApp to represent objects and relations in memory. The graph is built over Memcached. Facebook engineers store petabytes of data in this graph and have scaled it to 1 billion queries per second. I have covered this system in detail at InterviewReady during live classes. Glad to see the enthusiasm for research papers :D Cheers! #SystemDesign #Facebook #Graph https://lnkd.in/dGdM-nb7

49

Peter, Percona founder and Coroot Co-founder, discusses the common pitfall of over-collecting logs and metrics in Kubernetes environments. He offers two key pieces of advice: 1. Be selective in data collection 2. Utilize efficient storage technologies Peter highlights that some organizations have "almost as much infrastructure supporting collecting and processing telemetry" as their main application, which he finds "absolutely bizarre." He emphasizes that modern systems with data compression and vectorized processing can be "at least 10x more efficient" than older systems for managing Kubernetes logs and metrics. Watch the full interview: https://ku.bz/GMQr-2gvT

7

The NVL function was once a favorite of mine that I to return a not null value when my input variable was null . Years ago, I learned the hard way how this function, if not carefully used, can be detrimental to your code performance.   I turned to COALESCE as a better substitute among other functions. In fact, COALESCE is a short-circuit function and NVL(expr1, expr2) is not. NVL will always evaluate expr2 whether expr1 is null or not. Unfortunately, most blogs, posts, stack flow, training doc fail to point out this characteristic .   I usually advise developers against using NVL due to its performance implication. For existing code when provided with the opportunity, replace NVL with COALESCE. However, be mindful of how both functions handle implicit conversions.   Learn from the source about COALESCE https://t.ly/S86gb and NVL https://t.ly/QRgo5 #oraclesql #plsql #oracle

3

[Blog][Mastering Apache Pulsar with Python SDK: A Comprehensive Guide] This comprehensive guide aims to introduce the Apache Pulsar Python SDK, dive into its core concepts, and provide detailed code examples to help you leverage its full potential in your enterprise environment. https://buff.ly/3zEimkC

1

💡 Optimizing Costs with Dragonfly: A Technical Breakdown 💻 For technical teams looking to maximize performance while minimizing cloud infrastructure costs, Dragonfly offers significant advantages. By cutting down memory overhead, leveraging CPU-specific optimizations, and using a shared-nothing architecture, Dragonfly is designed to be resource-efficient while maintaining high throughput. Whether you’re scaling horizontally or aiming to optimize performance, Dragonfly can help you reduce operational costs without sacrificing speed. In this blog, we break down the technical strategies that Dragonfly uses to cut costs and improve efficiency. 🔗 Read more here: https://hubs.la/Q02RDNVr0 #CloudOptimization #TechEfficiency #DevOps #DatabasePerformance #Dragonfly#CostReduction

3

Loft Labs has announced a major update to its platform: vCluster Platform v4.0, which introduces externally deployed virtual clusters that enhance flexibility and offer new options for Kubernetes users aiming to optimize their environments. In this episode, Lukas Gentele, CEO of Loft Labs, shares insights into the company’s progress, their role within the Kubernetes ecosystem, and new and upcoming features. Gentele highlights the company’s focus on cost optimization and multi-tenancy, empowering organizations to build efficient, scalable Kubernetes environments. Gentele says, “I think we're becoming a key piece of the Kubernetes puzzle, enabling people to make the most out of their Kubernetes architecture.” #kubecon Cloud Native Computing Foundation (CNCF)

12

AWS Sagemaker is 40% more expensive than EC2 instances with suboptimal developer experience and technical limitations. Here's a blog comparing TensorFuse (YC W24) as a Sagemaker alternative. https://lnkd.in/gewcPNU2

21

2 Comments

🙌 In the latest 8.7 release of SingleStore, we are happy to introduce automatic schema and pipeline DDL inference that simplify the initial setup process, enabling users to quickly start leveraging their data and reduce time to insights for customers. Why is this important for customers? Read the blog to find out: https://bit.ly/45LrqQP #data #SingleStore #database #devs

12

1 Comment

How can you handle long pauses of your data in #ApacheFlink? Since your stream is stopped, setting an idle timeout won’t work. In the latest video in his Apache Flink in Action series, Dan Weston explores one possible solution. Give it a watch and then share your solution in the comments! Watch it here ➡️ https://lnkd.in/gjmAv46B

50

1 Comment

"From PostgreSQL to Lakehouse using Hudi Streamer, Debezium & Kafka" Soumil S. joined us last week for the Episode 3 of Lakehouse Chronicles! He presented an amazing demo to tackle a real-world problem (CDC) of bringing data from operational sources to a lakehouse using Hudi Streamer. Here is a Github repo & Youtube recording 👇 🌟 Github: https://lnkd.in/dRarunaT 🌟 Youtube: https://lnkd.in/duhCKMed #dataengineering #softwareengineering

31

2 Comments

❄️ Snowflake <> Kafka, in minutes. It is unwise to publicly expose your Kafka brokers to the Internet. It paints a giant public target on your most sensitive business data. Teams that value the security of their customers keep their Kafka private. This however creates the challenge of how to move data between private Kafka and other systems like Snowflake. Ockam makes it trivial to create end-to-end encrypted, very high-throughput, secure portals to private Kafka brokers. Your Kafka could be anywhere – in a data-center, in a private VPC, in any cloud, in a private kubernetes cluster, in a BYOC deployment, etc. Here is an example: The Snowflake team shared with us how many of their customers are unable to securely and easily move insights from analyses in Snowflake to other operational systems. These enterprise customers have tried all sorts of janky ways to make it work – from punching insecure holes in their firewalls to elaborate multi-region private links that took them many months to setup. We worked with Snowflake to create a simple Snowflake Native App. This app uses Ockam to securely connect Snowflake with private Kafka. It then enables Change Data Capture (CDC) from any Snowflake table to any Kafka topic. A few simple clicks and you have insights streaming from Snowflake to private Kafka anywhere. Safely and securely. https://lnkd.in/gEAiGgPn 👈 More here

55

2 Comments

I’ve been working on this idea of Indexless Observability. A big problem of Observability is cost and performance. The solution to performance is to maintain a giant, complex index on high performance storage. This in turn makes cost much worse, driving people to ingest less data to manage it, which defeats the whole point of the exercise. Indexless Observability is the attempt to see how far you can get without a traditional indexing mechanism, or an SSD. What capabilities can you maintain with a smarter approach to field tracking and the lowest cost storage available, I.e S3 object storage. Is this something you guys would like to see?

31

5 Comments

Handling missing data points is not as easy as it sounds for #streamprocessing "Fancy" mechanisms like "watermark" won't show you new result if the count is 0 for a tumble/hop window. Check the nice video from Confluent. Dan mentioned a solution to add special events in the data stream(say Kafka), for 0USD transactions. Well, it works but will impact your COUNT/SUM/AVG, unless you add many filtering. I would suggest use DataGen or RandomStream in Timeplus as a heartbeat stream, UNION 2 streams together, run TUMBLE window with GROUP BY window and type, then you will get count=0 and fire alerts. SQL like this: WITH cte AS (SELECT 'real' as type FROM stream1 UNION SELECT 'datagen' as type FROM datagen) SELECT window_start, type, count(1) as cnt FROM tumble(cte,1m) GROUP BY window_start, type HAVING cnt=0 Leave comments below if you have any thoughts, or join timeplus.com/slack for more discussions.

8