It's a new week, it's a new MaterializedView newsletter. If you're into data infra I *highly* encourage reading this one. It's on DuckDB and data warehouses. As always Chris Riccomini shines with a clear, thoughtful and pragmatic analysis. On Chris' writing, DWHs, and Postgres: First, I'm new to the data infra space. When I first spoke to Chris 13 months ago, I knew nothing. We wanted to build an Elasticsearch competitor on Postgres, and as Chris pointed out we were "green." We had everything to learn. We still have everything to learn. Reading MaterializedView has been an incredible source of pragmatic analysis on the data infra space. In a space where cargo culting is prevalent, Chris keeps it real. I attribute a lot of our learning progress to Chris' advices. Seriously, read it. Building a startup is like finding a needle a haystack. The world is filled with tools, can you find where there's a gap? Then fill the gap and start pushing against complacent solutions to widen the gap into a big opportunity. Data infra in particular is a space where every tool does 90% of the same thing, but the 10% that's different matters tremendously. That 10% is "the gap". It's where there's room to innovate. In small, unsexy ways which can grow to be big over time. We started building an Elastic competitor because, well, Elastic has many flaws. It's great, but it has many flaws. So many investors have told us to build a data warehouse. The analytics market is so big, they say. "Phil, ES is 10b$, Snowflake is 50b$!". But as Chris points out: DWHs are good. People love them. They're expensive and good. They're expensive *because* they are good. We decided to focus on a "smaller" but more painful problem instead. We also played a lot with DuckDB. As Chris points out, it's not a DWH. And competing on price is hard. Meanwhile, Postgres is eating the small workloads market because, well, people love it and it's so extensible. To build a Postgres product, you need it to be 100% Postgres. Not 99%, but truly 100%. We learned this the hard way over the last year. We've kept our focus narrow -- a better Elastic --, and are soon releasing fast on-disk analytics in Postgres. The first version will come out with v0.12, this week. And we're building it in a true Postgres way. No shortcuts. It won't be a data warehouse. It won't be DuckDB-based. It's still part of our quest to build a better Elastic competitor. We're calling it fast facets. To stay up to date, follow our main repository, `paradedb/paradedb`. And to get a truly fresh read on the data infra space, read @materializedview. Chris' writing distills products and market dynamics into their core principles, and those are how teams and products win.
Tarik Remila
1mo
🔥
Building Denormalized, the easiest way to work with real-time data
1moWell said -- link for the lazy https://github.com/paradedb/paradedb