Benny Chow

Lately, we have observed a transition from Cloud Databases(Snowflake etc) to Iceberg OpenLakehouse format. Ambaricloud team is working on Snowflake to Iceberg Automation and end-user training. Why: Low-cost computing. SF is still a favored engine for Gold layer data. Less stickiness. More engine options to run on Iceberg tables. Cloud storage Parquet files Iceberg table format Catalog Status: Polaris -- Started Walking. Hopefully, it will Jog or Run soon! UnitCatalog -- Infant Glue -- Safe bet for now. Comments? Snowflake Cloudera Dremio Upsolver Databricks

8

1 Comment

Same Chicken (Iceberg), different Curries (Engines) Cloudera: Read Snowflake Data from Polaris External Sync Catalog I am able to "read" and integrate Snowflake data through Polaris with other data in Cloudera Spark. Cloudera Snowflake

17

If you are ingesting data from Kafka/Kinesis into Snowflake you should watch this short video by Charles Tan and Shawn Gordon on how you can use DeltaStream to shift left your workloads to reduce cost and latency! #Flink #Kafka #Snowflake #ShiftLeft https://lnkd.in/gnZ45MTa

13

Databricks SQL Serverless is now generally available on Google Cloud! Unlock instant, elastic compute and elevate your data analytics with faster query performance and reduced costs. Focus on delivering insights that drive business value, rather than managing infrastructure. Available across 7 GCP regions and 40+ regions globally, SQL Serverless brings you the best of both worlds—unmatched performance and simplified analytics. Learn more here: https://lnkd.in/eZCu4VAJ

2

✨ If you have Databricks, do you need Snorkel AI? If you plan to customize #LLMs with your own enterprise data the answer is an easy yes. You need high quality data to build production-quality AI, but most unstructured enterprise data is inaccurate, incomplete, and inconsistent. Often you have a lot of data for common scenarios but very little for high-value edge cases (obviously!). Good news: Snorkel Flow integrates seamlessly with Databricks at every stage of the AI development workflow, from accessing data & models to deploying production-quality applications via Unity Catalog. For a guided walk-through check out this 5min demo and learn more at https://lnkd.in/gTiQ4wD8 #AIInnovation #ArtificialIntelligence #dataintelligence

31

7 Comments

If you ingest data from Kafka/Kinesis into Databricks, you should watch Charles Tan and Shawn Gordon's short vide on how to shift left your data pipelines to reduce cost and latency! #ApacheFlink #Kafka #StreamProcessing #RealTimeData https://lnkd.in/ggMsi9KU

14

1 Comment

Databricks disclosed that they are expecting to cross $2.4B in revenue run-rate growing ~65% Y/Y by H1’25 (Jan FYE). This was alongside top-tier metrics, including 80%+ GMs, 140%+ NDR, and a multi-product story where ~50% of customers use 6+ products (incl. their Data Warehouse product which generated $400M in revenue up 4.0x Y/Y). Given their accelerated growth, Databricks is among the few scaled software companies, alongside hyperscalers, capitalizing on the AI boom. Equally noteworthy is their market share gain against competitors. Considering their growth trajectory, we explored various scenarios to determine when Databricks will surpass Snowflake in scale. The question is not if, but when, and we believe it is sooner than most anticipate.

1,010

26 Comments

🚨 Databricks Game Changer 🚨 Row and Column Level Security for Unity Catalog -- For the first time on any platform, you can apply fine-grained access controls across Apache Spark, data warehousing, AI, and real-time processing—all within a single platform.

11

Pyiceberg: Polaris Iceberg to Delta. PyIceberg, DuckDB, and Daft are not just secondary players in the data ecosystem. These tools deserve recognition for the essential role they play—they help reduce compute costs and optimize workflows. They deserve our respect! ✅

3

1 Comment

Should be a very interesting webinar if you have the time to attend. A look at how to leverage Kumo.AI GNNs to easily enable AI on your data stored in Snowflake.

11

In case you missed it, we have now released the second issue of our Confluent Chronicles – a fun and entertaining take on the fundamentals of stream processing, Apache Flink, and Confluent! We had such great feedback on the first issue that we couldn’t just stop there – you can check out and download the latest edition here: https://lnkd.in/gEFxA7af Would love to hear what you think of it!   #ApacheFlink #ApacheKafka #Confluent

39

1 Comment

🎉 The Snowflake Model Registry is now available for general use through the Snowpark ML (snowflake-ml-python) package version 1.5.0. 🚀 With the Snowflake Model Registry, you can securely store, manage, and use machine learning models in Snowflake, where your data is already stored. The registry supports most popular types of Python ML models and can be accessed from both Python and SQL. 💻rom both Python and SQL. The registry supports the following types of models: ⚫ Snowpark ML Modeling ⚫ scikit-learn ⚫ XGBoost ⚫ LightGBM ⚫ CatBoost ⚫ PyTorch ⚫ TensorFlow ⚫ MLFlow PyFunc ⚫ Sentence Transformer ⚫ Hugging Face pipeline ⚫ Other types of models via the snowflake.ml.model.CustomModel class https://lnkd.in/geGsmYUZ #snowflake #infostrux #datasuperhero #snowflake_advocate

10

Fast just got faster. Our enhanced Redis Query Engine on Redis Software is generally available today. It boosts speeds up to 16X faster for existing Redis workloads – faster than any other vector database we tested. Your search capabilities just got a major upgrade. Ready to see how we compare? Check out our benchmarks and see why we're leading the pack in data and AI infrastructure. https://lnkd.in/ejwmiTyJ #RedisAdventures #VectorDatabaseChampions #RedisQueryEngine #DataSpeed #Realtime #dataplatform #FastestDatabase #DataRevolution

29

Come meet us at the Snowflake summit in SF next week and learn how we're applying specialized models (large, small, expert, etc.) to enterprise data giving your teams super data powers. #SnowflakeSummit #DataCloud #SanFrancisco #Networking #Data #BI #AI #GenAI #LLMs

28

2 Comments

Using Kumo.AI in your Snowflake environment is like peanut butter and chocolate - better together! Check out this webinar to learn more.

10

1 Comment

As part of Databricks' LakeFlow Council, I've had the opportunity to see the work that is happening being the scenes on LakeFlow Connect, Pipelines, and Jobs. From what I see, "Pipelines" is what DLTs should have been, and MUCH more. About 80% of what I'd like to see appears to be getting implemented. Future appears bright for this one thus far! PS: Shared this with Bilal Aslam's permission.

90

10 Comments

** New Blog Post Alert ** Check out the latest blog post at Infostrux Engineering Blog, which discusses how to create Snowflake Network Policies. Including the latest user-level policies https://lnkd.in/gNiYKXiv #snowflake #datasuperhero #snowflake_advocate #infostrux

26

1 Comment

❄️ Snowflake Is in Talks to Buy Reka AI for $1 Billion Reka supports multimodal input (image, video w/ audio, text, and documents such as PDFs)

2