Giran Moodley

Interested in a preview of Airflow 3.0 (Airflow 2.0 was 4 years ago). Take a look at Julian LaNeve talking about the latest release and a preview of what is coming. https://lnkd.in/eFPvwbHZ

27

Great new article by Li (Luke) Yu on how to use #ray to perform feature extraction for #llms on #databricks: https://lnkd.in/d5qv3Xmk Key insights include: 🔧 How to setup and configure a Ray cluster on Databricks with #GPUs 📘 How to overcome tight context windows with various summarization approaches

50

4 Comments

#DatabricksSQL warehouses provide up to 12x improved price performance over standard interactive clusters. You can now use Databricks Notebooks on SQL Warehouses to write and schedule Git-backed, multi-statement, and parameterized SQL👇 https://dbricks.co/4agOeJo

2

❄ Snowflake 𝐋𝐚𝐮𝐧𝐜𝐡𝐞𝐬 𝐀𝐫𝐜𝐭𝐢𝐜 𝐄𝐦𝐛𝐞𝐝 – 𝐓𝐡𝐞 𝐖𝐨𝐫𝐥𝐝'𝐬 𝐇𝐢𝐠𝐡𝐞𝐬𝐭 𝐑𝐚𝐧𝐤𝐞𝐝 𝐏𝐫𝐚𝐜𝐭𝐢𝐜𝐚𝐥 𝐓𝐞𝐱𝐭 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐔𝐬𝐞 𝐂𝐚𝐬𝐞𝐬! 🚀 We're thrilled to introduce the 𝐒𝐧𝐨𝐰𝐟𝐥𝐚𝐤𝐞 𝐀𝐫𝐜𝐭𝐢𝐜 𝐞𝐦𝐛𝐞𝐝 - our latest family of state-of-the-art text embedding models now available! These models are designed to enhance retrieval performance efficiently and are open-sourced under the Apache 2.0 license. 🔍 𝐊𝐞𝐲 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: • 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞: Our largest model outperforms others with a smaller size, offering a significant advantage in retrieval tasks. • 𝐀𝐜𝐜𝐞𝐬𝐬𝐢𝐛𝐢𝐥𝐢𝐭𝐲: Models range from x-small to large, catering to diverse enterprise needs. • 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐜𝐲: Optimized for lower latency and cost, making them ideal for enterprise search applications. 💡 These models integrate seamlessly into your systems, providing top-notch retrieval capabilities with minimal setup. Available now on Hugging Face and soon in Snowflake Cortex. 📈 𝐒𝐭𝐚𝐲 𝐀𝐡𝐞𝐚𝐝: Leverage these models to power up your applications with advanced AI capabilities. 🔗 Read more about this release: https://lnkd.in/dnv25qwp #Snowflake #AI #MachineLearning #DataScience #TechNews

27

🚨 dbt Labs made some hugeee new announcements today at Coalesce 🤯: -Cross-platform dbt Mesh 🕸  -dbt Cloud now supports the Apache Iceberg table format 🧊  -Visual drag-and-drop interface for less technical users to develop in dbt without needing to know SQL 👨‍💻  -dbt Copilot, an embedded AI engine with various user experiences across the dbt workflow 📈  -dbt Cloud integrates with AWS Athena (GA) ✅ & Teradata (Preview) ✅  - AND More (Advanced CI, Auto-exposures with Tableau, a new Power BI integration for the semantic layer, Data health tiles, and more!) 🔥 This all brings together our ethos, 'One dbt', the idea that dbt provides a common unifying framework bringing together people, platforms, and workflows in support of the analytics development lifecycle. https://lnkd.in/gb_hEU9G

28

2 Comments

Streaming Tables and Materialized Views now supports Row filters and Column Masks. If you want to get started quickly with RW and CM : https://lnkd.in/eaNNJpci #Databricks

60

3 Comments

Snowflake Arctic, a new large language model that’s open, enterprise-grade, and efficient, is now available on Microsoft #Azure AI.   This state-of-the-art model uses a Mixture-of-Experts architecture that activates 50% or fewer tokens than comparably sized models. Arctic excels in a variety of tasks, including SQL code generation, sophisticated instruction following, and complex data analysis and is available for unrestricted personal, research, and commercial use.   We’re upholding our promise to deliver the best selection of high-quality models so our customers have choice and flexibility.     https://lnkd.in/g3mxCMfa #SnowflakeArctic#EraOfAI #GenAI

417

7 Comments

Meet Llama 3.2 on Databricks: faster and now multi-modal. The Llama 3.2 release pushes the frontier of enterprise #GenAI with new smaller models for latency + cost-sensitive use cases and larger multimodal models for visual understanding. Customers can easily get started tuning the Llama 3.2 models securely and efficiently on their data. Available today in Databricks Mosaic AI. 

7

Introducing Meta Llama 3 on Databricks! With Llama 3 on Databricks, enterprises of all sizes can deploy this new model via a fully managed API. Create high-quality, production-scale GenAI apps using the best model for your use case while securely leveraging your organization's unique data.

6

Understanding Apache Iceberg's Consistency Model Part 2 - details of concurrency control and data conflict checks to allow Apache Iceberg to handle multiple concurrent writers correctly. https://lnkd.in/dcznGKPX

47

3 Comments

New blog post on understanding Apache Hudi's consistency model using TLA+, with special focus on multi-writer scenarios. https://lnkd.in/dnMHWRv9

59

📣 At Databricks, we want to help customers build more #inference-friendly #llms. Our MixAttention architecture demonstrates that reductions in KV cache size make it possible to maintain model quality while improving inference speed and reducing memory footprint. Read our latest blog post to learn more: https://lnkd.in/gcHxM4DX

133

3 Comments

A new era of enterprise AI has arrived with the introduction of the #SnowflakeArctic ❄️- a truly open, breakthrough large language model. Snowflake Arctic offers top-tier intelligence with unparalleled efficiency at scale and joins the Snowflake model family, which also includes the best practical text-embedding models for retrieval use cases. Discover more about how Snowflake is accelerating AI innovation for all users by clicking the link below. #AI #innovation #technology #artificialintelligence https://lnkd.in/gzKjUHS2

18

We are excited to announce the General Availability of Row and Column Level Security in Unity Catalog on AWS, Azure and GCP! This is your gateway to a new world of unified data security, where you can confidently share data with multiple users and groups, all while maintaining control and ensuring that sensitive information remains protected. Read the full blog post to learn more about how you can enable fine-grained access controls using Row Filters and Column Masks.

2

Meet Llama 3.2 on Databricks: faster and now multi-modal. The Llama 3.2 release pushes the frontier of enterprise #GenAI with new smaller models for latency + cost-sensitive use cases and larger multimodal models for visual understanding. Customers can easily get started tuning the Llama 3.2 models securely and efficiently on their data. Available today in Databricks Mosaic AI. 

9

Want to experience faster loading with Neo4j? Check out Eric Monk's blog on parallel loading. #dataloading #neo4j #cypher

7

1 Comment

Snowflake Cortex Analyst is finally out in Preview! 📬 This is an announcement from the Snowflake Summit, introducing a fully-managed Snowflake Cortex feature that allows you to build applications capable of providing reliable answers to business questions using your structured data in Snowflake. ** Why is it Important? 📣 ❇ Business users can ask questions in natural language and receive direct answers without writing SQL ❇ It is also available as a convenient REST API that can be seamlessly integrated into any application. Benefits from Snowflake security systems include the ability to customize how and where users interact with results ❇ To achieve high accuracy in text-to-SQL, Cortex Analyst utilizes an AI setup driven by advanced LLMs and a user-defined semantic model ❇ Simplify model selection, architecture maintenance, GPU capacity planning, and more with a lower cost of ownership https://lnkd.in/g922D_7v #snowflake #infostrux #datasuperhero #snowflake_advocate #genai

14

We've all had to deal with Hadoop, Spark or any Apache related tool in our ETLs to some extent. However, it's not so common to be involved in lifting up the services from scratch nowadays (lucky us!). That's why I want to share one of the dockerized playgrounds I've been using to tinker with these technologies in my local environment: μcluster. Take it as an interesting exercise if you're interested in learning more about Docker! https://lnkd.in/dPfTNbqF

6