Maxim Lukichev

EK's Lulit Tesfaye and Jeff Jonas of Senzing teamed up on EK's latest blog, in which they discuss typical problems that would necessitate Entity Resolution, how to unify enterprise data with a Semantic Layer, and the role of entity-resolved knowledge graphs. #semanticlayer #knowledgegraph #entityresolution https://lnkd.in/ePujrbve

9

Snowflake’s Arctic-TILT model, powering our Document Al, beats GPT-4 with just 0.8B parameters, securing a top spot in the standard benchmark for document understanding DocQVA. This efficiency not only reduces operational costs but also enhances the scalability of solutions built on Arctic-TILT. This also means that it does not require large GPU instances or resources to leverage the Arctic-TILT model in Document AI, making it more accessible and adaptable to different customer sizes and needs.

449

7 Comments

Just out! Chapter 2.3, Domain modeling, explores how you can defend against poorly thought out, poorly understood architectures. In this chapter of the playbook, we walk through applying domain modeling to our model use case. https://bit.ly/3z8Sc9y #modeling #DDD

1

Using Qdrant with Apache Airflow can revolutionize how you automate and manage complex workflows for semantic querying. It makes handling large-scale vector data much simpler, and Airflow's monitoring features ensure everything runs smoothly. In this tutorial, you'll learn to use the Qdrant Airflow provider to write a DAG that generates embeddings in parallel and performs semantic retrieval based on user input. Explore the guide here: https://lnkd.in/eMpqD2tN P.S. How are you using Airflow in your data workflows? Comment below! 🚀

48

3 Comments

Our last poll was about data and transparency. Today, Databricks has created The State Reader API , a new capability to help users have access and analyze internal state data for full transparency and ease of use __ Will this new capability make a real difference from what we are used to use so far? #Databricks #Newcapability #datatransparency #accessibility #easeofuse #nuboTed

2

2 Comments

Developers can now store, index, and query vectors in PostgreSQL using pgvector, eliminating the need for a dedicated vector database. This blog from Yugabyte expert Brett Hoyer deep dives into pgvector, and explains how to store text embeddings as vectors, execute similarity search, choose between indexes, and scale with distributed SQL.💡 Find out more!⬇️ https://hubs.la/Q02JyYrM0 #YugabyteDB #pgvector #vector #AI #Postgresql #cloudnative #distributedsql

5

Last week saw the General Availability of dynamic functionality for Databricks Workflows, in the form of the parameterized ForEach activity, but what does that mean? And why should we care? For a long time we've been using external orchestration tools whenever things had to be flexible, metadata driven or simply at a large scale - but with these changes we have a cheap, flexible way of achieving some pretty complex orchestration inside of Databricks itself! In this video Simon looks at a quick pattern for running a medallion-style lake processing routine, all driven by some JSON metadata! https://hubs.la/Q02NmWB_0 #DataEngineering #DataLakehouse #Spark #Azure #DataOrchestration #Finally

31

🌐 Boost Your Workflow with AI! 🌐 Our Senior Data Engineer, Jaime Salas, explains "Mastering AI in Snowflake: Transform Your DBT Workflow!" on the Infostrux Engineering Blog. 📘 Highlights: -Utilize LLMs with Snowflake Cortex -Integrate AI into DBT -Practical examples 🔗 Read more: https://hubs.ly/Q02BX71b0 #Snowflake #AI #DBT #DataScience #TechInnovation

9

✨ Master AI in Snowflake: Enhance Your DBT Workflow ✨ Our Senior Data Engineer, Jaime Andres Salas, guides you in integrating LLMs with Snowflake Cortex into your DBT transformations. Key points: 🔹 Sentiment analysis with AI 🔹 Join sentiment data with reviews 🔹 Create recommendations for feedback https://hubs.ly/Q02BX1sW0 👉 Read the full article #AI #Snowflake #DBT #DataScience #LLM

8

We have out-of-the-box support in LlamaIndex for Snowflake SOTA arctic-embed models (which outperform other leading embedding models in benchmarks) - it's a one-liner through our Hugging Face integration! And in ~4 more lines of code, you can build a RAG indexing pipeline over your data with these embedding models. Check out the screenshot below for how to get up and running with this - we might have more detailed benchmarks soon, stay tuned 💡

214

3 Comments

Quality data, tailored solutions. Cogito's enterprise grade data labeling solutions (EDLS) are domain-specific, compliant & backed by diverse annotation teams. We prioritize data authenticity & integrate with RLHF. Scale with confidence and unlock the true potential of your AI/ML models. #datalabeling #dataannotation #ai #ml #datasolutions #RLHF

9

Have you tried running the DBRX models from Databricks? In our latest blog post we provide a tutorial to get you started with the model on Ori’s public cloud. We were very impressed with DBRX’s performance on the Hugging Face Open LLM Leaderboard before the updates last week. While DBRX Instruct did not score as highly under the new scoring system, it's still one of the best open models for Language Understanding (MMLU), Programming (HumanEval), and Math (GSM8K). We also used BeFOri, the benchmarking framework from Ori to determine DBRX Base performs inference most efficiently between 40 and 256 concurrent requests on one of Ori's 8 x H100 PCIe bare metal servers. Check out the blog to learn how you can use BeFOri to optimise self-hosted LLM performance on any infrastructure! https://lnkd.in/gi55WEks

43

Want to simplify your telemetry data? In #o11ycast ep. 74, Liudmila Molkova explains how semantic conventions make it possible. Listen now: https://hubs.ly/Q02Txlxw0

1

This week on the blog, Ami Ikanovic discusses the "DIY is better" mentality for #datamasking and how it might seem like a quick win initially, but the long-term costs – time, complexity, and risk – are too high. https://lnkd.in/eUXyXKQf #datasecurity #datasecops #datagovernance #Snowflake

19

Learn How to Run Hugging Face Models Programmatically Using Ollama  and #Testcontainers 🤖💻 Dive into the future of AI/ML with easy model deployment 👋👋 Say goodbye to complex setups and hello to seamless integration 👉👉 Docker, Inc blog https://lnkd.in/g_F89xCk by Ignacio López Luna

8

🚀 New Blog: Navigating High Cardinality in Metrics 🚀 High cardinality data offers precision but can inflate storage and computational costs. Our latest blog dives into how to manage this balance effectively in observability metrics. Key tips include: 🔹 Avoid Improper Instrumentation 🔹 Optimize Histogram Buckets 🔹 Use Consistent Tags ❇ Check out Harold Lim's insightful blog here: https://lnkd.in/ekFhrftu #observability #o11y #metrics #cardinality And stay tuned for our upcoming blog, where we’ll tackle high cardinality in events, logs, and traces!

21

We’re constantly pushing the envelope to bring accurate, real-time grid forecasting to the energy industry. In this blog post, we explore how we leverage Snowflake to enhance our short-term grid forecasts. Learn how this partnership enables us to deliver more precise insights, optimize energy use, and support a resilient and sustainable future for the grid.💡 https://hubs.li/Q02L1cqj0

28

📢 New Tutorial: Building a Real-Time CDC Pipeline with MySQL, Kafka, and Parseable Want to track database changes in real-time? We've published a comprehensive guide on implementing Change Data Capture (CDC) for modern data architectures. Learn how to: • Set up a complete CDC pipeline • Capture MySQL database changes with Debezium • Stream data through Kafka • Analyze changes in real-time with Parseable • Create monitoring dashboards for audit logs Perfect for teams looking to implement: ✅ Real-time analytics ✅ Data synchronization ✅ Audit logging ✅ Fraud detection ✅ Zero-downtime migrations Ready to modernize your data pipeline? Check out our step-by-step tutorial: https://lnkd.in/gS9Ahnd2 #DataEngineering #CDC #ApacheKafka #RealTimeAnalytics #DataPipeline #OpenSource

13

2 Comments