Caio Moreno, Ph.D.

IBM and the Spanish Government have agreed to a collaborative partnership to further AI strategy while building Spanish language AI models. #DataScience #AI #ArtificialIntelligence https://hubs.li/Q02t15Wq0 If you enjoyed this post, follow me for more insights on AI and data science!

5

1 comentario

In this article, Marco Peixeiro explores the inner workings of Moirai and apply the model in a forecasting project (using Python) and compare its performance to data-specific deep learning models:

15

Does your tool support a fully implemented Iceberg-based open lakehouse architecture?" 1. COW, MOR table format 2. Hidden Partitions --Partition Evolution 3. Data-As-Code. --Branching, Tagging 4. Schema Evolution Snowflake Cloudera Databricks Dremio Upsolver Trinio Amazon Web Services (AWS)

3

My latest presentation for the Croissant ML working group on the building distributed #AI based on Dataverse, Kaggle, Hugging Face and other data platforms, with LLM serving as "navigation interface" between data nodes in the network. https://lnkd.in/e8vUiWqY

20

Discover how a graph data model can help represent explicit relationships in data for use in #RAG and how that can be done with two popular #graph models: Property Graphs and RDF Knowledge Graphs, both natively supported by Oracle Graph. https://lnkd.in/e-UTfF2Y

1

Many discuss AI and innovation, but the conversation often overlooks the practical implementation of AI-based systems. To address this critical aspect, my colleagues Roberto Hernández Ruiz and Ferran Aran Domingo have launched the first post in a series dedicated to exploring the productionalization of AI-based solutions. GFT Technologies #artificialIntelligence #solutions https://lnkd.in/dr9qDDwH

43

2 comentarios

NEW BLOG POST! Let's continue our DP-600 learning journey! You've heard that Star schema IS the preferred way to model data in your BI solutions. But, how do you create one in the #MicrosoftFabric Lakehouse? It's easier than you can imagine! https://lnkd.in/dP5eaZj5

31

1 comentario

Karini AI Customers: Supercharge Your GenAI Workflows with Databricks Vector Search (Now GA!) On May 8th, Databricks announced the general availability of Databricks Vector search (https://lnkd.in/dZiYYyNv), opening doors to a powerful new way to enhance your GenAI applications. Databricks Vector Search is a vector database built into the Databricks Data Intelligence Platform and integrated with its governance and productivity tools. We are very proud to announce that you can leverage the power of Databricks Vector search within your Karini AI workflows without writing a single line of code! Using Karini AI’s no-code data ingestion pipelines integrated with Databricks Vector Search, you create a vector search index from a Delta table. The index includes embedded data with metadata. You can then query the index to identify the most similar vectors and return the associated documents. You can structure the index to sync when the underlying Delta table is updated automatically. But wait, there's more! Karini.ai takes vector search to the next level. Here's why: 🐘 No-code recipe magic: Karini.ai's intuitive interface allows you to craft powerful search experiences without coding. Simply drag, drop, and configure pre-built "recipes" to integrate Databricks Vector Search seamlessly into your GenAI applications. 🐘 Effortless integration: Stop worrying about complex data pipelines. Karini.ai handles the heavy lifting, ensuring your GenAI application smoothly connects to Databricks Vector Search. 🐘 Enhanced user experience: Deliver results that users genuinely crave. Combined with Karini.ai's GenAI capabilities, Vector search leads to more insightful and relevant interactions within your applications. 🎉 The result? Supercharged GenAI applications that understand user intent like never before. Here are some exciting possibilities: ⚖️Personalized content recommendations: Imagine a legal research assistant who finds relevant cases and surfaces related legal arguments and discussions. 🛍 Intuitive product search: Develop a shopping experience that understands a user's search for "comfortable running shoes 👟 " and delivers options prioritizing cushioning, breathability, and specific brand preferences. ☄ Advanced document exploration: Empower researchers to uncover hidden connections within vast datasets, accelerating discovery and innovation. Ready to unlock the potential of vector search within your GenAI applications? Head over to Karini AI and explore the exciting possibilities! #KariniAI #Databricks #VectorSearch #GenAI #EnterpriseAI

4

#TBT to Marvik's presentation at the #AISummit by Databricks. "Generative AI at Scale Using GAN and Stable Diffusion" #GenAI is under the spotlight with its diverse applications, but deploying these models at scale presents many considerations. This presentation will explore various architectures and discuss optimization hacks for the sophisticated data pipelines essential to generative AI. https://lnkd.in/dC6AGNgx

28

𝐀 𝐖𝐚𝐫𝐦 𝐖𝐞𝐥𝐜𝐨𝐦𝐞 𝐭𝐨 “𝐒𝐩𝐚𝐫𝐤 𝐘𝐨𝐮𝐫 𝐃𝐚𝐭𝐚: 𝐑𝐃𝐃 𝐄𝐬𝐬𝐞𝐧𝐭𝐢𝐚𝐥𝐬 Welcome to the very first installment of “Spark Your Data: RDD Essentials”, a new article series dedicated to helping you get comfortable working with Apache Spark’s foundational data abstraction: Resilient Distributed Datasets (RDDs). If you’re reading this, chances are you already know what Spark is—a fast, scalable, and flexible open-source computing framework widely used for processing large volumes of data. Spark makes it easier to run complex transformations on big datasets and comes with a rich ecosystem that supports everything from batch analytics to machine learning and stream processing. In this series, however, we won’t be focusing on Spark’s broad capabilities or diving deep into RDD theory. Instead, our mission is straightforward: to build practical, hands-on skills with RDDs through code. Each post, starting with the next one, will feature simple, targeted code snippets that help you understand how to: Create RDDs Apply transformations to reshape, filter, or aggregate data Execute actions that bring results back to the driver program We’ll keep the explanations direct and the learning curve manageable, guiding you step-by-step so you can quickly pick up the essentials of working with RDDs in a real environment. What’s Coming Up? Starting from the next article, we’ll jump right into RDD basics—no heavy theory, just practical insights that you can apply immediately. Expect new content three times a week—every Tuesday, Thursday, and Friday—to keep you steadily moving forward and building your skill set. Stay Tuned Get ready to roll up your sleeves and dive into code-driven learning. By the end of this series, you’ll have the confidence to use RDDs comfortably in your daily work. Keep an eye out for the upcoming posts, and let’s spark your data together! Want to learn Spark components in depth with practical examples, assignments, quiz and 24*7 support? Check out our Udemy couse "Apache Spark and Databricks for Beginners - Learn Hands on below" https://lnkd.in/gct4DFET #ApacheSpark #SparkRDD #DataEngineering #BigDataAnalytics

4

So excited about this partnership! Meta to release the Llama 3.1 series of models on Databricks – available to customers today. These models expand context length to 128K, add multilingual capabilities, and include Llama 3.1 405B — the largest and highest-quality openly available foundation model.    With this release, customers can serve and fine-tune the Llama 3.1 models, connect them to our RAG and agentic systems, easily generate synthetic data to better customize their models, and leverage them for scalable evaluation.    Databricks has integrated Llama 3.1 throughout our Data Intelligence Platform.

5

Excited to share my latest blog on #elasticsearch Labs! 🚀 In this post, I dive into an alternative approach for parsing PDFs in #rag workflows, specifically tackling the challenges of extracting and making sense of tables in PDF documents. #openai #vectordb #genai #rag #elasticsearch

66

#LLMs #PromptEngeneering #KnowledgeGraphs Great article about how to combine Graphs and LLMs https://lnkd.in/dSy2psdY

3

New Blog: Using Apache Hudi & Iceberg tables in Databricks with Apache XTable (Incubating) 🎉 In this hands-on blog, I show how we can use XTable to interoperate among the 3 open table formats within Databricks environment. I explore a use case where a team of data scientists run ML experiments using MLflow on top of Hudi/Iceberg tables by consuming these tables as Delta Lake. Some key points to consider & how XTable helps: ✅ While predominantly focused on Delta tables, this team also would like to access Apache Hudi/Iceberg datasets from other teams to enhance their ML models. ✅ In the past, the data platform team has either duplicated these datasets to rewrite them as Delta Lake or conducted one-off migrations (costly). ✅ Data Platform team's goal: the ability to read table formats other than Delta, should be seamless & dependency-free, without the need for additional configurations. ✅ Irrespective of whether the source table is Hudi, Delta Lake or #ApacheIceberg, users should just be able to read the dataset as if it were a Delta Lake table using spark.read.format(“delta”).load(“table_name”) Link in comments. #dataengineering #softwareengineering

123

6 comentarios

We just drop a new release for TornadoVM - version 1.0.4. This one includes new features such as Tensor types based on the Panama Memory API, connection with ONNX, better memory management and power metrics in the profiler + plenty of bug fixes. We continue improving and bringing innovation to Java by enabling acceleration of data parallel applications on modern hardware. Our mission is to provide acceleration as easy as possible, as transparent as possible, and run as fast as possible using the latest technologies, hardware and software. Release notes: 🔗 https://lnkd.in/dJAtUayK #graalvm #java #ai #gpu #hardwareAcceleration

32