Youssef Mrini’s Post

Still digesting the gusher from Databricks #DataAISummit last week. Going from the Tabular acquisition that could either help unify or fork the open table landscape to the bulking up of Unity Catalog for converging data & AI governance; the introduction of a Variant data type for Delta Lake that provides the long-awaited hook for vector data to get treated as a first-class citizen in the lakehouse; and the onrush of enhancements to core data warehousing where AI is being used to compress 30 years of learning curve in developing an enterprise-grade SQL engine (I still need to play back Shant Hovsepian's part of the keynote). For us, actually, the highlight was Mosaic AI (weren't they Mosaic ML last year, but we digress...) with some pioneering work developing an MLOps lifecycle for AI agents. It's part of a trend that we're seeing towards -- choose the name -- a composite or ensemble approach, or mixture of experts, etc. where AI tasks are decomposed to orchestrated step-by-step processes where we find the right tools and/or algorithms for each step of the job. And by decomposing, hopefully we'll be able to make the AI more auditable. We've got a backlog to work through from Snowbricks month, so watch this space soon for our take. In the meantime, hearty thanks to Sonya Vargas (Ruzheva) and Kayli Berlin for unleashing the fire hydrant on us poor wretched analysts 😎.

You want to fine tune models, but you don't know where to start? In this video Michael Shtelma, Lara Rachidi and myself explore the entire process, from generating synthetic data to evaluating model performance, demonstrating how to fine tune LLMs. 💡 Key Moments: 1. Why fine-tuning matters: Fine-tuning smaller models boosts efficiency for specialized tasks, reducing costs and improving accuracy. 2. Thoughts on fine-tuning: Discussion on the ideal data size and strategies for balancing SME input with synthetic data. 3. Demo: Fine-tuning a Llama 3.1 model for European Union capital requirements regulations. The demo shows how synthetic data generation can be done and then fine tuning on Databricks using the synthetic data and the model training API. #AI #MachineLearning #Databricks #MosaicAI #LLMs #FineTuning #RegTech #FinancialCompliance #DataScience #ModelTraining #SyntheticData #TechInnovation #LLMOptimization #DataEngineering

I did an interviewed around how the role of the data partitioner will evolve and how architectural patterns can support them in the age of AI 🤖 Thank you Databricks and dbt Labs for putting it together! Some main takeaways: 🗺 New role in the "Analytics Engineer" that work as a link between Analysts and the raw data. 🔍 Distributed data ownership with centralised governance and infrastructure provide transparent data and by that increases the trust in the data. ⬅ Increase quality in data by shifting left on ownership. https://lnkd.in/dGGiT9Ju

Ready to level up your Data Science & AI skills? Join our Advanced Certification Masterclass for FREE @ https://bit.ly/3v0yevY Dive deep into advanced techniques and methodologies to become a data science powerhouse. Elevate your career and stand out in the competitive field of data science and AI! Huryy! Limited seats only! : : : #Edureka #RidiculouslyCommitted #TeamEdureka #LearnWithEdureka #Upskilling #Onlinelearning #Onlinecertification #ml #datascientist #dataanalytics #neuralnetworks #businessintelligence #machinelearningalgorithms #datasciencetraining #datasecurity

🚀 Transform your data game with Databricks and GenAI. Read our latest blog to discover how these technologies can drive innovation and insights for your organization: https://buff.ly/4dlms0A Contact us now to transform your data strategy: https://buff.ly/3UewzeB #Databricks #GenAI #ArtificialIntelligence #BigData #DigitalTransformation #DataAnalytics #DataScience #TechTrends #Xorbix

Let's explore what "Data Pipeline"is! With Metafied explore all the aspects and benefits of Data Science which will help Data Management easier #DataScience #MachineLearning #AI #Metafied #NeuralNetworks #DeepLearning #TechTips #LearningAlgorithms #Optimization #Coding

If your organization is using Databricks then join us to meet other Databricks practitioners. If you are not using Databricks yet, then come and learn the amazing capabilities of the Databricks platform and how you can apply Machine Learning and AI to your big data.

Exciting news for data scientists and analysts using Dataiku! 📊🎉 We're thrilled to announce a major update to the TitanML Dataiku Plugin, bringing powerful new features and improvements to enhance your data science workflows. 🔹 Official Snowflake Integration: Seamlessly deploy TitanML within Snowflake and connect with Dataiku's robust data science capabilities. 🔹 Enhanced JSON and Regex Support: Improve data manipulation and pattern matching within your Dataiku projects resulting in reduced hallucinations. 🔹 Chat Templates: Revolutionize AI interactions in your workflows with structured conversations and custom prompts. 🔹 Improved Documentation: Comprehensive guides and examples to help you leverage the full potential of the plugin. This update represents a significant leap forward in bringing advanced AI capabilities and integrations to the Dataiku platform. Read our latest blog post to learn more about these exciting new features and how they can empower your data science team (Link in the comments!) What new feature are you most excited about? Let us know in the comments! #DataScience #Dataiku #AI #MachineLearning #DataAnalytics #Snowflake

Learn to build the end-to-end data science pipelines from data ingestion to data visualization using Pandas pipe method. Read here: https://lnkd.in/dTU2pwYY #pandas #ml #machinelearning #pipelines #datascience #ai