DagsHub

What a RAG system looks like from the inside

Object detection is going to be pretty much everywhere

LLMs are versatile tools that require specialized training to reach their full potential. Fine-tuning is the process of adapting a general-purpose LLM to excel at specific tasks or within particular domains. Similar to customizing a recipe with unique spices, fine-tuning infuses an LLM with the knowledge and abilities necessary to meet specific organizational needs. Without fine-tuning, LLMs function as broad knowledge bases, often lacking the depth or focus required for practical applications. This can result in irrelevant, inaccurate, or even harmful outputs. In business settings where precision and reliability are paramount, the consequences of an unrefined #LLM can be severe. DagsHub provides a centralized workspace for #datascientists to manage their entire project lifecycle, from #data to models, while fostering open collaboration.

For quite a long time I have been focused on writing a lengthy and detailed article on different approaches to develop a robust ML model one of which is "Continual learning or CL". The idea of CL arises from the fact as to how humans are capable of learning complex matters while preserving the old information. We also tend to leverage the these old information to learn new information quickly. We are adaptable. But it is not the same with the ML systems. They have to be retrained again on a new set of data. This of course is time-consuming and potentially expensive. In AI continual learning is the process of injecting or adding new information to a trained model while preserving the old information, mimicking human cognitive processes. I got an opportunity to write this article on CL with DagsHub along with Michał Oleszak and Daniel Tannor where we explained the various elements involved in CL -- types, approaches, and challenges -- as well as provided a practical approach to learning CL in PyTorch. You will learn a lot of valuable insights from this article. You can find the article link in the comment below.

Why are transformers so good at understanding language? The answer is Self-Attention. Self-Attention lets transformers focus on different parts of the inputs all at once instead of one piece at a time. It's kind of like giving the model the ability to understand the big picture by mapping the relationships between all of the little pieces within the data. And this is how they pick up on complex patterns and connections. One cool detail is that Self-Attention actually lets the model learn about the order and the spacing of the words itself without providing it explicitly. And that's part of why it's so powerful. So in other words, Self-Attention is not just another tool. It's actually what unlocks a lot of the power of modern transformers in LLMs.

We've enhanced our experiment tracking to let you see your model's predictions and outputs as they evolve during training. Visual insight into model behavior is critical, yet often overlooked in ML workflows. So we're introducing an integrated experiment artifacts view on DagsHub. Key benefits: - Real-time visual feedback: Watch your model learn through images, audio, and even 3D visualizations - Comprehensive artifact support: View text, model files, and even CSV files alongside metrics - Seamless integration: Works with the OSS MLflow API you're already using - Coming soon - HTML, Notebooks, artifact diffing and more. How it works: 1. Use mlflow.log_artifacts() to attach files to your experiment 2. Go to the experiments tab in your DagsHub repo 3. Visualize artifacts directly in the experiment view, no context-switching required. As ML practitioners, we know that numbers alone don't tell the whole story. Now you can literally see your model's progress, catching potential issues early and gaining deeper insights. What other visual tools would enhance your ML workflow? Share your thoughts below! Thanks Tal for building, Anna for design, and the entire team for shipping ⛴️🙏. Also, thanks MLflow for being awesome!

Check out this awesome post about image embeddings benefits, industry use cases and best practices. Thanks Ignacio Peletier Ribera

We're very lucky to be working with the top data scientists at MACSO Check out the full case study

I’m really proud to share our amazing partnership results with MACSO. Their ambitious ML team led by Hwan is doing mind-blowing work at the intersection of AI, edge computing, AgTech, and more. From pinpointing sources of air pollution to revolutionizing livestock monitoring, MACSO is proving that huge breakthroughs can happen. I'm proud that DagsHub gets to partner with them on this journey of innovation. By providing intuitive tools for experiment tracking, data management, and seamless collaboration, we've been able to help MACSO: 🚀 Increase experiment speed by 30% 🚀 Reduce data prep time by 50% 🚀 Boost team collaboration efficiency by 30% As Hwan put it: "DagsHub has been a game-changer for us. It not only streamlined our ML workflows but also ignited our team's creative potential, allowing us to experiment fearlessly and innovate rapidly. DagsHub is not just a tool; it's a catalyst for transformation in ML development.” From all of us at DagsHub, we're honored to lock arms 🤝 with the brilliant minds at MACSO. Their ability to reimagine what's possible in AI, AgTech, and edge computing is amazing. Check out the comments for the full case study #machinelearning #mlops #edgeai #agritech #datascience #startup

When something does go wrong, having data validation in place gives us confidence that our data pipelines are not the failure points and allows us to more easily zoom into other areas to investigate the nature of the problem. Awesome post by Eryk Lewinson