Christoph Janz’s Post

If you’re building a product that uses machine learning, has a lot of distributed data, and you want to build with privacy in mind… look into federated learning. Simply put, federated learning (FL) trains centralized models on decentralized data. It works like this: 1. Deploy a model to user devices or endpoints where data lives 2. Each device fine-tunes the model locally, alongside the data 3. The model updates are sent back to the central server. Importantly the data is not shared. 4. The central server aggregates the updates and integrates them. The improved model can be redistributed and the process can repeat. Raw data doesn’t leave the device. Just like any solution to a complex problem, there are challenges in deploying with FL, but frameworks (TensorFlow Federated), libraries (PySyft), and teams of experts are out there to help. We’re at an interesting moment in tech where the barrier to building is lower than ever before. We need tooling that supports responsible development to be just as easy. FL has always been one of my favorites - it's just one tool available but seems like magic 🪄 enabling privacy-forward, data-driven products.

𝐆𝐚𝐥𝐥'𝐬 𝐥𝐚𝐰: Every complex system that works evolved from a simple system that works. Keep this is in mind when developing a machine learning system. It's easy to ignore it. That happens when someone thinks of a long wish list of features and then designs a system to satisfy all of them. Here's how the procedure should go: ◾ 𝐃𝐞𝐭𝐞𝐫𝐦𝐢𝐧𝐞 𝐰𝐡𝐚𝐭 𝐲𝐨𝐮 𝐰𝐚𝐧𝐭 𝐚𝐧𝐝 𝐰𝐫𝐢𝐭𝐞 𝐭𝐡𝐚𝐭 𝐝𝐨𝐰𝐧 𝐢𝐧 𝐭𝐞𝐫𝐦𝐬 𝐨𝐟 𝐦𝐞𝐭𝐫𝐢𝐜𝐬. Those metrics should be sensitive/attributable to your model, aligned with your overall goal, not gameable and measurable in the short run. ◾ 𝐂𝐫𝐞𝐚𝐭𝐞 𝐚 𝐰𝐨𝐫𝐤𝐢𝐧𝐠 𝐞𝐧𝐝-𝐭𝐨-𝐞𝐧𝐝 𝐩𝐢𝐩𝐞𝐥𝐢𝐧𝐞 𝐚𝐬 𝐪𝐮𝐢𝐜𝐤𝐥𝐲 𝐚𝐬 𝐟𝐚𝐬𝐭 𝐲𝐨𝐮 𝐜𝐚𝐧. Once you have this, you have something to iterate from. The model inserted here should be guaranteed to be reasonable. If you're doing time series forecasting, use a hold constant forecast. If you're doing computer vision, you a small opensource model, like VGG16. ◾ 𝐒𝐭𝐚𝐫𝐭 𝐭𝐡𝐞 𝐞𝐫𝐫𝐨𝐫 𝐚𝐧𝐚𝐥𝐲𝐬𝐢𝐬. What does this barebones model fail to do? For the most important issue, propose a solution, implement it, test it and measure if it works. If it doesn't, propose another solution and start again. ◾𝐈𝐭𝐞𝐫𝐚𝐭𝐞. Repeat the error analysis until the model is satisfactory. This is a slow process, but the only way. Also, keep track of old versions, since making changes can damage the model, and you are likely to need to revert back. This is generic advice. For an in depth look, I suggest: ◾𝐃𝐞𝐞𝐩 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐓𝐮𝐧𝐢𝐧𝐠 𝐏𝐥𝐚𝐲𝐛𝐨𝐨𝐤: This gives exact procedure about everything involved in getting a complex system working. https://lnkd.in/gc5J6nSa ◾𝐃𝐞𝐞𝐩 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠, 𝐂𝐡𝐚𝐩𝐭𝐞𝐫 𝟏𝟏: It covers the practical methodology in developing working DL systems. https://lnkd.in/gBKWFcT2

Can a public sector organization implement machine learning to facilitate transformation?   It can, and we’ve seen (and helped) it happen! The Court Administration of Latvia needed a faster and future-proof solution to anonymize court documents for publication. Our team optimized the process using machine learning. The organization now has the necessary skills to apply machine learning and machine teaching to this critical workflow. And more: 👉  They reduced their manual processing time from 20 minutes to less than one minute per document. 👉 The introduction of machine learning models improved the accuracy of anonymized documents, reducing the need for manual changes by 80-90%. 👉 The team gained an extra 1,000 days to dedicate to other activities instead of filling out paperwork manually. And there is more: while the customized machine learning and machine teaching applications now perform much of the heavy lifting, the new process still involves human intervention, so there is no loss of control or quality assurance. Learn about this client story here: https://lnkd.in/dm6ufAHS #machinelearning #intelligenceautomation #OwnYourTransformation

How much can you save with open-source LLMs (by use case)? GPT-4o costs 0.0015c per output token. Llama3.1-70B is c. 17x cheaper, depending on the provider. According to LMsys, 3.1-70B performs on par with Claude Opus and GPT4, so for most use cases, you are extremely unlikely to witness a performance degradation when switching to it. How much savings can this price difference generate? (1) Suppose you have a chat interface with small outputs (70 words) and 15 messages per customer. You could decrease your inference costs from 1.6c per session to 0.1c.  (2) Suppose you have a sales tool that generates 1,000 emails monthly (200 words each) for your customers. You could decrease your cost per customer from 3 dollars to 15c.  (3) Suppose you use LLM to process and label large documents, with 1,000 documents per customer monthly and 5,000 words in a document. Your cost with OpenAI would be around 15 USD monthly per customer and c. 2.5 USD with Llama. While this difference will likely be extremely small for early-stage firms, large companies will likely see an investment into switching to open-source pay off in a few days. 

I love this conference! See the agenda for my session 'Team Checklists: Are You Using Them Enough?'

That's exactly why Data Engineering is crucial for examining data quality. Other data methodologies could also address some of these potential issues; for instance: 1. The problem of "Not having enough Data" could be addressed using the Data Augmentation techniques. 2. "Non-representative training data" & "Low Diversity": Synthetic Data also can be used to enhance datasets and increase diversity. If datasets are imbalanced and not representative of the population they aim to serve, Generative AI can be utilized to augment the data. Synthetic Data also helps counter the lack of diversity in Data. But for sure, all of these methodologies require high-quality datasets and are time-consuming. What other ways could be used? Love to know your idea! #AI #data #bigdata #dataengineering #machinelearning #datapreprocessing #datacleaning

Data is the most fundamental requirement to build a good machine learning system. Common challenges: - Not having enough of it - Poor-quality - Nonrepresentative training data - Not enough relevant features - Too many irrelevant features - Low diversity Garbage in, garbage out.

Data is the most fundamental requirement to build a good machine learning system. Common challenges: - Not having enough of it - Poor-quality - Nonrepresentative training data - Not enough relevant features - Too many irrelevant features - Low diversity Garbage in, garbage out.

This weekend, I worked on the topic of context. Imagine this scenario; in your LLM training or vector dataset, you have terms that are shared between multiple teams, products, services, or knowledge artifacts. When the LLM performs the calculation to respond, it gets "confused" about what data to return, often providing data from the "wrong" team, product, service, or knowledge artifact. This happens when all the data is presented as one unified dataset. The AI is not confused or wrong, the issue lies with how we present the data to the LLM. I built a prototype using OpenAI and Pinecone that introduces the concept of data context to the LLM. So when you talk with the LLM, it will attempt to determine the context of the situation, before it attempts to access data (knowledge base), or run functions (call AI-agent). As it answers, it defines the context that it is answering in. So if it does not believe you are asking about a specific team, product, service, or artifact, it will respond with "generalized" responses, meaning the response comes from the model's shared training data. When it believes you are talking about a specific team, product, service, or artifact, it will then shift the context to that specific topic, and gain access to a specialized vector database with specific knowledge and capabilities of that topic (or a specialized function that calls a specialized AI-agent). So there is no more getting "confused" because we have created a new construct, where specific data and functions become compartmentalized. Did this already exist? I'm not sure, but it was fun to build!

If your job isn't data entry but you do it anyway... This video may give you some street cred with management. I go through how to use the Azure AI Document Intelligence platform to read values off generic documents, invoices, and receipts. It is a neat tool that allows no code people to learn what is possible with Document Intelligence and how it could help alleviate some of the mundane work.