Post de CEIMIA

This is a mind-blowing announcement. Mistral AI, the French company that has been fighting tooth and nail to water down the #AIAct's foundation model rules, is partnering up with Microsoft. So much for 'give us a fighting chance against Big Tech'. The first question that comes to mind is: was this deal in the making while the AI Act was being negotiated? That would mean Mistral discussed selling a minority stake to Microsoft while playing the 'European champion' card with the EU and French institutions. If so, this whole thing might be a masterclass in astroturfing, and it seems unrealistic for a partnership like this to be finalised in less than a month. Many people involved in the AI Act noted how Big Tech's lobbying on GPAI suddenly went quiet toward the end. That is because they did not need to intervene since Mistral was doing the 'dirty work' for them. Remarkably, Mistral's talking points were extremely similar to those of Big Tech rather than those of a small AI start-up, based on their ambition to reach that scale. The other question is how much the French government knew about this upcoming partnership with Microsoft. It seems unlikely Paris was kept completely in the dark, but cosying up with Big Tech does not really sit well with France's strive for 'strategic autonomy'. Especially since the agreement includes making Mistral's large language model available on Microsoft's Azure AI platform, while France has been pushing for an EU cybersecurity scheme to exclude American hyperscalers from the European market. Still today, and I doubt it is a coincidence, Mistral has announced the launch of Large, a new language model intended to compete directly with OpenAI's GPT-4. However, unlike previous models, Large will not be open source. In other words, Mistral is no longer (just) a European leader and is backtracking on its much-celebrated open source approach. Where does this leave the start-up vis-à-vis EU policymakers as the AI Act's enforcement approaches? My guess is that someone will inevitably feel played. https://lnkd.in/dQQ3_QiZ

As OpenAI and Meta introduce LLM-driven searchbots (on top of Bing and Google already splashing "AI" all over search), I'd like to once again remind people that neither LLMs nor chatbots are good technology for information access. Chirag Shah and I wrote about this in two academic papers: 2022: https://lnkd.in/g8Pswfe3 2024: https://lnkd.in/gNcSqQXZ We also have an op-ed from Dec 2022: https://lnkd.in/grBHJRWB Why are LLMs bad for search? Because LLMs are nothing more than statistical models of the distribution of word forms in text, set up to output plausible-sounding sequences of words. https://lnkd.in/gJXVDGf3 If someone uses an LLM as a replacement for search, and the output they get is correct, this is just by chance. Furthermore, a system that is right 95% of the time is arguably more dangerous tthan one that is right 50% of the time. People will be more likely to trust the output, and likely less able to fact check the 5%. But even if the chatbots on offer were built around something other than LLMs, something that could reliably get the right answer, they'd still be a terrible technology for information access. Setting things up so that you get "the answer" to your question cuts off the user's ability to do the sense-making that is critical to information literacy. That sense-making includes refining the question, understanding how different sources speak to the question, and locating each source within the information landscape. Imagine putting a medical query into a standard search engine and receiving a list of links including one to a local university medical center, one to WebMD, one to Dr. Oz, and one to an active forum for people with similar medical issues. If you have the underlying links, you have the opportunity to evaluate the reliability and relevance of the information for your current query --- and also to build up your understanding of those sources over time. If instead you get an answer from a chatbot, even if it is correct, you lose the opportunity for that growth in information literacy. The case of the discussion forum has a further twist: Any given piece of information there is probably one you'd want to verify from other sources, but the opportunity to connect with people going through similar medical journeys is priceless. Finally, the chatbots-as-search paradigm encourages us to just accept answers as given, especially when they are stated in terms that are both friendly and authoritative. But now more than ever we all need to level-up our information access practices and hold high expectations regarding provenance --- i.e. citing of sources. The chatbot interface invites you to just sit back and take the appealing-looking AI slop as if it were "information". Don't be that guy. Update 11/5: I created a newsletter post about this topic, with responses to some of the frequent replies: https://lnkd.in/gEMEZ2Tn

This is a mind-blowing announcement. Mistral AI, the French company that has been fighting tooth and nail to water down the #AIAct's foundation model rules, is partnering up with Microsoft. So much for 'give us a fighting chance against Big Tech'. The first question that comes to mind is: was this deal in the making while the AI Act was being negotiated? That would mean Mistral discussed selling a minority stake to Microsoft while playing the 'European champion' card with the EU and French institutions. If so, this whole thing might be a masterclass in astroturfing, and it seems unrealistic for a partnership like this to be finalised in less than a month. Many people involved in the AI Act noted how Big Tech's lobbying on GPAI suddenly went quiet toward the end. That is because they did not need to intervene since Mistral was doing the 'dirty work' for them. Remarkably, Mistral's talking points were extremely similar to those of Big Tech rather than those of a small AI start-up, based on their ambition to reach that scale. The other question is how much the French government knew about this upcoming partnership with Microsoft. It seems unlikely Paris was kept completely in the dark, but cosying up with Big Tech does not really sit well with France's strive for 'strategic autonomy'. Especially since the agreement includes making Mistral's large language model available on Microsoft's Azure AI platform, while France has been pushing for an EU cybersecurity scheme to exclude American hyperscalers from the European market. Still today, and I doubt it is a coincidence, Mistral has announced the launch of Large, a new language model intended to compete directly with OpenAI's GPT-4. However, unlike previous models, Large will not be open source. In other words, Mistral is no longer (just) a European leader and is backtracking on its much-celebrated open source approach. Where does this leave the start-up vis-à-vis EU policymakers as the AI Act's enforcement approaches? My guess is that someone will inevitably feel played. https://lnkd.in/dQQ3_QiZ

"They said it could not be done": pleias releases world’s first LLMs trained on exclusively open data. Pleias-3b, Pleias-1b and Pleias-350m are all based on the two trillion tokens set from Common Corpus. Pleias models are the first models compliant with the European AI Act. Beyond the copyright issue, we took care to anticipate other aspects of regulation, such as acceptable use policy and ensuring enough multilingual capacities to be widely usable on the European market. While current small SLMs struggle in other language than English or Chinese, even our smallest model is fully conversant in the main European languages. Our models do not only set new standards for open and ethical training. We’re publishing two specialized models for knowledge retrieval, Pleias-Nano (based on Pleias-1b) Pleias-Pico (based on Pleias-350m), which are competitive with best existing models (Llama, Qwen) ten times their size. Both models come with increased capacities for source analysis and grounding statements with references and quotes. They fill a new niche of tiny multilingual RAG models and are ideally fit for local AI applications running on CPU. They are already an integral part of our scientific assistant ScholastikAI we are building for the Mozilla builder program. Pleias 1.0 models could not have existed without the open science ecosystem for AI. The 3B model was trained on the Jean Zay supercomputer as part of the Grand Challenge with support from Etalab - Direction interministérielle du numérique (DINUM). We developed our 1.2B model in a collaboration with TractoAI. The weights of all the Pleias 1.0 models are available on Hugging Face under an Apache 2.0 license, as part of the Common Models collection: https://lnkd.in/enK6sA4R As part of our open science commitment we have further made available the dataset (Common Corpus) and our training code based on Nanotron. https://lnkd.in/ea_eS2TD The release of Pleias 1.0 is both the end and a start of the journey. We have shown it is not only feasible to train models on open data but we can go beyond the existing state of the art. And yet so much remains to be done to build thriving pretraining commons. I hope for 2025 to see more time, efforts and investments allocated to actually open, ethical and high-performing LLMs and SLMs. This is not only critical for scientific reproducibility or social acceptability but simply the fundamental basis to create a lasting and virtuous market for AI.

This is a mind-blowing announcement. Mistral AI, the French company that has been fighting tooth and nail to water down the #AIAct's foundation model rules, is partnering up with Microsoft. So much for 'give us a fighting chance against Big Tech'. The first question that comes to mind is: was this deal in the making while the AI Act was being negotiated? That would mean Mistral discussed selling a minority stake to Microsoft while playing the 'European champion' card with the EU and French institutions. If so, this whole thing might be a masterclass in astroturfing, and it seems unrealistic for a partnership like this to be finalised in less than a month. Many people involved in the AI Act noted how Big Tech's lobbying on GPAI suddenly went quiet toward the end. That is because they did not need to intervene since Mistral was doing the 'dirty work' for them. Remarkably, Mistral's talking points were extremely similar to those of Big Tech rather than those of a small AI start-up, based on their ambition to reach that scale. The other question is how much the French government knew about this upcoming partnership with Microsoft. It seems unlikely Paris was kept completely in the dark, but cosying up with Big Tech does not really sit well with France's strive for 'strategic autonomy'. Especially since the agreement includes making Mistral's large language model available on Microsoft's Azure AI platform, while France has been pushing for an EU cybersecurity scheme to exclude American hyperscalers from the European market. Still today, and I doubt it is a coincidence, Mistral has announced the launch of Large, a new language model intended to compete directly with OpenAI's GPT-4. However, unlike previous models, Large will not be open source. In other words, Mistral is no longer (just) a European leader and is backtracking on its much-celebrated open source approach. Where does this leave the start-up vis-à-vis EU policymakers as the AI Act's enforcement approaches? My guess is that someone will inevitably feel played. https://lnkd.in/dQQ3_QiZ