Pichao Wang’s Post

"Our study found that large language models can predict how linguistic information is encoded in the human brain, providing a new tool to interpret human brain activity. The similarity between the human brain’s and the large language model’s linguistic code has enabled us, for the first time, to track how information in the speaker’s brain is encoded into words and transferred, word by word, to the listener’s brain during face-to-face conversations." https://lnkd.in/eYF_7XAz

Three language experts: Three reasons why AI doesn’t model human language https://lnkd.in/gUMDvVH9 The third issue they note is that LLMs can generate “impossible” languages that don’t follow the principles that govern all human languages of which researchers are aware. And they can learn them just as well as an actual human language. And they don't know the difference.

🔍 Rethinking Multimodal Reasoning in AI 🔍 The field of multimodal AI continues to evolve, and a fascinating step forward comes with the paper "Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination" . This study introduces the Visual Inference Chain (VIC) framework a game changing approach that decouples visual and textual inputs to enhance reasoning in multimodal large language models (MLLMs). The research challenges the conventional "thinking while looking" paradigm, showing that reasoning before engaging with visual inputs significantly reduces hallucinations and improves accuracy in tasks like visual question answering and object recognition. By employing the "thinking before looking" strategy, VIC enables MLLMs to generate structured reasoning chains using textual context alone, resulting in greater precision and robustness. As someone deeply involved in AI and data science, I find this approach remarkable. Hallucinations where AI generates content not grounded in the input have been a persistent challenge in multimodal systems. VIC’s ability to mitigate these issues represents a significant leap forward, offering better reasoning for both general and complex tasks. This research not only deepens our understanding of multimodal reasoning but also opens the door to more reliable and scalable AI applications across diverse industries. If you're curious about the future of multimodal AI, I highly recommend exploring this paper. 💬 How do you think frameworks like VIC will shape the evolution of AI reasoning? Let’s discuss! 📖 Read the full paper here: https://lnkd.in/ggesKMg7 #AI #MLLM #MultimodalReasoning #DataScience #Innovation

#DataSpacesTuesday ◾ Navigating the language data landscape: Introducing the European Language Data Space 👉 https://lnkd.in/enq6pVJT In today's digital age, AI-based tools like ChatGPT and search engines are transforming how we interact with language. But the development of state-of-the-art language technology and natural language processing applications relies heavily on access to high-quality language data. The European Language Data Space (LDS) takes up this issue, fostering the trustworthy exchange of language data and resources. 𝗛𝗲𝗿𝗲'𝘀 𝘄𝗵𝗮𝘁 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗸𝗻𝗼𝘄: Many language technology providers struggle to gather this vast amount of language data necessary for competitive AI development. The LDS aims to create a specific digital marketplace, providing stakeholders with easy access to high-quality data and offering a platform for research and collaboration. Excitingly, the LDS infrastructure is currently under development, with the first prototype set to launch in mid-2024. The Language Data Space Connector will be based on the Eclipse Foundation Dataspace Components. Don't miss out on this opportunity to stay informed and inspired. Check out the Data Spaces Radar Report today!

Spoken dialogue plays a crucial role in human-AI interactions, necessitating dialogue-oriented spoken language models (SLMs). To develop versatile SLMs, large-scale and diverse speech datasets are essential. Additionally, to ensure hiqh-quality speech generation, the data must be spontaneous like in-wild data and must be acoustically clean with noise removed. Despite the critical need, no open-source corpus meeting all these criteria has been available. This study addresses this gap by constructing and releasing a large-scale spoken dialogue corpus, named Japanese Corpus for Human-AI Talks (J-CHAT), which is publicly accessible. Furthermore, this paper presents a language-independent method for corpus construction and describes experiments on dialogue generation using SLMs trained on J-CHAT. Experimental results indicate that the collected data from multiple domains by our method improve the naturalness and meaningfulness of dialogue generation. #AI #SpeechModels #DialogueGeneration #SpokenLanguage #LargeScaleDatasets

There are several teams like us using AI for language revitalization. If everything goes right, we should be able to save a lot of dying language in the next decade or so. https://lnkd.in/geRj2Vhu

📢 Happy to share that our pre-print, "How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not", is now live on ArXiV! In this paper, we investigate the impact of each component in the Multimodal architecture that connects Speech Foundation Models (SFM) and Large Language Models (LLM). We evaluate different configurations by comparing a diverse range of adapter modules, SFMs, and LLMs across two widespread speech-to-text tasks: Automatic Speech Recognition and Speech Translation. Our findings clarify the impact of each component on performance, guiding us toward successful and seamless integration. 🔍 💡 🔗 Read more here: https://lnkd.in/da-D64Cy #LLM #Multimodal #SpeechRecognition #SpeechTranslation #MachineLearning #AI #NLP #Research ✍🏼 w/ Francesco Verdini, Pierfrancesco Melucci, Francesco Cariaggi, Marco Gaido, Sara Papi, Szymon Mazurek, Marek Kasztelnik, Luisa Bentivogli, Sébastien Bratières, Paolo Merialdo, Simone Scardapane 🙏 Part of the project Meetween joined by Translated

Natural language instructions induce generalization in networks of neurons

📢 Nearly two weeks ago I got a chance to share the most recent reasoning 🧠 capabilities of Large Language models in Sentiment Analysis @ #NLPSummit hosted by ringcentral event. For those who missed and still wish to dive into most recent advances of Generative AI in that area, I am happy to share the recording of the past talk which is now available on YouTube: https://lnkd.in/es9yVjFZ After watching this video you will be aware of: ☑️ how well LLMs reasoning can be used for reasoning in sentiment analysis as in Zero-shot-Learning, ☑️ how to improve reasoning by applying and leaving step-by-step chains (Chain-of-Thought) ☑️ how to prepare the most advanced model in sentiment analysis using Chain-of-Thought. The report is expected to provide results and conclusions based on experiments on the RuSentNE-2023 corpus for open models and proprietary (ChatGPT) for reasoning in the field of sentiment analysis on English and non-English texts Additional Links 📜 Paper: https://lnkd.in/ew4tTfgN ⭐ Code: https://lnkd.in/eRe_MSq7 Special thanks to John Snow Labs for hosting #NLPSummit and provided opportunity to present the related studies! #nlpsummit #nlp #sentimentanalysis #generativeai #reasoning #zeroshotlearning #finetuning #cot #chainofthoughts