Alperen Yakut

I unfortunately usually have to say no to physical events due to constrains but this LLM hackathon is an example of one of the best actual drivers of Stockholm => AI capital of Europe Shoutout to builders and even more: organisers Simon Kristina Burak. Tons of many great submissions, honouring to help by being jury. Note, I mean "AI capital" as in _AI powered software that creates value_. Not on training foundation models and similar. mentioning just 3 highlights: 1: https://pushinprod. vercel. app – analyses + visualises any github repo instantly + lets you ask an LLM questions about it 2: https://chestmate. vercel. app – ploy game (a bit emberassed to post this...), a tinder LLM based simulator with "intellectual" people 3: https://voteup-six. vercel. app – one-line to add feature requests, and "AI product manager" and a public roadmap to your site Many super impressive not mentioned above, since I don't have a link, DM me and I will add.

146

14 Comments

New partnership between Stack Overflow and Open AI. I will be super interested in seeing how this impacts the short and long term accuracy of LLMs in the future. In this video Theo shares a very thought-provoking picture of what this could mean for LLMs in the future. https://lnkd.in/gph_wGPs

1

2 Comments

One thing I’ve learned in my first 10 months at Meta… Don’t ask a senior eng what to do or how to do something. Instead, explore different approaches and ask the senior eng which they would recommend. For example, when devising the tech plan for a new feature that requires storing and retrieving new data, don’t simply ask what data solution you should use. Instead, scour internal wikis, existing code patterns, etc, to learn about the data storage solutions available. Then list out the pros and cons specific to your use case in terms of latency cost, capacity usage, reliability, implementation complexity, etc. Sending that breakdown to the senior eng (along with your own proposal for what you think the best choice is) will not only make their lives easier, but prove to them that you’re capable of working at the next level. Of course, there are exceptions. For example, if there’s a quick question that a senior engineer could answer in 30 seconds, but would take you 2 hours to figure out on your own, it makes sense to ask. What do others think? What types of questions should you ask senior eng and how should you decide whether to ask a question or figure it out on your own? #softwareengineer

1,329

138 Comments

🚀 Exciting News from RAI, an open-source Gen AI agent framework for robotics 🚀 We’ve just released some major updates that significantly boost RAI’s capabilities, including a multi-agent architecture as well as improvements to the Speech-to-Speech system! 🎤 Here’s a quick snapshot of what’s new: 💡 Key Updates: - New Multi-Agent System: RAI is now powered by two agents - HMI Agent (handles communication via voice or text) - RAI Agent (takes care of tasks/missions, handles complex tasks) Tasks can be added through the HMI (like our streamlit, text based app) or via ROS 2 CLI, and the RAI Agent will manage the task and provide real-time feedback until completion. - Voice Interface Overhaul: Our voice interface has been revamped with better detection, now supporting any recording device and OpenWakeWord for wake word detection. 🔎 Additional Enhancements: - RAI What I See: Smart Image Capture - NoMaD: Transformer-based navigation system - Global configuration file for easier genAI vendor management - Local ASR using Whisper - Compatibility with the latest 🦜LangChain version - Pydantic V2 compatibility Check it out and let us know what you think! Link: https://lnkd.in/dwCqJmN9 #AI #ROS2 #Robotics #MultiAgentSystem #SpeechToSpeech #RAI #TechUpdate

41

3 Comments

I'm excited to announce that our talk on the Agent-based AI Framework for ROS 2 has been accepted for ROSCon 2024 🐢! In recent years, there have been significant advancements in Visual-Language-Action (VLA) models. However, these advancements have not yet enabled these models to independently perform real-life robotic tasks. Rather than solely pursuing cutting-edge AI models aimed at complete transformations, we prioritize practical applications that deliver clear benefits in real-world scenarios. Our approach emphasizes seamless integrations and the creation of robust tools to effectively support everyday robotic tasks. If you're interested or have any questions, please feel free to reach out! #ros #ros2 #roscon #robotics #vla

8

I have a lot of knowledge and interests, but I really want to focus on making LLMs smaller for specific tasks. The models would still do all the same things as the original LLM, but they would be better at certain things. They would also be cheaper and take up less space to run. These models would be able to generalize better than older models, meaning they could handle more situations. But they would still be more affordable compared to the original LLM. I am pleased to note that the 14 million parameter model ( https://lnkd.in/ef_9rxc7 ) that I published on Hugging Face has attracted over 170 downloads within a week. Machine learning specialists in the field of e-commerce require solutions that can be immediately applied to everyday tasks.

13

This is a very eloquent podcast about quantum cognition and the work we do Qognitive

1

🚀 Introducing My Novel Neural Network: A Fusion of Topology, Manifolds, and Bayesian Nonparametrics 🚀 I’m thrilled to share my latest blog post where I delve into an innovative approach to neural networks. By integrating higher-dimensional topology, manifolds, and Bayesian nonparametrics, I've developed a model that offers a sophisticated solution to handling complex data structures. In the post, I walk through the implementation of this novel neural network using the Iris dataset as a practical example. The core components include: - Higher-Dimensional Topology & Manifolds: Leveraging a 3-dimensional hypersphere (S^3) to represent data in a higher-dimensional space. -Bayesian Nonparametrics: Utilizing a Dirichlet Process Mixture Model (DPMM) for dynamic clustering. - Neural Networks: Employing a TensorFlow-based feedforward neural network to learn and make predictions on the manifold. 🔍 Highlights: - Detailed step-by-step implementation. - Mathematical theory behind each component. - Practical insights into combining advanced machine learning techniques. If you're passionate about cutting-edge data science and machine learning, this is a must-read! Check out the full article here: Novel Neural Networks: Integrating Higher-Dimensional Topology, Manifolds, and Bayesian Nonparametrics Let’s push the boundaries of what’s possible with neural networks! 🌐📊💡 #MachineLearning #DataScience #NeuralNetworks #ArtificialIntelligence #DeepLearning #Innovation #Bayesian #Topology #Manifolds #AIResearch

2

📣 New work from our group introducing Seed-TTS, a family of large-scale speech generation models capable of synthesizing highly expressive, human-like speech. 📄 Paper: https://lnkd.in/gbRfMcm8 🔈 Demos: https://lnkd.in/gEfbNU4J 🎙️ Seed-TTS excels at zero-shot in-context learning for speech generation. We propose two techniques for speech factorization using self-distillation and preference biasing using reinforcement learning. We also present a non-autoregressive, fully diffusion-based variant of the model and show it achieves comparable performance to the language model-based variant while enabling speech editing capabilities.

17

FlexAttention generated a lot of buzz in the PyTorch Community this week. I want to explain why this API is an important milestone for the compiler team. FlexAttention is the first major Eager-like API powered by torch.compile() under the hood.  It opens up a new venue for deploying torch.compile() with few hassles to end-users. Since we compile a library code that is co-designed with the compiler, compile-backed Eager APIs avoid most scenarios of compiling user codes that may require expert attention (e.g., fixing graph breaks, recompilation, composability, or performance problems). The compiler team is exploiting more cases of "small-scope" compilation (also known as Higher-Order Kernels, HOK) to increase the adoption of torch.compile(), especially among power ML users who are also performance experts, enjoy the hackability of PyTorch Eager, and do not want the compiler to take control of the whole world. Stay tuned, and hope to meet you in PTC'24, 09/18-09/19, in SF (registration open, https://lnkd.in/eH8a-3J2). https://lnkd.in/eJqUda4N

334

2 Comments

I echo Pere Martra' s urgency for the European Union to advance in artificial intelligence and other key sectors. While Europe boasts incredible educational systems and social safety, its conservative approach often limits the entrepreneurial spirit seen in places like the United States and parts of Asia. These regions not only embrace but celebrate risk as a pathway to innovation, which starkly contrasts with the European caution that can hinder our global competitiveness in rapidly evolving fields such as AI and blockchain. To truly lead in technological innovation, Europe must strike a balance between its commendable focus on security and a more dynamic approach to business and technological advancements. Promoting a startup-friendly culture, reducing regulatory constraints, boosting venture capital funding, and fostering an ethos of 'intelligent risk-taking' are critical. Such steps are vital not just for keeping pace, but for steering global innovations, leveraging our rich intellectual and educational heritage to play a pivotal role in the next technological revolution.

3

Building with LLMs? Learn from two leaders who have built, and importantly **deployed** AI products 🎉 Tons of wisdom from these two ⬇

1

Karel D'Oosterlinck along the Contextual AI team just published a fascinating paper on improving language models like GPT-3 and ChatGPT. They found that current alignment methods can sometimes include irrelevant information, which can hurt performance. Their team developed new techniques that not only address this issue but also nearly double the performance of model alignment using the same amount of data. This gives AI developers more control over their models. Check out the paper here: https://lnkd.in/evdx3fyc 

33

1 Comment

The policies being driven forward by Thierry Breton around the EU AI Act have been utmost damaging for the European tech industry, and they need to be revised as soon as possible. We need to better balance driving European innovation through data collection and modeling with conflicting needs such as data protection and safety. We need to ensure we can turn these developments into businesses that can be the pillar of the next generation of companies that provide productivity growth in Europe. We have no other choice - either we do this here, or we will have to take over products developed in the US and in China, as it already happened with search engines, cell phones, and social networks. We wouldn't want to let this happen with AI applications in robotics or digital assistants. #euacc #euaiact #aiact

9

What a honour to take part in building resources to teach primary school students about AI! 🦾 Well done to the Code for Life team!! Resources are available for all primary teachers and you can find out more below ☺️ Here’s a teaser: https://lnkd.in/eeAvt3C9

22

1 Comment

A picture is worth... ~20 words? CLIP is limited at 77 tokens, which is already constrained, but when considering the "effective length", the paper "Long-CLIP: Unlocking the Long-Text Capability of CLIP" (https://lnkd.in/dky8Hf8N) puts it around 20 tokens. "This is because the majority of training texts are likely to be considerably shorter than 77 in the original CLIP model" When using CLIP with longer context, we need to put some ideas to work. We could use a summarization with spaCY or use Llama2, as presented in the paper "A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions" (https://lnkd.in/d6gEqJuR) (authors include Vasu Sharma). These methods are not cheap, but in the same paper it is stated that "average-pooling embeddings over these longer descriptions would be ineffective" so it looks there's no scape for additional compute costs...

19

1 Comment

🤖 The term #Agents is an overloaded one at this point in the development of the AI/ML field. In #reinforcementlearning, an Agent is an entity that takes in an Observation from the Environment, produces an Action or sequence of Actions, then receives a Reward telling it how it did against some reward function. 🌍 → 🗺️ → 🤖→ 🦾 → ⭐️ → ... An #LLM Agent is somewhat less well defined: we know it has something to do with outputting an action sequence of tokens to ultimately solve a problem prompted by a human, some of which may include: 🛠️ Calling out to tools (calculators, web searches, api calls) 🤔 Reflecting on the problem at hand 📝 Creating a plan 🔀 Shaping the control flow and structure of the DAG in which it's operating 🧠 Access to a memory buffer (vector DBs, graph DBs) Which is why I'm seeing many folks in the space start to describe these kinds of AI applications as "Agentic" as opposed to trying to pin down what exactly an Agent is. As the field develops we'll probably converge on a set of properties/capabilities/abstractions where Agents become better-defined. If you're interested in this space, come join Arize AI and Union.ai in Bellevue for two talks about evaluating LLM Agents and building Agentic RAG apps!

22