Igor Bobriakov

🔥 2-Min Survey: Help Fix the Insights Pipeline, Boost Your Next Review At Ara Platforms we're *obsessed* with the data -> insight -> action pipeline and removing roadblocks. On the data (insight producer) side, we want to make it easier to generate and publish useful insights. On the stakeholder (insight consumer) side, we want to make it easier to ask questions and get useful answers quickly. The process feels like a broken market—data people produce insights that don't get used while stakeholders don't can't get insights to important questions in a timely manner. If you're on either side of this equation I'd love to chat—please reach out via DM! If you can't it would mean a ton if you could fill out one of these quick Google forms. The end goal is to improve everyone's performance reviews Insight producers (data scientist, analysts, researchers): https://lnkd.in/emtMV6um Insight consumers (product managers, leadership, anyone who uses data to make decisions): https://lnkd.in/eEJ3Wse7

4

1 Comment

I'm a computer scientist who now works in biotech and writes poetic blogposts in Russian. How come? What do AI, poetry, and biology have in common? Language. At school I hated Russian grammar class. Learning a language by its rules is like learning to dance from a book. Even if you master a thousand rules and a million exceptions, it doesn't guarantee you'll write well. A natural language isn't designed; it's evolved. It wasn't developed, it developed. Studying grammar helps, but the only real way to master a language is through developing intuition – by reading, writing, and speaking a lot. Turns our the same applies to computers. For years, we tried to teach them natural language through complex logic, grammar rules, and it went even worse than my Russian lessons. Remember those dumb chatbots from a decade ago? But then, large language models (AI) arrived, and instead of programming on linguistic rules, we just had them "read" a lot, basically every text in existence. And now, ChatGPT writes in almost every natural language better than humans. But the most exciting part lies ahead. See, biology is also a language in a way — the Language of Life. Amino acids are the "letters," proteins the "words," biochemical pathways the "sentences," and every cell is an "epic" of 94 million words. This language, too, is the product of evolution, yet far more complex than Russian. For centuries, we've tried unsuccessfully to decode it by rules, and we've only managed to create something akin to the first chatbots. But soon, AI will change everything. At OPTIC we're building a translator at the crossroads of AI and biology. I believe that once we find an optimal, scalable way to collect daya and let machines read all the existing "texts in the language of life," we'll develop a model capable of writing in it. Can you imagine the possibilities when the "ChatGPT moment" happens in biology? A cure for every disease? Personalized therapy for everyone? An end to aging? — "Ok, BIOPTIC, cure Parkinson's disease." — "HDEKKPKNFFGWKDHEQCMHPKSIM…🫶" This is what drives me to work every morning.

443

15 Comments

MarkovML was founded on the belief that your data is the ultimate competitive advantage. As Peter Norvig famously stated, "𝙈𝙤𝙧𝙚 𝙙𝙖𝙩𝙖 𝙗𝙚𝙖𝙩𝙨 𝙘𝙡𝙚𝙫𝙚𝙧 𝙖𝙡𝙜𝙤𝙧𝙞𝙩𝙝𝙢𝙨 𝙗𝙪𝙩 𝙗𝙚𝙩𝙩𝙚𝙧 𝙙𝙖𝙩𝙖 𝙗𝙚𝙖𝙩𝙨 𝙢𝙤𝙧𝙚 𝙙𝙖𝙩𝙖." This adage is particularly relevant in today's landscape, where Large Language Models (LLMs) are becoming commoditized. This has again been proven by the LLama3.1 paper (links in the comments). The difference between success and failure often hinges on having right data and how effectively it's managed and utilized by business. Here are some of the best practices to maximize value for your enteprise. 👉 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗗𝗮𝘁𝗮 𝗤𝘂𝗮𝗹𝗶𝘁𝘆 ✨ Regular Audits: Ensure data accuracy and integrity through consistent and thorough audits. ✨Validation and Updates: Ensure quality metrics are in place to ensure consistency and relevancy. 👉 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗬𝗼𝘂𝗿 𝗗𝗮𝘁𝗮 𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆 ✨ Inclusive Data Collection: Diversify your data inputs to ensure your datasets genuinely represent global perspectives and scenarios, enhancing the applicability and relevance of your models. ✨ Broad Representation: Strive for datasets that encapsulate a variety of demographics, industries, and use cases. This diversity helps improve the generalization of your models across different contexts and applications. 👉 𝗘𝗻𝘀𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗶𝘀 𝗨𝗻𝗯𝗶𝗮𝘀𝗲𝗱 ✨Bias Detection: Models are biased because the data used to train them is. Identify & fix the root cause. ✨Fairness Metrics: Establish fairness metrics and ensure your models deliver equitable outcomes across all user groups. 👉 𝗔𝗱𝗼𝗽𝘁 𝗮 𝗣𝘂𝗿𝗽𝗼𝘀𝗲-𝗗𝗿𝗶𝘃𝗲𝗻 𝗔𝗽𝗽𝗿𝗼𝗮𝗰𝗵 𝘁𝗼 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 ✨Clear Objectives: Ensure you establish your expectations from the model and understand the trade-offs. ✨Alignment with Business Goals: Models scale efforts constrained by time and resources. Have clear business metrics and measure model performance against these to maximize value. 👉 𝗘𝗾𝘂𝗶𝗽 𝗬𝗼𝘂𝗿 𝗧𝗲𝗮𝗺𝘀 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗥𝗶𝗴𝗵𝘁 𝗧𝗼𝗼𝗹𝘀 Give your knowledge workers the right tools to automate routine data tasks and efficiently transform raw data into strategic insights so the business can make informed decisions faster. Superior data and the right tools aren't just assets; they're your competitive edge—invest in them for long-term success. #DataQuality #DataDiversity #AI #MachineLearning #BusinessStrategy #MarkovML #GenerativeAI #GenAI #Llama3.1

31

3 Comments

If you're wondering how AI will be monetized, and how we'll come to pay for it, this article by Abde Tambawala and https://lnkd.in/gYJxgPSS is worth a read. If you're building an AI-driven product and are thinking about how you'll price and package it, this is a must read.

7

1 Comment

We used Bunka.ai to summarise the Nectar Dataset, a collection of prompts designed for model fine-tuning. Understanding the content of those datasets is key to understanding model outputs, but this process takes time and resources when done manually. We found that those prompts are quite diverse, covering topics ranging from society to mathematics. This diversity is definitely an improvement compared to other datasets, which sometimes tend to focus too narrowly on Data Science subjects, making it difficult for models to generalize to everyday topics later on. Great work from Banghua Zhu and the team at Berkeley. Thanks Ciprian Bangu for the Use Case. BunkaMap: https://lnkd.in/gfUFJ8jp Bunka package: https://lnkd.in/gNbvTTEX Nectar Dataset: https://lnkd.in/ggT-V2zS

27

1 Comment

Thrilled by Dr. Adam Bujak's Splendid Presentation at ABSL Conference in Cracow! Adam Bujak's recent presentation at the ABSL conference in Cracow was nothing short of inspiring! His insights on navigating the treacherous waters of the AI swamp left the audience spellbound. As he eloquently shared strategies to avoid being drawn into the murky depths of AI complexity, I could feel the collective nod of understanding from the crowd. Adam's clarity and passion were contagious, igniting a fire within each of us to tackle the upcoming challenges head-on. And here's the cherry on top: KYP.ai, the remarkable tool he introduced, promises to be our trusty guide through this AI wilderness. With KYP.ai, we're equipped to address the very challenges Adam highlighted. It's like having a compass that always points toward success! So, let's embrace the future with open arms, fueled by Adam's wisdom and powered by KYP.ai. Together, we'll conquer the AI swamp and emerge stronger than ever! #ABSLConference #AIExcellence #KYPaiMagic

63

3 Comments

As AI applications and agents capabilities improve, the demand for monetised high quality data from publishers and content platforms will skyrocket. 🚀 This will include data being served during inference. In order to choose context relevant data, semantic retrieval is key. The recent paper behind the Contextual Document Embeddings(CDE) model introduces innovative methods for enhancing embeddings for domain specific retrieval: 1️⃣ Contextual Embedding Architecture: The encoder architecture is modified to incorporate information about neighbouring documents during the embedding process. This allows the model to understand how a documents content is related to the rest of the corpus improving domain specific retrieval performance. 2️⃣ Domain-Adaptive Training: During fine-tuning, the corpus is clustered into self-similar pseudo-domains allowing the model to learn variations within a domain. This allows a singular pre-trained model to generalise across different domains. 3️⃣ Multi-stage Embedding Process: During the embedding process a group of documents are passed to the model where concatenated embeddings representing sub-domains is generated. Embeddings from individual documents are then encoded on top of the concatenated embeddings to produce a final embedding. The results? 📊 State-of-the-art performance on the MTEB benchmark for small models (<250M parameters) without needing large batches or mining hard negatives. The Domain-Adaptive Training can be done offline which speeds up inference times. In simple terms: you get top results without needing massive resources—making it easier and faster to implement in real-world retrieval scenarios. 📑 Read the paper here: https://lnkd.in/gn6P8NTR #AI #DocumentEmbeddings #Publishing #TechInnovation #AIInfrastructure #ContentMonetization

13

Exciting breakthrough in NLP: Graph Language Models (GLMs) 🚀 Just discovered GLMs - a powerful fusion of traditional Language Models and Knowledge Graphs. This innovation offers: • Rich text understanding from pre-trained LMs • Enhanced relationship representation via graph structures • Improved performance in complex tasks (e.g., question answering, information retrieval) • Potential for more context-aware and accurate AI systems Key benefits: 1. Faster development cycles by leveraging existing LM capabilities 2. More robust and interpretable models 3. Simplified integration of domain-specific knowledge 4. Scalable architecture for handling large-scale data GLMs could revolutionize how we build intelligent systems, offering a more holistic approach to language understanding and knowledge representation. Have you experimented with Knowledge Graphs or Language Models? What potential applications do you see for GLMs in your work? #NLP #AI #MachineLearning #KnowledgeGraphs #TechInnovation

1

A Data Geek’s Take on GenAI: Building a data product people love is not an easy task. Curious how LLMs and RAG are changing the game? (2 minutes read) https://lnkd.in/d8Jiap96 #DataExperience #GenAI #PersonalizedData #DataStrategy #RAG

8

All the hype around GenAI means that every software maker seems to be stuffing chat interfaces into their products whenever they can. For the most part, the jury is still out on whether this is a good idea or not. However, design goes deeper than just the user interface, so it’s also useful to know about how the designs interact with the rest of the software. Once you move beyond chatbots into things like agents, there are also thorny questions around which bits of your workflow should still be done by a human, and which bits can be completely automated. True insight in this context lies in a gray area, across software, UX and AI. I just came across a wonderful podcast on designing modern AI applications https://lnkd.in/d_CuHM-C Here are my pickings from this podcast 𝘊𝘰𝘨𝘯𝘪𝘵𝘪𝘷𝘦 𝘢𝘯𝘥 𝘊𝘰𝘮𝘱𝘰𝘴𝘢𝘣𝘭𝘦 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵𝘶𝘳𝘦: Adopting a cognitive and composable architecture allows for modular, flexible AI systems. This design ensures that different components can be updated independently, providing a seamless and adaptive user experience. 𝘛𝘩𝘪𝘯𝘬 𝘰𝘧 𝘎𝘦𝘯𝘈𝘐 𝘚𝘰𝘭𝘶𝘵𝘪𝘰𝘯𝘴 𝘢𝘴 𝘚𝘬𝘪𝘭𝘭𝘴, 𝘕𝘰𝘵 𝘑𝘶𝘴𝘵 𝘊𝘩𝘢𝘵 𝘐𝘯𝘵𝘦𝘳𝘧𝘢𝘤𝘦𝘴: Instead of viewing GenAI solutions as isolated chat interfaces, consider them as a collection of specialized skills, similar to Alexa. Each skill can perform specific tasks, contributing to a cohesive and versatile system. 𝘏𝘶𝘮𝘢𝘯 𝘪𝘯 𝘵𝘩𝘦 𝘓𝘰𝘰𝘱 𝘍𝘪𝘳𝘴𝘵, 𝘈𝘶𝘵𝘰𝘮𝘢𝘵𝘪𝘰𝘯 𝘚𝘦𝘤𝘰𝘯𝘥: Integrating human oversight initially ensures that AI systems learn accurately from feedback and reduce errors. This approach builds trust and reliability before full automation is implemented. 𝘓𝘓𝘔 𝘧𝘰𝘳 𝘕𝘦𝘹𝘵 𝘉𝘦𝘴𝘵 𝘞𝘰𝘳𝘥, 𝘈𝘐 𝘈𝘨𝘦𝘯𝘵 𝘧𝘰𝘳 𝘕𝘦𝘹𝘵 𝘉𝘦𝘴𝘵 𝘈𝘤𝘵𝘪𝘰𝘯: Use LLMs for tasks that involve predicting the next best word, such as text generation. For decision-making and process automation, employ AI agents to determine the next best action. 𝘏𝘺𝘣𝘳𝘪𝘥 𝘈𝘱𝘱𝘳𝘰𝘢𝘤𝘩: 𝘋𝘦𝘵𝘦𝘳𝘮𝘪𝘯𝘪𝘴𝘵𝘪𝘤 & 𝘗𝘳𝘰𝘣𝘢𝘣𝘪𝘭𝘪𝘴𝘵𝘪𝘤 𝘈𝘐: Combining deterministic AI (rule-based systems) with probabilistic AI (ML/LLMs) enhances effectiveness and reliability, depending on the use case. This hybrid approach leverages the strengths of both methods. 𝘗𝘳𝘰𝘮𝘱𝘵 𝘰𝘳 𝘘𝘶𝘦𝘳𝘺 𝘊𝘭𝘢𝘴𝘴𝘪𝘧𝘪𝘤𝘢𝘵𝘪𝘰𝘯: Accurately classifying prompts or queries ensures the right model or API is used to serve the request. This improves efficiency and the relevance of responses, enhancing user satisfaction.

10

Graphy now powers over 100,000 data storytellers 📈 This quote from Stan Massueras really nails it: "Great ideas are only impactful when communicated clearly. Graphy enabled us to transform data into compelling stories, and those stories into meaningful action." Thanks to everyone using Graphy to turn data into stories that drive decisions, inspire teams, and make complex insights easy to act on ❤️

54

10 Comments

What does the hype about #AI mean? Here is the answer I gave at Calcalist כלכליסט's event: "When we gained early access to DALL-E 2’s beta version in 2022, it took just five minutes of exploration to understand our industry was going to drastically change. Lightricks’ management team and I immediately grasped that we were witnessing a paradigm shift in our domain. The way we approach pixel creation, editing, and manipulation was being revolutionized before our eyes. Complex challenges that once required PhD-level expertise were suddenly within reach of a much broader audience. For us this means unlocking new business opportunities and avenues for innovation. The AI hype isn't just noise - it's the sound of paradigms shifting and new possibilities emerging, fostering environments where human creativity and AI capabilities can synergize.” More from my conversation with Daniel Shichman of WSC Sports and CTech by Calcalist editor Elihay Vidal in the first comment. (📸 orel cohen)

90

3 Comments

🚨 Launch Alert 🚨 Team, after months of hard work we’re super excited to introduce Graphy AI. We all share graphs and communicate with data, some of us on a daily basis. But, the process of creating graphs and communicating insights is difficult and time-consuming. People spend time in meetings just explaining the data 🤯 rather than focusing on taking action. With Graphy AI, you can go from a spreadsheet to an actionable graph in just 3 easy steps: 1️⃣ Add your data or generate a dataset with AI. 2️⃣ Graphy will create the best charts from that data. 3️⃣ It will then explain your key insights. The charts are ready to present or share in the tools you already use. We’re live on Product Hunt, and we’d love your support: https://lnkd.in/dj-8tCH4

106

19 Comments

We just published the much-anticipated part 2 of our experiments, which revolves around our analysis of improving Agentic function calling accuracy across GPT-4-Turbo, Claude, Opus, and Haiku. Surprisingly, Claude is worse than Opus and Haiku, while GPT-4 does an amazing job and maintains a lead over all of them. We write in detail about improving function calling accuracy from 35% to 75%. If you are building and working on similar stuff, do let me know about your experience. https://lnkd.in/gb_mWdkX

37

I’m super excited to speak at the Turing Fest on the 9th of July. My talk will start with a simple question: How can you measure the value of a dashboard or a report? Answer: The value of a dashboard or a report is measured by the useful actions it prompts. But sadly, the reality is that after investing hundreds of thousands of dollars in your data stack, data is not as useful as it should be, and people don’t act on it. My presentation will focus on helping you fix that! By the way, the title I’m using is from my favorite tweet from Gwenyth: https://lnkd.in/dQkxCkG5 Join my talk, or DM me if you haven’t got your ticket yet. I’ve got a discount code for you ;)

34

3 Comments

Here's a very relevant product update, coming off a BIG update from Open AI yesterday. This week, we put elbow grease into making Zoë, our AI data analyst, better understand when she can’t do something (or can't do it well). Language models are notoriously overconfident (we’ll see over the next few weeks if Open AI’s o1 release changes that). To fix that, we now have several different systems we use to give Zoë better ability to realize when she either can’t do something or is doing something that is reasonable but might not be accurate. That lets her caveat to the user both if she can’t do something entirely and, if she can, but she’s not confident that she understood the user’s ask. Here's an example where she creates a personal field, then knows that she is not able to promote it to the global cognitive layer, and directs the user on how to promote it.

18

4 Comments

🔍 Sara Nóbrega's new article on The Ultimate Guide to Finding Outliers in Your Time-Series Data hones in on an intriguing aspect of machine learning - outliers detection in time-series analysis. This comes across as a pivotal refinement in the field of Data Science. 🧩 Outliers, those pesky gatecrashers in our data, with the notorious potential to misguide forecasts, jumble up statistical models, and obstruct decision-making processes. When left unrecognized and unmanaged, they can present skewed viewpoints, leading to misguided decisions. 🔬 As Data Science practitioners, dealing with outliers must be deemed as not just an optional task but rather an obligatory step in data preprocessing paths. ▫️Yet, detecting outliers is often easier said than done. ▫️Effectual machine learning methods, streamlined tools, and proficient strategies are thus of the essence. 💼 Deep-diving into machine learning from a professional perspective - An erratic outlier today can be a breakthrough opportunity tomorrow. A careful, more precise analysis can weave disruptive outliers into significant patterns and trends, unveiling hidden insights waiting to be discovered. 📈 From the standpoint of an optimist - We're drawing closer to a time when outliers are no longer mere deviations, but rather outliers become crucial messengers of change and deviation. 💡My take on Sara's informative guide is twofold: 1️⃣ It's exciting to note that the boundaries of what once were seen as challenges in machine learning are now avenues for innovation and discovery. 2️⃣ It points towards the future where our focus will not simply be on attaining superior model performance but also on comprehending those unequalled elements that enkindle that performance. ☀️ Above all, let's embrace our outliers. After all, in an era of incessant transformation, they may be the sparks to ignite our pathways to advancement. #DataScience #MachineLearning #OutlierDetection #TimeSeriesAnalysis #SaraNobrega

1

Faster, more accurate, less data needed = BaseModel.AI

5