Sean O'Neill

Technically having a long context is still not enough

9

1 Comment

The availability of training data is the biggest unmet need in AI. After hundreds of conversations around training data, I've put together the patterns that are most commonly holding back data deals today https://lnkd.in/eEg8KWUw

141

2 Comments

The recent Vansbourne report on Business Wire has cast a spotlight on the financial intricacies of generative AI projects, revealing a concerning trend of budget overruns. According to the report, a staggering 80% of these projects encounter significant challenges due to issues with data availability and quality. This problem often stems from the initial proof of concept (POC) phase, where AI/ML engineers swiftly configure the project using available data to demonstrate the LLM's capabilities in handling basic queries. This rapid demonstration can lead to overly optimistic expectations from management, based on these preliminary results. As projects transition from POC to full-scale applications, the real challenges emerge. Management's expectations, cemented by the initial successes, anticipate that the LLM will adeptly handle all queries related to the business, products, or services. However, the reality of accessing and refining the necessary data—often buried in complex databases or scattered across unstructured formats like social media feeds—presents a significant hurdle. The engineering effort to make this data usable, involving its integration and processing through MRAG pipelines, is commonly under-budgeted, leading to the first major area of cost overrun. Moreover, projects typically require ongoing development in areas like reinforcement learning, critic models, and incremental fine-tuning—areas still under active research but crucial for the LLM's success. The necessity for robust guardrails, an often-underestimated aspect, adds another layer of complexity and potential expense. Given these extensive challenges, which are frequently not fully addressed, the report’s estimate that projects typically exceed their budgets by 63% appears conservative. Indeed, the actual figures could be much higher, potentially in the triple digits. In response to these industry challenges, Innovation Hacks AI has developed a structured approach to initiating generative AI projects. This process begins with a discovery session that focuses on identifying the most pertinent business use cases. By evaluating potential solutions based on ROI, cost, implementation complexity, and time, and engaging in comprehensive brainstorming, the team strategically prioritizes solutions. Furthermore, Innovation Hacks AI employs a robust platform capable of ingesting data from 80 different sources. This system cleans the data and offers a Discovery Workbench that enables users to query, explore, and realistically assess what solutions can be built. This proactive and structured approach ensures that projects are set up for success from the outset, addressing key data challenges and aligning project goals with realistic business impacts. By doing so, Innovation Hacks AI not only mitigates the risks of cost overruns but also enhances the overall success rate. Link to the report: https://shorturl.at/W0gLn For inquiries, reach out to [email protected]

6

If you're wondering how AI will be monetized, and how we'll come to pay for it, this article by Abde Tambawala and https://lnkd.in/gYJxgPSS is worth a read. If you're building an AI-driven product and are thinking about how you'll price and package it, this is a must read.

7

1 Comment

🌐 In the world of supply chain technology, the power of conversation can unlock doors we never knew existed. I recently stumbled upon a fascinating article from TED Ideas about the art of talking to strangers https://lnkd.in/gwYksjZE and couldn't help but draw parallels to our industry. In an ecosystem as interconnected as supply chain technology, each interaction, no matter how brief, has the potential to spark innovation or forge lasting partnerships. The article shares invaluable insights on approaching conversations with openness and curiosity—qualities that can lead to breakthroughs in solving complex logistics challenges. Here are a few takeaways I found particularly relevant to us in the #SCTech world: Embrace the Unknown: Just like in a conversation with a stranger, diving into uncharted technological territories can be daunting but rewarding. It encourages innovation and problem-solving. Listen Intently: Active listening can reveal underlying issues in supply chain processes that might not be apparent at first glance. Ask Questions: Inquisitive minds lead to breakthroughs. Engaging with experts or even novices outside your immediate circle can provide fresh perspectives on age-old problems. I encourage you to read the article and think about how its principles can apply not just in your personal life but in navigating the complex landscape of supply chain technology. Let's approach each conversation, each opportunity to collaborate, with the same curiosity and openness we would with a new acquaintance. Who knows what solutions we might uncover or what partnerships we might forge? #SupplyChainInnovation #Networking #Collaboration #SCTechSherpa

4

As innovators, we are constantly trying to push the boundaries. Regulation and red tape only tends to come into peripheral vision (at best) when you're in the throes of trying to make new, cool things happen. But with AI becoming more and more ubiquitous in the enterprise landscape, we need to keep more than one eye on the rapidly evolving regulatory environment. So what should we Aussies be taking from the recent EU AI Act? - Increased regulation is coming here. A matter of when not if. In the early blockchain days it felt pixelated, where regulatory pressure hovered over the shoulder of innovators until the picture sharpened over time. What will AI regulation look like for Australian businesses? Will regulation strangle innovation or will we find the right balance? - By categorising AI based on risk levels, the EU are trying to encourage innovation while safeguarding against potential harms. - Not just humans-in-the-loop but humans to the fore! The EU AI Act places a strong emphasis on protecting human rights and ensuring AI is for the benefit of of society. I liked the quote from the recent Figma conference on this topic: "AI won't take your job, AI will be your job". We can't predict the future, but the EU AI Act is a good starting point for understanding how things are progressing elsewhere and what is to come here in Australia. Let’s shape a future where AI is both innovative and responsible - and where human impact is a primary consideration in our innovation workshops not an afterthought. Veloceo #AI #AIEthics #EUAIACT #Innovation #Technology #Australia #ResponsibleAI #FutureOfWork #Veloceo

10

My thoughts on the RAG Survey Paper I recently read. Highly encourage you to check out some of the emerging patterns. https://lnkd.in/e43T7QSc

75

7 Comments

Parade has led the way in driving collaboration for freight brokers with their shippers & carriers, and GenAI has been a core accelerator to realizing the full potential of digital freight matching capabilities. One challenge customers face is driving end-user adoption with new web and mobile apps (you'd be surprised how many there are in freight). By leveraging LLMs on top of our digital freight matching workflows, we've met the carrier where they are and how they are willing to interact—whether it's an email conversation or a click in a mobile app.

47

How to monetize AI is a tricky and important decision for any company, and there's no quick fix that works for everyone. The good news is there are frameworks that can help! This fantastic write up from Abde and Sam is a must read for anyone trying to monetize an AI-based product.

5

1 Comment

Automakers are desperate to transition from being Manufacturers to being Data and Services businesses. Unfortunately, GM and the rest of the automakers are too late for the age of corporate data ownership. Individuals own their data and the role of corporations is to facilitate the sale of that data for a %. Corporations that cling to the old paradigm will be broken in court. Link: https://lnkd.in/gyyk_WGT

6

Matthew Red is stirring the pot with some bold predictions about AI's impact on supply chain management, and it's not what you might expect. He's not talking about minor tweaks or incremental improvements. He's forecasting a seismic shift where AI could automate up to 90% of tactical decisions. Think about it—decisions like determining inventory levels, shipping logistics, and even predicting how weather affects demand could soon be automated. Are we ready to embrace this automation revolution, or are we clinging to outdated methods that are ripe for disruption? Listen to the full episode to explore Matt's insights and decide where you stand on the AI revolution in supply chain.

17

This was an inspiring article that, once again, affirms for me the need for AI regulation. The recent formation of a group by AI dataset licensing companies in Germany to promote ethical data sourcing is certainly a significant step forward. Ethical data sourcing ensures that AI-driven solutions are transparent and user-centric. Crucial to this effort is the role of UX/UI design and data management. Effective UX/UI design makes AI tools intuitive and accessible, while robust data management practices guarantee transparency and integrity. At Defijn, we understand the importance of ethical design principles in developing AI-driven solutions. Our commitment to user-centric design ensures that our digital products not only meet but exceed user expectations. Take a look at the story behind this development in the link below and let me know your thoughts on the role of ethics in data handling.

11

A good article by my colleague Lucas Schellenberg. One highlight: "Operational excellence in the age of AI is not just about automation but optimizing the entire production lifecycle to enhance quality, increase speed, and reduce waste." I would suggest, more broadly, that the introduction of AI into the production process will continue the contribution of technology as a factor in the production function -- and will extend and deepen technology's presence in growth theory (growth reality) to a degree that even Nobel Laureate Robert Solow could not have envisioned. This will, of course, likely entail 'factor substitution' to a degree not seen since the Industrial Revolution. That will be a difficult journey for many firms in many industries. But firms that do not embrace factor substitution (or cannot do so due to contractual arrangements governing existing factors of production) will likely fall further and further behind firms that do embrace it. That is not only unpleasant for particular firms, but it will change the competitive capability of entire nations. -rd

16

1 Comment

Catching up on some of the great presentations from #PEPR24 presentations, and came across Damien Desfontaines’ talk on synthetic data. I obviously disagree with lots of it, but I think Damien generally makes his points in good faith. So I’m going to (try to) do the same in response. The part of his talk that stood out to me as most obviously incorrect and uncharitable was the discussion of the incentives for improved privacy measurements in synthetic data. I’ve included his graphic below as a summary of his argument that there’s no commercial incentive to improve empirical privacy evaluations of synthetic data for context. Sharing three thoughts: 1) If there’s no commercial incentive for better privacy evaluations with synthetic data, differential privacy companies shouldn’t exist. The complexity and form factor of differential privacy imposes enormous incremental friction to configure and use relative to synthetic data. If there’s no commercial case for better synthetic data privacy metrics where none of that friction exists, then how could there possibly be commercial demand for more friction and better privacy metrics — the asserted value of differential privacy? As far as I know, Tumult Labs isn’t a charity, so someone there must believe there’s a market for stronger privacy guarantees — we agree with that, even if Damien doesn’t 😂 2) That’s the theoretical argument, but I’m a synthetic data guy, so I’ll make an empirical argument, too: prospects and customers care deeply about the rigor of privacy tests for synthetic data and it is a buying criteria in *every* deal in our funnel. For *privacy* use cases (more on this in the next paragraph), buyers are not just asking for better metrics; they’re demanding it. My data on this is MUCH better than Damien’s — since he doesn’t actually sell synthetic data to customers — and he’s completely wrong. 3) Despite their website copy, most synthetic data companies are delivering to non-privacy use cases *at least as often* as they’re delivering to privacy-enhancing analysis use cases. Synthetic data is a really useful technology for data quality work, test data management, and anonymization. And in practice, most synthetic data platforms are most heavily used for data quality work (imputing NULLs, class rebalancing, dataset augmentation, etc.). Privacy is — at best — a secondary criteria for these use cases, so the basic privacy metrics are sufficient. At Subsalt, we work exclusively on anonymization use cases and we win deals on the basis of our privacy work. We’re living proof that there’s a business case for more rigorous privacy work in synthetic data. I don’t expect this to solve the eternal (and cursed) debate about empirical vs. theoretical privacy metrics, but I do think it’s worth clarifying that if you want to know about the structure of a market — in this case the market for synthetic data — you should listen to someone that knows at least something about that market.

13

5 Comments

To start, I have to concede that the quote from our website (yes, Subsalt is the anonymous company in Damien Desfontaines’ post; I don’t think he even tried that hard to make it anonymous 😂) overstates the real risks of re-identification in synthetic data. The copy will be changed, because it actually is a good example of sloppy privacy messaging. Mea culpa. But despite being a pretty good dunk, it’s not outcome determinative on the issue we’re actually discussing. The relevant question is: is there market demand for more rigorous privacy metrics in the synthetic data space. The answer is unambiguously “yes.” Customers are not satisfied and they justifiably don’t believe the most commonly used metrics add up to “anonymization.” So why haven’t most synthetic data vendors adjusted their “privacy scores” to accommodate this dissatisfaction? The most commonly referenced companies in synthetic data go to market with freemium versions of their product that can be adopted and used by end-users without a multi-stakeholder purchasing process. The users that actually adopt these tools tend to use them disproportionately for *non-privacy* use cases. And more stringent privacy tests are not neutral to the UX for these users: they’re a source of friction and complexity. So the versions of these synthetic data products you can try off-the-shelf stick to basic privacy metrics that don’t introduce friction. The dirty little secret in synthetic data is that VERY few synthetic data workloads in production are premised on sufficient data anonymization through synthesis. That’s because synthetic data deals where anonymization is the primary use case have traditionally died in the risk, privacy, legal, and compliance departments *because those stakeholders want more rigorous privacy guarantees* that conflict with the needs of the non-privacy users of synthetic data platforms. If every large organization on earth had deployed a synthetic data platform for privacy use cases in spite of this weakness, you’d be able to conclude that there’s a lack of demand for more rigorous privacy work in the space. But all of the synthetic data vendors (combined) have a relatively tiny penetration into the market. The biggest source of today’s “non-consumption” (e.g. the fully uncaptured portion of the addressable market) is the misalignment between the standard synthetic data form-factor and privacy measurements and buyer demand for anonymization. Despite the sloppy copy on our website, that is the latent demand we hear from prospects and customers every day and we take the problem very seriously. I’d be happy to share more about our privacy metrics and approach, because I share the deep concerns and commitment to providing rigorous privacy preserving tools so organizations can do more with sensitive and regulated data — even if I don’t share the skepticism of all empirical privacy metrics.

9

2 Comments

🔥 2-Min Survey: Help Fix the Insights Pipeline, Boost Your Next Review At Ara Platforms we're *obsessed* with the data -> insight -> action pipeline and removing roadblocks. On the data (insight producer) side, we want to make it easier to generate and publish useful insights. On the stakeholder (insight consumer) side, we want to make it easier to ask questions and get useful answers quickly. The process feels like a broken market—data people produce insights that don't get used while stakeholders don't can't get insights to important questions in a timely manner. If you're on either side of this equation I'd love to chat—please reach out via DM! If you can't it would mean a ton if you could fill out one of these quick Google forms. The end goal is to improve everyone's performance reviews Insight producers (data scientist, analysts, researchers): https://lnkd.in/emtMV6um Insight consumers (product managers, leadership, anyone who uses data to make decisions): https://lnkd.in/eEJ3Wse7

4

1 Comment

Today, Reuters reported on Russia’s election-related disinformation activities in the EU. This week, it was reported that Russia’s influence groups used OpenAI to spread false and harmful information. The US Election 2024 is coming. It will be the most important political event on Earth. And it is not only about competing spins, or narratives clashing. With flows of hyperinformation, with the intent of malign actors to deceive and manipulate attention, and with the need of public and private organizations to be prepared and react fast—it is about staying vigilant and tracking the pulse of digital discourse. This is where AI can help. Mantis Analytics is a risk management company. To help you navigate the election-related informational risks landscape, we created our US Election 2024 AI-driven monitoring solution. If you analyze election dynamics, research main trends, want to follow the debates, or detect emerging narratives and threats, feel free to contact us.

7