Richmond Alake

This Thurdsday is looking fun! A picture says a 1000 words by Sophie Finnigan - Senior DS, Faculty: ​While LLMs have harnessed the power of words and taken the AI world by storm, visual data offers a rich source of information. In a field previously dominated by CNNs, computer vision (CV) is now learning from LLMs, shifting towards transformer-based architectures and training foundational models. While large-scale models offer generalizability and versatility, many real-world use cases demand edge-deployment of CV models. Edge models address the challenge of resource-constrained compute and have come a long way in recent years. But can we leverage the power of large-scale model improvements for edge-compatible models? This talk will review CV at both ends of the scale and explore challenges and strategies to bridge the gap. Running multiple models on the same GPU, on spot instances by Oscar Rovira,  Mystic AI ​Oscar will unveil the top cost optimizations for ML inference in the cloud. Discover the power of GPU fractionalization and the benefits of using Spot instances, along with practical examples to boost throughput and reduce costs for your GenAI applications. 🔑Key Insights: - Learn about GPU fractionalization and its impact on ML inference efficiency. - Understand the advantages of utilizing Spot instances and overcome potential challenges. - Real-world examples showcasing increased throughput and cost savings for GenAI applications. Sign up here -> https://lu.ma/93ew7j0o

10

The Future of Data Engineering with LLMs Podcast  Let's see when we have time to

3

I've decided, finally, to create a "beginners guide to neural networks" series of posts. The first post is up now. I'd be delighted if you gave it a read. It's all based on my own work in the field so it comes from a practical rather than theoretical perspective. #ai #neuralnetworks https://lnkd.in/eYiq96cj

2

WAIT, it's not over; Meta just dropped the LLM Compiler! 🧑‍💻 > Beats GPT-4 on code size improvement and disassembly > Achieves 77% of the optimising potential of an autotuning search and 45% disassembly round trip 🔥 > Built on top of CodeLLaMa with improved code optimisation and compiler reasoning. > Allows commercial use Two model types: > LLM Compiler: the foundational models, pre-trained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimisations >LLM Compiler FTD, which is further fine-tuned to predict the best optimisations for code in LLVM assembly to reduce code size and disassemble assembly code to LLVM-IR > Perfectly emulating the compiler 20% of the time ⚡

621

13 Comments

🚀 🚀 Exciting news! OpenAI Strawberry (o1) has arrived, ushering in a new era of inference-time scaling in production. As Richard Sutton's Bitter Lesson reminds us, learning and search are the only techniques that scale indefinitely with compute. It's time to shift our focus to the latter. Here's why Strawberry is a game-changer: 1️⃣ Reasoning doesn't require massive models. By factoring out reasoning from knowledge, a small "reasoning core" can leverage tools like browsers and code verifiers, potentially reducing pre-training compute needs. 2️⃣ Compute shifts to serving inference. LLMs as text-based simulators can explore multiple strategies, similar to AlphaGo's Monte Carlo Tree Search (MCTS). 3️⃣ OpenAI's foresight on inference scaling is evident. Recent papers support this:   ➡ Brown et al. showed DeepSeek-Coder's performance jump from 15.9% to 56% with increased sampling.   ➡ Snell et al. demonstrated PaLM 2-S outperforming a 14x larger model using test-time search. 4️⃣ Productionizing o1 presents challenges: determining search termination, defining reward functions, and managing tool integration. 5️⃣ Strawberry creates a data flywheel. Correct answers generate valuable training examples, improving future GPT versions. This breakthrough could revolutionize AI development. Excited to see where this leads! #OpenAI #ArtificialIntelligence #MachineLearning #InferenceScaling #AIInnovation #DeepLearning #TechTrends Courtesy: Jim Fan

27

3 Comments

I recently spoke at the #london #anayticsengineering #meetup to share insights on evolving role of Gen AI in data engineering. Despite the click-bait'ish title, the talk focussed on how deeper integration of #GenAI into our workflow only offers positive reinforcement and assistance to the work of Data Engineers. This transformation is not just about adoption but adaptation, ensuring technology amplifies our strategic impact. You can watch the full session here : https://lnkd.in/emVccWTy #dataengineering #data #analyticsengineering

84

11 Comments

Since the world is constantly changing, static ML models are destined to fail in production. How can we prevent this? On May 16th, I will give a talk at the MLOps Community meetup in London about monitoring models in production. Specifically, I will focus on detecting the model failure, finding the root cause, and resolving it. The event is packed with interesting talks about ML and LLM applications. Thanks, Gleb Lukicov, for putting it all together! See you there! 👋🏻

12

1 Comment

Diary entry 4 of my Kafka transactions formal verification work. This entry is all about the initial Fizzbee spec and some benchmarks comparing it to TLC (the TLA+ model checker). https://lnkd.in/eifEayC7

13

If you're free tomorrow afternoon/evening check out the London Clojurians talk I'm giving about working with data in Clojure. Clojure's data ecosystem is super active and there have been tons of developments in the last few months we're excited to share: - 🤖 better integration with pure-JVM ML models - 📶 the new tablecloth columns API - 📊 the (very) beginnings of a new approach to visualizing data with a grammar-of-graphics style API - 🐍 python inter-op (not new, but something that isn't as well known) And lots more. I'll go over how to explore a few realistic datasets and gain insights using basic ML models and time series analysis, all in Clojure. Also it's free! https://lnkd.in/gkPXJwnY

12

2 Comments

It's not too late to join us tomorrow to learn how lean teams are using generative AI in their workflows! Hear from this all-star panel: Kate Jansons Johns, CEDS Philip Weldon Danny Frank Ann Marie Lane #ediscovery #generativeAI

30

1 Comment

Nice to see some positive news for cloud sovereignty last week! GitHub has said that by the end of the month, its enterprise cloud customers will be able to store their code data in the EU. This is a big deal - with sovereign clouds, developers are free to innovate knowing where their workloads are stored. They'll have greater control over international data transfers, which can leave information exposed in countries with different data protection rules. Control and data sovereignty are growing priorities for many organisations, and I expect others will be following Civo's and Github's lead in offering these solutions. Want to learn more about cloud sovereignty? Check out my recent blog on the topic here: https://lnkd.in/esC9_pyF

27

2 Comments

Out now! Part 2 of my blog post series, Understanding Neural Networks, in which I introduce the two main ways of connecting the artificial neurons, introduced in part 1, to create virtual brains. Hope you like it. Assuming, of course, that you see this post. The LinkedIn algorithm is, I believe, averse to sharing content that links to content outside the LinkedIn platform. However, I am equally averse to acquiescing to algorithms. https://lnkd.in/eQ9DEbvf

7

Great product ideas can come from anywhere. Just look at Stanley Kubrick's movie 2001: A Space Odyssey. Kubrick wanted to make sure that HAL knew that it was going to be disconnected. One problem though, Kubrick didn't know how HAL could overhear the private conversation between crew members. The solution is something that we, as Agile leaders, can learn from and apply in our companies. Check out this YouTube Short for the lesson. https://lnkd.in/gyNyUKVk

14

2 Comments

Working on some personal AI projects tonight (will share a GitHub repo soon). I’ve spent a few hours over several days in one specific chat with o1-preview. I asked it a particularly interesting question, and you should too. It’s great to have an AI sidekick to help get work done and be super supportive! #AI #QuestionsToAsk #Automation #PowerOfReflection #ChainOfThought

3

🚀 Building AI with Foresight, Not Hindsight: The EPSRC Framework 🚀 The UK's Engineering and Physical Sciences Research Council (EPSRC) is pioneering a proactive approach to responsible AI with its AREA framework. Forget reacting to ethical dilemmas after they arise – AREA encourages us to anticipate the impact of AI before it's unleashed! 💡 Why AREA Matters: 💡 - Proactive Ethics: Tackling ethical considerations head-on, ensuring responsible AI development from day one. - Taming Autonomy: Guiding the development of autonomous systems, which are rapidly changing our world. - Practical Toolkit: Equipping developers and policymakers with the tools to navigate the ethical landscape of AI. The EPSRC framework is shaping a future where AI is not just innovative, but also ethical and beneficial for all. Let's build AI with foresight, not hindsight! #responsibleAI #AIethics #innovation #EPSRC #AREAframework #futureoftech

8

UX has always been my most loved talent💞. After so many years working with startups and tech giants, I'm talking about UX and my new expertise, AI. this is one of my favorite talks, focusing on the connection between humans and AI and the future of the user experience with AI capabilities. I'm proud to speak in this wonderful Google event, Google I/O, in one of my favorite cities, Edinburgh. Join me! it's going to be fun! #google #googleIO #UX #AI

1

"Tomorrows Software Engineer" was a recent article in the British Computer Society, indicating trends for software engineers to consider. After the obvious (#1 cloud, #2 serverless/containerisation) comes "composable architectures"... <<Traditional software engineering thinking around application development has been focused on stability of the production software. With the business imperative for agility now prominent, the need to change is a great concern. Software engineers need to use composable thinking, treat everything as changeable and foster exploration and creation to support new business opportunities.>> Interesting to see this as #3. But maybe not surprising given initiatives like MACH in e-commerce, or the increasing libraries and catalogs of connectors. components and processes in low-code and model-driven tooling. Also interesting: this reads like a quote from a Gartner report! 😉 Link for BCS members: http://bit.ly/3zLPgAa

11

We set up infrastructure for managing Metaflow at Nutrien and the AWS integration is so profoundly sweet. The batch and secretsmanager integration not withstanding, the S3 module, which I believe is just a wrapper around the odd cantrips that exist within boto3, is so seamless. The `get_many` and `put_files` functions are smooth and quick. I did not even know there was such a thing as a S3 Transient Error from slamming S3 with put requests, but Metaflow does and it gracefully handles the retries out of the box. Truly S-tier python package for working with AWS. https://lnkd.in/gTe3ApAc #AWS #Metaflow #MLops #boto3

17

https://lnkd.in/epNxY4fb I've been thinking about the way that large language models seem to be able to generalise to unseen problems lately and I've been thinking that the spectral bias of neural networks is probably a key component of that. Between this and the double descent problem, it's becoming increasingly clear that massively overparameterized neural networks are biased towards simple and smooth solutions to a problem. Often, the simplest representation of a domain is also the 'correct' one. Consider a dataset that consists exclusively of uniformly sized black dots against a white background. The most compact encoding that contains all the information for this dataset is just the x and y co-ordinates of the dot. These values also happen to be the real generator of the dataset. It’d be reasonable to expect a beta VAE to converge on this representation based on spectral bias and it would be the most generic encoding. Is a similar thing happening in large language models? When presented with conversations and papers about scientific matters where the next word depends on what is true, perhaps the simplest policy is to understand the topic and to model reality first - language second.

3