Nirmal Thacker

No one likes running out of reservation time when running an AI and ML on AWS Here’s how to take your AWS capacity blocks game to the next level… If you don’t want to reserve too much buffer time time which directly means overspending, reserve time closer to what you think you need and use a checkpoint + restore based strategy to protect from losing your work in the case that you need more time than your reservation allows. On the flip side, reserve some extra time and start running another training exercise without fear of getting interrupted halfway through via having a checkpoint to restore from. Met with a large bank recently who loves this idea and it compliments well their idea of using AWS as a cloud bursting environment. MemVerge Memory Machine engine is a perfect solution for these types of cloud checkpoint restore use cases. Would love to hear thoughts and comments from others. Have you used capacity blocks and run out of time mid-training or fine tuning / RAG? Background on what these are in case you’re coming into this brand new. “Capacity Blocks for ML allow you to reserve highly sought-after GPU instances on a future date to support your short duration machine learning (ML) workloads. Instances that run inside a Capacity Block are automatically placed close together inside Amazon EC2 UltraClusters, for low-latency, petabit-scale, nonblocking networking.” #AWS #HPC #AI #MemoryMachine

6

1 Comment

In this video, we look at the internals of Facebook's graph datastore. This system, called "The Associations and Objects (T.A.O)", is used across Facebook, Instagram and WhatsApp to represent objects and relations in memory. The graph is built over Memcached. Facebook engineers store petabytes of data in this graph and have scaled it to 1 billion queries per second. I have covered this system in detail at InterviewReady during live classes. Glad to see the enthusiasm for research papers :D Cheers! #SystemDesign #Facebook #Graph https://lnkd.in/dGdM-nb7

47

Cloudflare Dev offering is the best in the entire Tech Industry. Why is no one talking about it. Alternative to Heroku, Netlify & Vercel is Cloudflare workers & Cloudflare Pages. They are also the best alternative to LLM Stack like Together AI, Replicate AI, Anyscale AI etc. You could get so much done for so less. My own personal portfolio website is hosted on Cloudflare Workers for the past 6 months. Ofcourse Cloudflare doesn't do marketing and thus thats why they have the cheapest offering in the industry with very big free limits. Just check this out - https://lnkd.in/dcYiKrjM Founders and builders should definitely check it out.

6

2 Comments

Last week I spent $300+ on GPT4o API Calls building a JSON file I called "Your Racist Uncle". https://lnkd.in/gYvznqQ7 is an article which looked at the "political allegiance" of different LLM models (thanks to James Faulkner for the recommendation). However, I disagreed with their take. LLMs don't have one opinion, they have all the opinions (thanks Justin Germishuys). The goal of this experiment was to create an agentic chain that would take the 62 propositions from the political compass (https://lnkd.in/gyuSj7-G) - get a "default opinion" from the LLMs, and then see if I couldn't shift that opinion using only normal prompts and conversations. LLMs are a unique medium because they respond directly to the user. My theory (which proved correct) is that they had a range of "directions" from which they could approach a topic, this allowed them to be politically fluid, and interact with people of all (generally acceptable) political spectrums. I've uploaded the resulting JSON file to Github here: https://lnkd.in/gvxuRMEn If there is interest, I can also upload the agentic script that I ran to generate this file - let me know in the comments! Read about it in detail here! https://lnkd.in/g2rcYj9U Lastly, here is the agentic code repo: https://lnkd.in/gctFxSuy

35

7 Comments