John O'Neill

So far summarization has been the killer AI app for me. Others like code generation have been less impressive in my experience. But it’s really really good at distilling down text to a useful summary. A few places I’ve seen that include a slack bot that summarizes a slack thread, forums that summarize long topics, incident.io incident summarization, and asking ChatGPT to summarize or pull information from a document I upload to it.

27

3 Comments

The best kind of hacking projects are the ones you care about breaking. At Viam, we ask our engineers to spend 20% of their time working on side projects. The projects that are the most impactful are the things that people care about if they break. I want engineers to look for things in their personal life that they want to fix or make smarter rather than a flashy, but mostly useless, gadget. These projects don’t have to be big or expensive to make an impact either. There is already simple software out there that can be a massive problem solver. If you look at my own hacking projects, they are all things that impact my daily life. For example if my cat food feeder breaks, my cat gets hungry (and mad), if my sprinkler system that runs on Viam breaks, my lawn dies, etc. etc. All of this is to say that by using your time to fix real world problems you inspire others to do the same, and open up a world of opportunity. What device in your life came to mind while reading this? What is holding you back from making it smarter?

71

4 Comments

This is a very exciting effort and I think it has the potential to be a real "game changer" #foundationalmodels #AI #ML #pioneeringtechnology #metaverse

11

It was hard, but we now have a primitive at Gadget that lets us do absolutely 0 downtime atomic changes to an application's structure or data under the hood, with 0 requests dropped. I call it "request catching" but it was inspired by this Braintree eng post I read long ago For stateless workloads like serverless functions, 0-downtime deploys are pretty easy. You stand up the new one next to the old one, use a load balancer to start sending requests to the new one, and slowly stop sending requests to the old one. But for stateful workloads, it's much harder. You can't just start a new Postgres and send some queries to the new one and some to the old one without lying to somebody about state. If you want to be consistent, there has to be one source of truth. Fancy DBs have quorums for this, but PG doesn't. If you want to be consistent, you're effectively forced to have some tiny-but-real window of downtime in between draining the old one and starting the new one. That said, there's nothing forcing you to return errors to clients while that is happening. If your infrastructure supports it, you can pause requests while this changeover is happening under the hood, and then resume them once it is over. Clients will notice a longer request that normal, but it will be served like normal and they'll be none the wiser! What's required to do this though is a deep deep integration between the load balancing infrastructure and the thing doing the database deploy under the hood. We've built this up using a distributed locking tool, where all requests start by taking out a shared lock. When we're ready to do the maintenance under the hood, the maintenance workflow takes out an exclusive lock, blocking all requests from proceeding. The workflow does the atomic change it needs to make knowing it has exclusive access which means nothing will change under its feet, and then releases the exclusive lock as fast as it can so those requests can proceed. As an optimization, we only check the lock on the request path if an app is marked as near a maintenance window. This is only possible because of Gadget's architecture where we bundle together the load balancer, the request processing layer, and the database access tier such that we can reliably intercept all requests and all database transactions and do this lock checking. Its these nasty, in-the-weeds details that make me so proud of Gadget -- we can do world-class engineering stuff under the hood on behalf of all our apps without them ever noticing. If you're interested in doing this on your stack some OSS tools have the primitives you need! We use nginx for our load balancer, Temporal for the reliable workflow execution and we're currently using postgres advisory locks for hte locking. I think the Braintree post has been lost to time after they got acquired, but @simonw has a great post on the subject here: https://lnkd.in/eZmiktGT

51

Come check out this talk if you want to learn how we built distributed queries for Iceberg + Python!

4

What if LinkedIn had a projects tab? Imagine our network filled with side projects and hobbies, people looking for partners and collaborators. An engineer could team up with a designer to create a new device Writers could join scientists to publish blog articles Social scientists and Geospatial analysis look at maps to visualize social trends  There’s so much untapped potential out there. We often get stuck working alone, missing out on the rewards of valuable collaborations across different skills and backgrounds. Companies are all about making money, but not everything needs to be about that. We all have an inner curiosity that just wants to be let out. No need for a "why" or a "MVP" or a business plan. Just a simple project.

20

2 Comments

Layoffs suck. For me in particular, it's feels even worse when I hear generic positivity encouragements like "things will be okay" or "something better will come along" or the worst of them "you'll get back up stronger". But watching season 2 of Mr. Robot a particular scene really resonated with me. It applies to those who are currently facing uncertainties with their employment at this time, but cannot really swallow the illusory positivity pills everyone is sharing. Nothing wrong with being positive. I think it's helpful to others who orient their worldview that way. But for me in particular. I need a different kind of pill: that is truly identifying the suck with no filter. Here is what Mr Ray (A character in Mr Robot says about life in general). "My wife, when she drove that car that day, she did everything right. She always wore her seatbelts, hands attended to them, she was the most perfect driver I've every met. It was annoying. She stayed in the lines, never went above or below the speed limit, stopped at traffic signs. She followed every rule. And then one day none of that mattered." "Control is about as real as a one legged unicorn taking a leak at the end of a double rainbow." "You know that bullshit people say about how when you fall, you gotta get up. I reject that shit man." "You know why?" "The whole thing is a fall. You can't help but be in a perpetual state of grasping in the dark. It's not about getting up. It's about stumbling - stumbling in the right direction. It's the only true way to move forward."

17

5 Comments

On April 18th, just last week, Meta unveiled Llama 3, lauded as "the most capable openly available LLM to date". This remarkable achievement by the Meta team comes hot on the heels of Llama 2's release last July. Notably, the new models surpass their predecessors and competing offerings from other providers in terms of performance. Meta's commitment to open sourcing these powerful models while prioritizing model safety and responsible usage is commendable. (https://lnkd.in/ghv3RJRz) Yesterday, Microsoft introduced Phi-3 Mini, a member of the Phi-3 family of models, promoted as "the most capable and cost-effective small language models available". According to Microsoft, the key to achieve high performance in such a small package lies in the quality of the training data. The model has been open sourced and is now available on Ollama and Hugging Face. (https://lnkd.in/gKgSmrBn)   These compact yet powerful models serve as invaluable tools for creators eager to bring their visions to life or simply experiment with cutting-edge technology, particularly for those with limited access to resources.   Shortly after the release of Llama 3, reports began surfacing of enthusiasts running the 8B model on a Raspberry Pi 5 equipped with just 8GB of RAM—a modest $80 single-board computer. (https://lnkd.in/gbeQPhQ2) While the performance understandably reflects the limitations of such a platform, the fact that the model runs and generates an output is nothing short of magical. Similarly, the Phi-3 Mini model can run on ubiquitous devices such as smartphones.   Reflecting on my own journey, I recall the pivotal role that early access to a low-cost computer played in shaping my passion for coding. The gift of an 8-bit Commodore 64 home computer ignited my curiosity at a young age. Later, with the guidance of an outstanding Computer Science teaching staff at my high school IISS Marconi-Hack Bari, I embarked on ambitious projects like a networked multiplayer version of Battleship written in Turbo Pascal and 8086 Assembly, complete with custom sprites and lo-res graphics. These formative experiences paved the way for my career, eventually leading me to various engineering and leadership roles.   It all began with a simple, low-cost, modestly capable computer used for experimentation—a gateway to discovering my life's passion.   I encourage educators and parents alike to consider setting up similar low-cost experimentation environments. By introducing younger generations to open-source AI technologies, we can nurture their creativity and help them uncover their true passions. It's not just a weekend project—it's an investment in their future and the future of innovation and discovery. #education #youth #AI #LLM #future #innovation

24

🌶🌶🌶 Hot take… Old problem: if we put more devs on the project, we can deliver it sooner. Same thinking, new manifestation: we can hire more overseas devs for less total money to produce the same result as our US staff. Flaw in that thinking: product development is not an assembly line. Devs are not “interchangeable”. Software is as much an art as it is science. It’s a team sport. The people on your team matters. #tech #staffing #management #hiring #jobs #layoffs #software #startup #teamleadership

25

6 Comments

Geoffrey Hinton's contributions is significant enough that I won't quibble with the category for his prize. (Nobel committee could create a special category just for him for all I care.) Widely (and rightfully) recognized as a leading expert in deep learning, his research touches everything from * Foundational concepts such as backprop https://lnkd.in/gMunbBzA * Foundational model architectures such as AlexNet https://lnkd.in/gq2JPJ6h * Optimization algorithms such as Momentum and RMSProp * Visualization techniques such as t-SNE https://lnkd.in/gjuw3vhA I don't think it's possible to overstate how profound the impact of machine learning and artificial intelligence will have on society, promising to unlock discoveries, improving productivity. Even ideas about what mastery of language or reasoning means, will need to be re-thought. Professor Hinton deserves all the credit for all of this.

31

We've been making some cool examples of things that can be built with substrate.run lately. Here's a simple chatbot that can help you cook: https://lnkd.in/eEZm-nRJ If you're working on building a chatbot, take a look and see what it might look like to use Substrate 🧑‍🍳

19

2 Comments

It seems that with C/C++ -> Rust, specifically, you could do this by comparing the output compiled code as one of your verifications of correctness for ML learning This is something that e.g. copilot doesn't do - but it'd be straightforward to do so when doing training The expectation would be that the compiled rust code should be equivalent to the compiled C code, i.e. the assembly should be semantically no different As the ML learns, it may end up doing some funky rust code simply to get that to match - but that's probably a good thing, as this is the part that would require some manual intervention & you'd at least be guaranteed equivalent behavior

1

Meta uses monorepo to manage its code. I was skeptical about this approach when I joined the company a few years ago. How could it work with such a massive codebase? But I was wrong. I found working in Meta's monorepo to be a pleasure.  If you want to learn what working in Meta's monorepo is like (and why we are not using git), check out my latest post. https://lnkd.in/guaDRris #softwareengineering #productivity

4

this meme led to an interesting convo: friend: Borg are perpetually in a build phase. never take a step back to fix the cables. me: That's a lot of technical debt. friend: can you imagine how they just "hack together" a species's technology into their own stack? me: To tech folks, that kind of software assimilation is more frightening than actual assimilation. Imagine if they kept you human just long enough to complete the integration into their Frankenstack. You'd be *begging* to be assimilated.

3

2 Comments

I started writing regularly about AI. If you're an innovators or product developer eager to apply the latest in AI, whether you're in a startup or a big company, consider checking https://lnkd.in/gGdW8PYe I won't teach you ML, but I share real-world insights from launching - and sometimes failing at - numerous ML features on a large scale.

54

2 Comments

I frequently get asked by recruiters to help with scaling langchain to production. Our community voices the same challenge, and regular coffee talk boils down to the same; scaling AI applications feels way harder than anything we're used to as engineers, and existing Python solutions are a pain to work with. To me, it comes down to several mayor challenges: * Engineers haven't figured out an architecture that scales * AI applications are leaky abstractions and stochastic; great error handling and recovery is extremely important * Performance is key; lots of data, heavy io bounds. That's exactly why we build Swiftide (https://lnkd.in/dYzxTmZC). Build on parallel, streaming pipelines in Rust, it's blazingly fast, reliable, and, dare to say, easy to build with and easy to extend. And to boot, learning Rust is great for your career anyway, your favourite language is probably being rewritten in it as we speak. But what about agents, I hear you ask? Well, that's exactly what we're working on right now. Soon! If you are a company in need of scaling your AI solution, or just want to experiment with something different, feel free to shoot me a message and I'd love to think a long.

9

3 Comments

"Team" structure at Pacific Northwest National Laboratory is weird, in a good way. I'm used to "join a company, work on a product long-term". It's not like that at all here. It's less top-down and more "get a bunch of talent together and let them do their thing". Like a vetted, well-networked group of entrepreneurial spirits that organically make cool things. My manager isn't necessarily tied to my work. I don't have a desk on campus that is my home, and I don't know if I really want one. If I want to work on something else, I just... talk to people and do it (but I'd get myself in trouble if it was a bad time for my current projects). I get a code that lets me bill my time to that project [1]. My immediate contacts regularly mention, "You should talk to this person, check out this thing, I'll set you up with this person", etc. Networking is the innovative lifestream of the organization, and unusually important for individual career growth. Long-term software maintenance requires explicit funding proposals. I think this means more dead software, but also a healthy focus on what is important. It's very different, and IMO special. I've never felt so proud/excited to be associated with a professional organization. [1] I bill my time on a half-hour basis, every day. For Monday through Thursday this week, I have 11 durations assigned to 4 charge codes. Billing consumes a decent chunk of meeting time and my meat computer's clock cycles.

8