Host, TED AI Show | Ex-Google PM (3D Maps, ARCore, 360/VR) | Scout, A16z | VFX creator with 1.4M+ subs & 445M+ views | ੴ
20+ labs just worked together to open source Genesis - a physics engine with a VLM agent that turns text prompts into interactive 4D worlds. Think instant physics-accurate environments, camera paths, and character animations - all from natural language 🤯 → Like seriously, is there something this thing *can't* do? You've got a physics engine, robotics simulation platform, generative data engine, and photorealistic rendering (w/ ray tracing support). → Under the hood, it uses a VLM-based agent that wields simulation APIs as tools, generating physics-accurate environments, camera trajectories, robotic behaviors, and character animations. Tons of SOTA physics solvers + a range of materials. → They've rebuilt everything in pure python, somehow making the physics engine is 10-80x faster than Nvidia's Issac Sim used for Project Gr00t. Think physics sims running 430,000x faster than realtime while maintaining accuracy good enough for real world robot deployment. WTF! → The team (collab across 20+ research labs) has open-sourced the core physics engine and simulation platform (apache 2.0). Alas, the generative framework release TBD. Can't wait to see what the community builds with this. Link in comments below. Of course, all I can think is... 3D + AI for video creation :-) #ai #3d #cgi
oh why not... let's restart everything over again, but with AI now... "AI Physics Engine", "AI Game Engine"..... "AI Coffee Brewing" - being the latest "game changer ®" 🤣🤣🤣 Feeling the need for this = 0 🙄
This new simulator is a nice Christmas present indeed! Surprising it was kept quiet for so long. And a nice touch, Dutch beer!
Not sure what's more impressive, the tech itself or the fact that 20+ labs could keep a secret.
Pure python ground up implementation, and faster than SOTA simulation engines... I am reading it 300th time and still can't believe! Truly an amazing engineering masterpiece 👏 👏
Nice presentation. Love the llm imterface. Very nice overall accessibility and package. Whether this is groundbreaking or not depends on the asymptotic convergence of the solvers. Are they 1/n or 1/n²? The scales of macroscopic machines in reality are 1:1e9 (including precision). In neural robotocs 1:1e3 should be enough, but for current industrial robotics 1:1e4 or slightly more is needed. So, if you solve to 1:1e2 here in the demos, and have a linear solver, you will run at least 100 slower for precision robotics. For stick slip transitions, 1000 slower, I would say. So, unfortunately, communicating speed without communicating precision is not science. We did this 15 years ago, incompressiböe fluids on gpu. Time flies. https://youtu.be/kzFam0UKFoA?si=HUiNq4c3Vz28o5bu
"Think physics sims running 430,000x faster than realtime while maintaining accuracy good enough for real world robot deployment. " already makes me very suspicious im not looking at a well package commercial marketting buzz than something it indeed pretend to be.. But respect for being open source, we will find it out soon!
Movement in the World is an extension of thought in the world
The gs.generate() function is not available yet. "Currently, we are open-sourcing the underlying physics engine and the simulation platform. The generative framework will be released in the near future." https://genesis-world.readthedocs.io/en/latest/
Host, TED AI Show | Ex-Google PM (3D Maps, ARCore, 360/VR) | Scout, A16z | VFX creator with 1.4M+ subs & 445M+ views | ੴ
2dProject page: https://genesis-embodied-ai.github.io And if you like stuff like this: https://spatialintelligence.ai