Sakana AI’s Post

View organization page for Sakana AI, graphic

23,260 followers

Introducing “An Evolved Universal Transformer Memory” 🧠 (Blog: https://sakana.ai/namm/) Neural Attention Memory Models (NAMMs) are a new kind of neural memory system for Transformers that not only boost their performance and efficiency but are also transferable to other foundation models, without any additional training! Memory is a crucial component of cognition, allowing humans to selectively store and extract important notions from our ceaseless exposure to information and noise. Our work learns an artificial memory to replicate these capabilities thanks to the power of evolution, leading to smarter, faster, and more adaptable foundation models. Full Paper: https://lnkd.in/gkDzMCG3 Code: https://lnkd.in/gmvFSma7

  • No alternative text description for this image
Marcus Anzengruber

Tech Subject Matter Expert | 🇸🇪🇺🇸-innovation and policy alignment

1w

It’s fascinating how NAMMs differs fundamentally from H2O/L2! While those use static rules to compress memory, NAMMs learns what to remember, similar to how military intelligence must adaptively filter crucial signals from noise. The zero-shot transfer capability is particularly relevant for some application areas that I’m looking into.

Like
Reply

amazing, pushing the boundaries as always

Like
Reply
Like
Reply
Yeu Wen (耀榮) Mak (麥)

Augmenting and amplifying collective human decision-making under uncertainty and ambiguity conditions

1w
Like
Reply
See more comments

To view or add a comment, sign in

Explore topics