I often get asked about how the TPU chip came to be at Google?
There were actually three attempts to build an AI accelerator chip at Google that went on about the same time.
One of them was actually led by a Turing Award winner (I think everyone expected that to be the one that Google would use), there was the TPU (the project my team worked on), and one other team.
The VP that funded the project had funded it out of his “slush fund,” which was the leftover money from canceled projects that he had discretion over. One of the most important things about how the project came to be was this…
BECAUSE IT WAS JUST LEFT OVER MONEY, THERE WAS NO STARTING POINT OF “OH, THIS HAS TO WORK.”
There was this freedom where we could say, “Okay, we'll just build whatever we think makes sense” and we went at it completely de novo with a clean slate.
In my case, I didn't even know chip design.
That turned out to be an edge because we were not confined by the curse of criticality, where the someone says, “Thou shalt be successful, and you must follow this playbook to limit risk.”
My favorite part is the ending -
After the TPU architecture was selected (and ultimately went on to run half the compute at Google), I remember the Turing Award winner giving a talk where he said, “I don't know who came up with this systolic array in the TPU, but they must be really old because no one uses systolic arrays.”
I remember thinking, “Oh, that’s called a systolic array? It just seemed to make sense.”
Best-selling Author. Public Speaker. Founder of WOW
1wwow! can it beat you and me as the most succesful coaches in hockey?