Welcome to another deep dive. Today we're going to be looking at um, ohh, yeah, it's really cool paper. OK, attention is all you need. I love this paper and it's by some folks from Google. Uh-huh. And it introduced this concept called the transformer model, which is kind of taken over AI. Absolutely. And I think this is one of those papers that was kind of like a paradigm shift in how we think about AI. Definitely. So I'm excited to jump in and kind of unpack it with you. Sounds good. So the paper starts by talking about recurrent. Neural networks, yes, or RN's as they're known, right? And they talk about some of the limitations of these RN's. And particularly with dealing with these longer sequences of data. Yeah. They, they just, they struggle with that. Yeah. It's like they have a hard time remembering things. Yeah. If you were to read a book, right, and try to understand the last chapter. Yeah, only by remembering one word at a time. Exactly. That would be very difficult. It would be. That's basically what RN's are doing. Yeah. Yeah. And So what the transformer does is it does away with that whole sequential processing thing. OK. It says we're not going to do that anymore. We're going to. Just look at the whole thing and we're going to use something called attention mechanisms instead. That sounds much better, much better. So what did these attention mechanisms? So attention mechanisms allow the model to focus on certain parts of the input that are most important, like actually pay attention. Exactly that's.
