Game of Thrones’ Kristian Nairn Knows He’s ‘Not Going to Top’ Playing Hodor: ‘It’ll Always Be Different’ (Exclusive)

Kristian Nairn had no idea what he was signing up for when he landed his first acting gig.

The Northern Irish actor and DJ, 48, remembers auditioning for the role of Hodor in Game of Thrones thanks to his mother's encouragement.

"She'd read the books entirely, and that's what really turned me on to what the show was going to be. I had no clue," Nairn tells PEOPLE. "I was always interested to try out for auditions, [but] I definitely would've been more enthusiastic about the whole audition process if I'd known [what was to come]. But I don't think any of us knew. None of us knew what that was going to turn into."

Nairn, who stands at an extraordinary 6-foot-10, admits that no one on the cast of the groundbreaking HBO series was aware of just what was to come.

"The first couple of years, it felt like walking the plank. It was like trepidation. And then after that, [there] was a bit of expectation. After that, we were all kind of wishing to make sure it was as good as the previous seasons."

Hodor become an unexpected fan-favorite in the lead-up to his death in season 6, one of the show's most emotional and jaw-dropping moments. Looking back, Nairn says he's "pleased with what I did with him" — and he knows not to expect anything like Game of Thrones ever again.

"Coming from such a show as Game of Thrones, it's like — I'm not going to top this. That was my first big acting role." he says. "But I think for me, because of the character that Hodor was, I don't think I'll ever top it, but it'll always be different, because Hodor is very different. Anything I do that speaks will be a very different role than Hodor, because I'll never do a silent role again."

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

The actor, who most recently starred in Our Flag Means Death, has been reflecting heavily on the half-decade he spent as a pseudo-member of the Stark family as he "went on a deep dive" for his memoir, Beyond the Throne, which hit bookshelves on Sept. 24.

He's the first Game of Thrones alum to look back at the series in a memoir, and he admits it's not something he'd ever explicitly imagined he'd do.

"I never thought of myself as the type of person who would write a book or who would even want to, but during my journeys around the world with my manager, I would be having conversations over dinner all the time, just talking about my past. And I just remember the night he just looked up from whatever he was eating, and just said, 'Kristian, this isn't normal, you've got to write this down. It's not the regular life. People might be able to benefit from hearing this.' And that was enough for me."

Nairn was "concerned when I started to write that my memory wouldn't be good enough to remember all the things that had happened" on Thrones, but he surprised himself.

"I really did it like a domino-type effect. I would remember one thing and then that would take me off on a tangent," he shares. "And you have to believe my bedroom wall looks like a serial killer [wall]. It was like literal pins trying to connect, because I tried to connect stories on Game of Thrones to stories in my life in a sort of coherent way. So that was a big challenge."

The memoir traces his life story, from growing up in Northern Ireland during the Troubles, leaning into music as a DJ and into performance as a drag queen to landing his first-ever acting gig in Game of Thrones.

When he thinks about Hodor's journey in the unforgettable season 6 episode "The Door" — which finally revealed why the Stark family servant had only ever said one word, "Hodor" — Nairn remains in awe.

"Where else in the world would a character who says one word, has kind of been in the forefront of the background for five seasons, become one of the top scenes of a show?"

Recalling fans' "crazy" reaction to Hodor's backstory reveal and death, Nairn admits he's still not entirely sure how it all worked out in the end.

"People have asked me to explain what happened there, and I'm like, 'Well, it's not real. I don't have a scientific explanation.' I know we haven't ever time-jumped. If we have, it certainly hasn't made any of the press," he says. "I don't have an explanation... Think of it as a big quantum leap."

The episode revealed that Bran Stark (Isaac Hempstead Wright) had time-travelled into the past and seen teenage Hodor in Winterfell. The sighting caused an overlap in Hodor's past and future consciousnesses, and as he fell into a seizure, Meera (Ellie Kendrick) and Bran's repeated order to "hold the door" blended together into "Hodor," and when the seizure subsided, that was the only word he ever said.

"I read the script and was like, 'Okay, this is really cool.' But I couldn't imagine how cool it was going to be," Nairn recalls. "Even filming, it was monumental. It was a big scene to film. It took weeks. But I still didn't expect that reaction."

"It's bizarre, it really is," he says of the end of Hodor's story. "But you know what? It's such a privilege to have touched people in that way. If I never left another mark in the world — which I already have, so I'm happy about that — but even if I had left it at that, I think I would've been happy."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Game of Thrones can be streamed in full on Max, and Nairn's memoir, Beyond the Throne, is now available.