How Steve Guttenberg Escaped the Cocaine and Excess Culture of '80s Hollywood (Exclusive)

Steve Guttenberg is a lot of things: a famous actor, an accomplished author and a dutiful son who spent five years caring for his ailing father. One thing he never was? One of those '80s stars who had a raging drug problem.

"I tried it, but it wasn't a part of my life," the star tells PEOPLE of the cocaine culture that was sweeping both Los Angeles and New York in the 1980s.

"Some people I know would do it all the time, and I'd be like, 'How can you do this every day? Don't you feel lousy in the morning?' They're like, 'Yeah, you got to get it back together in the morning.' That wasn't for me," Guttenberg says.

He adds, "I get what it did. You'd try it and be like, 'Wow, this is really great. I'm like a God, I feel amazing. I'm going to go get a mathematics book and learn the whole thing in a night, and then I'm going to become a math teacher and I'm going to build a spaceship!' "

But Guttenberg, who now doesn't even drink — not because of a problem, but because he no longer likes it — says he was never a wild guy.

"I did have one of those moments after I was famous where I was in a basement of a friend of mine, and all my old friends from high school were around, and they were trying to be cool and they brought out some of the [cocaine]," he recalls.

"They were like, 'Hey Hollywood, you want to do a little of this?' I'm like, 'Well, yeah. Why not?' They put it out and then I accidentally sneezed, and it just blew whatever was on the table away. And those guys were like, 'Well, you're not that cool now,' " continues Guttenberg.

The star says that he actually left Hollywood the first time he tried to be an actor because he didn't like the culture.

"L.A. was fine. It was very laid back at the time. But acting is so competitive, and I didn't make many friends," he says.

Guttenberg eventually quit the biz and went back to school at SUNY Albany. But five months into his first semester, he got an audition for Boys from Brazil with Gregory Peck and Laurence Olivier.

"My dad was like, 'Try it and if you get it and don't like it, you can quit,' " he recalls of his late father Stanley, who he says was his biggest champion. (Guttenberg just released his memoir, Time to Than, about his childhood and caring for his ailing father during the last five years of his life.)

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Guttenberg landed the role in The Boys from Brazil and then says offers for films like Police Academy, Cocoon and Three Men and a Baby kept coming in.

And even though he's remained busy as a working actor, Guttenberg says he wasn't surprised when leading roles got more scarce.

"That's Hollywood!" he says with a laugh. "You have to be careful not to buy into the success. You have to remember the stars are in the heavens. You're just a human being who got lucky."

Time to Thank is available now wherever books are sold.

For more on Steve Guttenberg, pick up the latest issue of PEOPLE, on newsstands now, or subscribe here.