James Gandolfini Was 'Hungover' While Filming Iconic 'Sopranos' Beatdown Scene, Robert Patrick Says

It's safe to say that the late James Gandolfini wasn't messing around while filming his iconic Sopranos beatdown scene.

During an appearance on the Talking Sopranos podcast on Sunday, Robert Patrick — who played Davey Scatino — recalled his experience working alongside Gandolfini on the acclaimed HBO drama. Patrick, 62, said he "had never met Mr. Gandolfini" prior to getting "bitched slapped around" by him on-camera.

"I'm out front on a bench smoking a cigarette with Jimmy after we'd just done this read-through. And I think we're trying to get to know each other a little bit because, you know, we're supposed to be high school buddies," he said. "And if I remember it correctly, the conversation was kind of winding up. I said something to him like, 'Hey listen. You know, when we do the scene where you come for the money, you better bring your f------ A-game.'"

"He gave me a look and he had a cigarette and he went, 'Oh, I'll bring my f------ A-game,'" Patrick added.

Patrick next saw Gandolfini on the day they were scheduled to film the beat-up scene.

"He said, 'How's your balls?' ... I looked at him and I said, 'They're good.' And he said, 'I'm hungover, let's do this f------ scene,'" Patrick said. "And man, it was one take. We kind of went through what he was going to do [beforehand]. He was going to grab me and he was going to throw me around, and I'm going to start crying like a little girl. And I got to tell you, he scared the living s--- out of me man. I mean, I was so intimidated."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Patrick, nonetheless, still considered it to be a "fantastic experience." He added, "I think The Sopranos is definitely one of the highlights of my career."

Gandolfini played Italian-American crime boss Tony Soprano on The Sopranos. The HBO series ran from 1999 to 2007 for six seasons.

Gandolfini died in 2013 at age 51 while vacationing in Rome. An autopsy revealed he died of a heart attack.

This October, Gandolfini's son, Michael Gandolfini, will play a young Tony Soprano in a prequel film, titled The Many Saints of Newark.

Gandolfini's widow Deborah Lin recently reacted to seeing her stepson tackling his father's former role.

"It was almost surreal to see Michael in the trailer," Lin told The New York Post. "I know it took a lot of courage for him to take this role. It was very emotional. But the whole family is just so proud of him. His father would be very proud."