Joaquin Phoenix Says Joker 2 Weight Loss 'Felt a Bit More Complicated': 'Probably Shouldn't Do This Again'

Joaquin Phoenix put himself through the ringer for Joker: Folie à Deux.

The actor discussed the training involved in his new musical thriller, costarring Lady Gaga, during a press conference at the Venice Film Festival on Wednesday, Sept. 4, ahead of the Todd Phillips-directed sequel's world premiere.

As Phoenix told reporters in the press room, “I’m not going to talk through specifics of the diet ... but this time it felt a bit more complicated just because there was so much dance rehearsal that we were doing, which I didn’t have last time."

And while he did it in a "safe" way, the Oscar winner added, “I’m now 49; I probably shouldn’t do this again. This is probably it for me."

He also said that Gaga, 38, "lost a lot of weight" for the movie, to which she replied, "We transformed into our characters over a period of time, and we continued to hone in [on] every kind of detail."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Phoenix said later of his weight loss for the first film, "I know I was responsible last time for talking about it quite a bit, because it's difficult to do, so it becomes your obsession because you’re working through and trying to get to a certain weight."

"So then you can't help but talk about it, and then it just sounds like an actor going on and on about how much weight they lost," he continued. "By the end of that run, I was so sick of myself and angry at myself for making such a thing of it, because you just do what you're f---ing supposed to do. ... So this time, I was like, 'I’m not going to do that.' "

Joker: Folie À Deux is the sequel to 2019's Joker, which also was directed by Phillips, 53, and starred Phoenix, who won a Best Actor Academy Award for his portrayal of the sinister DC villain.

The new movie costars Harry Lawtey as Harvey Dent, Catherine Keener as Maryanne Stewart and Steve Coogan as Paddy Meyers. Zazie Beetz, Leigh Gill and Sharon Washington reprise their roles from the first film.

According to an official synopsis, “Joker: Folie À Deux finds Arthur Fleck institutionalized at Arkham awaiting trial for his crimes as Joker. While struggling with his dual identity, Arthur not only stumbles upon true love, but also finds the music that's always been inside him.”

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

Said Phillips of the sequel during Wednesday's press conference, "I would say on this one I’m more nervous. There are a lot more expectations on a second film. There’s more nervousness."

He previously confirmed to Empire that Phoenix’s portrayal of the iconic Batman villain will continue to stray away from the other iterations of the character.

“Arthur clearly is not a criminal mastermind. He was never that,” Phillips explained. The troubled character “has become this symbol to people. This unwilling, unwitting symbol now paying for the crimes of the first film, but at the same time finding the only thing he ever wanted, which was love."

Added the director, "That’s always what he’s been about, even though he’s been pushed and pulled in all these directions. So we tried to just make the most pure version of that.”

Joker: Folie à Deux is in theaters Oct. 4.