Mary Steenburgen Recalls Late Elf Costar James Caan Being 'Unexpectedly Sweet and Charming' (Exclusive)

Mary Steenburgen is fondly looking back on her time making Elf with the late James Caan.

In an exclusive chat with PEOPLE surrounding the Jon Favreau-directed holiday classic's 20th anniversary, the actress says she wasn't sure what to expect from Caan, who often appeared in tough-guy roles.

Steenburgen, 70, says she was pleasantly surprised on set by the warm personality of Caan, who died on July 6, 2022, at age 82.

"I didn't know what he would be like, and I found him unexpectedly sweet and charming," she says. "I had fun with him."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human-interest stories.

Steenburgen says her "first day of work" was filming the scene where Buddy (Will Ferrell) chugs an entire 2-liter of soda, douses his spaghetti in maple syrup and encourages his family to do the same.

"I had not seen even a design or anything about what Will was going to look like," Steenburgen recalls of Ferrell's now-iconic costume. "And then he comes in. ... And I'm tall, but he towers over me, and he comes in in this elf outfit, and then I'm having to eat the spaghetti with maple syrup."

"I just spent the whole day trying to get my lines out without laughing," the Oscar winner says.

According to Steenburgen, she, Caan and even Ferrell, 56, would sometimes "ruin a take a little bit from laughing," because "the visual" of Ferrell in costume as Buddy "was constantly funny."

"You didn't really get used to it," she tells PEOPLE. "I think we're all used to it more now than any of us were then, because we'd never seen it on posters or on sides of buses or any of that. And so we did tend to just enjoy it so much."

Steenburgen went on to star alongside Ferrell again in 2008's Step Brothers, where she again played the mother of his character.

Asked whether she'd be up for a sequel to Elf, she tells PEOPLE, "If somebody had an idea for that that was truly compelling, then I am sure that would be a fascinating and exciting thing. But I think it's also kind of to have respect for what was created. It was pretty perfect as it was. And sometimes, things are best left alone."

"So I'm not sure what I would want. I would only want that to occur if I read something and went, 'Oh my God. This has to be made,' and no one has really done that yet," Steenburgen explains.

But when it comes to the potential of collaborating with Ferrell again, she says, "Having done two such iconic films with him, and very different films with him, I know how brilliant he is, and I would be honored to work with him anytime."

For more on Elf's 20th anniversary, pick up the latest issue of PEOPLE, on newsstands Friday.