Ben Stiller Shares Why He Thinks Pete Davidson Is Popular with Women: 'He's Got Such a Charisma'

Ben Stiller is explaining why he thinks Pete Davidson is so popular with the ladies.

The Zoolander star, 56, was on The Howard Stern Show on Wednesday when Stern asked about Davidson, 28, and why "these women can't stay away from him." The Saturday Night Live star has had a string of high-profile romances, including his current relationship with Kim Kardashian.

Stiller started off by calling Davidson "an incredibly sweet guy," going on to say that "he's so personable. He's got such a charisma. He's funny, he's really funny, but he has become really famous, I think, for—"

Stern, 68, then cut in, saying, "Dating!"

"Yes," Stiller said. "And being on [SNL]."

"I get the sense that he's a guy who really loves to work, who cares about his work," the Meet the Parents actor continued. "And that's how he's going to be known. He's got a lot of great work ahead of him too."

Stiller also made the observation that Davidson's comedy on SNL is particularly impressive, since he's mostly "doing him" as opposed to inventing a character.

"It's not that easy to just be that funny and interesting and likable," he remarked of Davidson. "He's really taking his life and he's turning it into material."

RELATED VIDEO: Kanye West Says He Takes 'Accountability' for Now-Deleted Instagram Posts 'Harassing' Kim Kardashian

Of the King of Staten Island star's popularity, Stiller chalked it up to his "charm."

"People were attracted to watching him on the show, and he just has a real charm about him," he said. "I think he really wants to act, and he's making movies and taking roles on that are real acting roles."

Davidson has been previously linked to Ariana Grande, Margaret Qualley and Kaia Gerber, to name a few.

However, he has been involved with Kardashian, 41, since last fall.

Last week, a source told PEOPLE that the SKIMS mogul's "friends are excited to see her genuinely happy" in her relationship with Davidson. "Everyone around her is just happy she's happy," the source added.

The pair first made headlines in October while sharing an onscreen kiss on Saturday Night Live. During the episode, which marked Kardashian's hosting debut, the pair played iconic Disney couple Jasmine and Aladdin.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Later that month, they sparked dating rumors when they were photographed holding hands on a rollercoaster at Knott's Scary Farm in Buena Park, California. At the time, a source said "they hang in the same circles" and are "just friends," but the two continued to fuel romance speculation with various outings thereafter.

Earlier this month, Davidson called Kardashian his "girlfriend" for the first time in an interview with PEOPLE (The TV Show!) host Kay Adams.