Elle King on Avoiding the Nepo Baby Label with Dad Rob Schneider: 'I Do Work Really Hard'

• Elle King says the "nepo baby" label doesn't really affect her
• She says she worked "really hard" for career
• The musician is the daughter of actor Rob Schneider and model London King

Elle King is getting candid about the “nepo baby” label.

The “Ex's & Oh's” singer, 35, discussed the rise of her music career and family background as the daughter of actor Rob Schneider, 60, and former model London King on the latest episode of the Off the Vine podcast with Kaitlyn Bristowe, noting that she tried to create her own success apart from her family.

“I do work really hard,” Elle shared on the podcast, adding, “I don't think I ever really got scooted into the ‘nepo baby’ thing.” 

When Bristowe, 39, asked if her father’s career in Hollywood had an influence over the career or “direction” she chose in life, the musician said that it didn’t. 

“It was more of like a deterrent,” she admitted, adding that she and her father “weren’t super close” when she first “got signed” to her music label.

“We really weren't even speaking…,” she continued. “I was a lot closer with my mom's family in Ohio, and so I just have always gone by my mom's last name.” 

Elle went on to say that it's “easy to judge” her from an outside perspective, since if anyone “Googled” her, his name usually was the “first thing” that popped up — but it couldn’t have been further from the truth.  

“I was born in California, and nobody knows that I really grew up in Ohio, and [I] didn't even move to New York until I was a teenager,” she said. “So, I feel like it's just so unfair.”

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

“And, yeah, of course, there can be any situation in the world and be like, ‘OK. It's so much easier to just look surface level and not think that there could be any layers to anything ever,’” Elle added.

While her father didn’t help kick start her career, she does credit him for giving her his “pipes,” saying on the podcast, “My dad can sing.”

“He does the best Elvis I've ever heard,” Elle said. “It's pretty wild.” 

Meanwhile, the “Drunk” performer said she got her humor from her mom, who is “the funniest person that I've ever met.” Elle said London was the one who also helped her find her passion for music as she grew up.

“My mom was the one, you know, carrying my guitar and sneaking me into bars, and letting me stay out with my friends and taking the guitar home for me,” she shared. “My mom was always just, like, telling me to take lessons, and she would drive me everywhere. And she took care of me and always made sure that I had some form of a creative outlet."

Elle, who has earned a CMA award and an ACM award, previously discussed wanting to make a name for herself apart from her parents in an interview with PEOPLE in 2023. 

"Growing up as somebody who was chubby, awkward, funny, has OCD and is weird and quirky and awkward but super sensitive, I had to really fight to be OK as myself. So I didn't want to be known as someone's kid," she continues. "I wanted to be my own person. I never wanted to borrow money from anybody, and I've worked hard for my family and for me. I want to have a life built for myself, and I want to know that I did it."

Elle added that “it's an incredible amount of pressure” to be in the “shadow” of someone.

“I mean, God forbid you're good at something that happens and maybe was passed down in your family….,” she said. “My showmanship and my stage presence? Sure, I grew up watching my dad. My dad can totally command a theater or a small comedy club. And I grew up around really funny, talented, creative people. Of course that's going to leave an imprint on me."