Elle King on Relationship with Dad Rob Schneider: 'I Didn't Want to Be Known as Someone's Kid'

Elle King is opening up about growing up with a famous father.

The Grammy-nominated musician is the daughter of comedian Rob Schneider and former model London King. And while King has a strong bond with her dad today, their relationship was strained for years.

The "Drunk (And I Don't Wanna Go Home)" singer — who just released her country album, Come Get Your Wife — was born in L.A. but was raised by her mom and stepdad Justin in Ohio.

"I was the chubby girl with a dad that didn't visit very often, and everybody made fun of me and were mean to me about it," King tells PEOPLE of her childhood.

She appeared in her dad's 1999 comedy Deuce Bigalow: Male Gigolo. But when King decided to pursue her own career in entertainment, she began performing under her mom's last name — and was estranged from her dad at the time.

When King released her 2012 self-titled EP, "he wasn't even in my life. He got married and had a kid, and we weren't even talking," King says of Schneider, who shares daughters Miranda, 10, and Madeline, 6, with his third wife Patricia.

"Growing up as somebody who was chubby, awkward, funny, has OCD and is weird and quirky and awkward but super sensitive, I had to really fight to be OK as myself. So I didn't want to be known as someone's kid," King adds. "I wanted to be my own person. I never wanted to borrow money from anybody, and I've worked hard for my family and for me. I want to have a life built for myself, and I want to know that I did it."

She rose to fame with her 2014 single "Ex's & Oh's," has notched four Grammy nominations over the years and has found success in Nashville, so far earning a CMA award and an ACM award for her collabs with Dierks Bentley and Miranda Lambert, respectively.

King — who shares son Lucky, 16 months, with fiancé Daniel Tooker — has also reconnected with her dad after finding success on her own terms.

"My father and I have a beautiful and really wonderful, great loving relationship with awesome boundaries," she says. "I love my dad so much."

As for the recent nepo baby conversation happening, King says, "Is it easier for some people? Sure. But they didn't ask to be born."

She adds: "What I will say to that is, it's an incredible amount of pressure to be in a shadow. I mean, God forbid you're good at something that happens and maybe was passed down in your family. My dad is a wonderful singer. It doesn't take away from my abilities that my dad sings. My showmanship and my stage presence? Sure, I grew up watching my dad. My dad can totally command a theater or a small comedy club. And I grew up around really funny, talented, creative people. Of course that's going to leave an imprint on me."

For more on Elle King, pick up the new issue of PEOPLE, on newsstands everywhere now.