Fiona Palomo Says It Was 'Terrifying' Joining Outer Banks as Drew Starkey's Lover: 'About to Upset a Lot of Women' (Exclusive)

Fiona Palomo was feeling some nerves joining the tight-knit cast of Outer Banks — and it didn't help that she was coming on as a love interest to one of the show's most thirsted-after characters.

Palomo, 26, was first introduced as Sofia in season 3 of the Netflix series (before she was upped to series regular for season 4), and she tells PEOPLE it was "terrifying" to think about how fans would react to her character's blossoming romance with Rafe Cameron (Drew Starkey).

"It's like, 'I'm about upset a lot of women out there,'" she jokes. "But it's cool. It's incredible to be on a project...where people are so invested in the characters and what's happening."

"It's cool to be like, 'Oh God, I don't know how this character's going to be received, because it's like this character and it's this actor and it's all these little things,'" she adds. "But I think that just speaks to how big of a thing [Outer Banks] is, and I'm just so happy that people have been kind to me [thus] far."

She even brought it up to Starkey, 30, himself, who isn't as unaware of the internet thirst for him as it might seem. She recalls, "We would just be sitting on set, I'd be like, 'I know when this comes out, it's going to be like, ‘Stay away from him!’ And we would laugh about it. He's like, 'Yeah, probably.'"

Though it was "a little daunting to see how [Sofia] was going to be received" ahead of the Oct. 10 premiere of season 4 part 1, Palomo says, "I feel like the love — or the kindness — I put into Sofia, and just making sure that character worked well and everything, is getting blown back and it just feels really well-received and very loved and it's really nice. I'm really, really grateful."

It shows her how "invested" the fans are, she says before adding, "I've just seen really, really kind things, and they really like her, and they're even defending her and being like, 'Sofia, you deserve better.' I'm like, 'What's going on?'"

Palomo has nothing but flattering things to say about working with Starkey — who's played the angry, violent older brother to Madelyn Cline's Sarah Cameron since season 1 — as she describes him as "so kind, professional and welcoming."

He couldn't be more different from his character, she affirms. "I was flown out to Charleston to do a little chem read with him, I guess, and that's when I met him. And I remember — you have this daunting image of Rafe and obviously you're aware it's an actor — but he was just really nice."

That just speaks to "how amazing of an actor" he is, she says.

"He really cares for his craft so much, and I do too, so it was really beautiful to be able to have someone care that much to sit down and make sense out of these characters and have it feel like they've spent all this time together that you never see onscreen."

Building Rafe and Sofia's backstory together was an important part of their prep for season 4, Palomo says, since the new installment sees their relationship progress to the point that Rafe is making promises about their future together.

She knows, too, that just looking at Sofia, a "Pogue" with a working-class family, "You just want to sit down and be like, 'Sofia, we need to talk about this.'"

But she sees some nuance to the bad boy, good girl cliché of it all, particularly considering Sofia's family's cultural background. She shares, "[She’s] very warm and it's very easy [for her] to lean into people and trust and want to kind of take care of them. So I think she really thinks — [being with Rafe is] not intimidating to her. I think she's just very ready to embrace all his parts and everything that he is."

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

"It's really incredible that through some kindness and love, a person could potentially actually be more in touch with themselves," she says of how Sofia might be changing Rafe, who has been "through such crazy events," including the death of his father, Ward Cameron (Charles Esten) in season 3.

As for the rumors that Starkey is maybe the most aloof of the cast?

"I'm aware of this myself, but I also watch my castmates talk about how just unaware he is of his cell phone in general. Like, period," she says. "But that's cool. He's like this cool nineties kid. He's just like, 'I don't know where I left my phone. It could be anywhere on the set.' And it's cool. He's just living."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Part 1 of Outer Banks season 4 is now streaming on Netflix, and Part 2 premieres Nov. 7.