Kelsea Ballerini Recalls 'Lovely' Outer Banks Fan Interaction She Witnessed from Chase Stokes 2 Years Before They Started Dating

It's no secret that Kelsea Ballerini and Chase Stokes' love story started with a DM, but that wasn't actually the first impression the country singer got of the actor.

During an appearance on The Tonight Show Starring Jimmy Fallon on Oct. 30, Ballerini shared a story she's "never told" before about a sweet interaction she witnessed between Stokes, 32, and a fan during an early morning plane ride in 2021.

"I was on a flight, it was like a 5 a.m.-er, ungodly hour. I was getting ready to fall asleep and there was this young girl who came up to the person sitting behind me and was just, like, freaking out, like fangirling," Ballerini, 31, recalled. "And I heard him be so lovely to her and give her so much time and attention and intention at, like, 5 a.m."

"I turned around and I just, like, looked through the hole in my seat, and it was Chase," she said. "I hadn't seen [Outer Banks], but I don't live under the rock, so I knew him. And we were obviously in very different phases of life... We never met, never said hello, but I just tucked that away as like, wow, what a good thing to witness."

"It was just a beautiful thing to see," the singer, who just released her fifth studio album Patterns, said of the sweet interaction, that showed her Stokes is a "good, good man."

"Then a couple years later, it's funny how the universe works. I just slid into the DMs," she said of how things came full circle for her and the actor.

Ballerini also took the chance to clarify her recent comments that she and her boyfriend have a "deal" that explains why she has not watched Outer Banks.

Describing herself as a big "true crime girly," she admitted, "I don't watch a ton of TV unless it's like very crime [focused]."

Still, she said, "I wanna clear this up" about the headlines regarding Stokes' hit Netflix series, because "I don't wanna sound like an unsupportive girlfriend."

"So when we got together, everyone was coming up to him and they're like, 'John B, John B,'" Ballerini recalled. "And so he was like, 'I wanna start coming to shows.' I was like, 'I'm gonna start watching Outer Banks.' And he was like, 'You know what, I kinda love that you're just getting to know me here, in this way, human to human.'"

"So I've been to set," she continued. "I know... Like, I know. I get it."

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Part 2 of Outer Banks season 4 premieres Nov. 7 on Netflix.