'Southern Charm' 's Shep Rose and Taylor Ann Green Split: 'He Refused to Commit,' Source Says

Shep Rose and girlfriend Taylor Ann Green have gone their separate ways.

The Southern Charm stars have broken up, sources exclusively tell PEOPLE, after two years of dating.

According to insiders, problems in their romance came down to Rose's disinterest in being faithful.

"Shep has a real issue with monogamy and refused to commit to Taylor, telling her that he doesn't want to be with just one person and change his lifestyle," one source says.

"She's incredibly hurt, but she knows that she deserves a guy who will fight for her, not one who is so selfish," adds another. "He'll regret this."

Reps for the stars did not have any comment when reached by PEOPLE, but Southern Charm fans have seen seeds of that conflict on the show through the years.

Last season, at the reunion, Shep, 42, admitted he had been unfaithful to Green, telling host Andy Cohen, "there was some text messages" between him and another woman. "It was an old flame," he said, adding, "and I kissed a girl in a stairwell, but that's all it was."

Earlier this season, during a tense sit-down between Rose and Green, 27, the two discussed their future together, Green questioning Rose's happy reaction to a pregnancy scare they had.

"You're gonna high five to my pain and misery?" she asked Rose. "You didn't even ask how I felt. You just thought, like, 'Oh, no, there was a scare. What am I gonna do?' Like, you haven't ever really said, 'If you things were to ever happen, you can trust in me.' "

Green went on to tell Rose that she doesn't consider him to be "a safe bet," pointing to the fact that he'd told her that he "doesn't want to get married and settle down."

"If I was pregnant, I would want the rightful steps to be taken, which would be marriage," Green shared in confessional. "But I don't know how much he would step up to the bat, which is really s---ty."

Rose for his part did not offer Green much comfort. "The minute you tell me I need to do something a certain way, I will do the exact opposite ... but I want you," he said, adding that if Green did want a more formal commitment, the two could "cross that bridge when we come to it."

Well, they appear to have come to it — but only recently.

Just last month, Rose told PEOPLE that despite the season's trailer pointing toward rocky waters for the pair, there were "good things are on the horizon" for them.

"It is difficult to curate a relationship on camera," Rose said, blaming their problems on the struggles of dating in the public eye. "The way I describe it is all my years on the show, I was juggling one ball; that was me. Now I'm juggling me, I'm juggling my girlfriend and our relationship. Sometimes when you try too hard, it comes off even worse. I think I suffered a little bit from that, trying too hard."

Never miss a story — sign up for PEOPLE's free weekly newsletter to get the biggest news of the week delivered to your inbox every Friday.

He even praised Green for helping him grow out of his eternal bachelor ways.

"Taylor has been a great help as far as pulling me by the collar when I need to be pulled by the collar," he said, acknowledging that he's "a boisterous, passionate, idiotic human being."

"Honestly, the show can be a real big microscope. As much as it can be embarrassing...it also forces you to really look at yourself," Rose said. "I think I've come a long way. I'm basically an old man now, I guess. I'm coming to grips with that."

Rose previously starred in his own reality dating show spinoff for Bravo called RelationShep.

While promoting the series in 2017, he told PEOPLE about his desire to settle down, explaining that he feared the life of "lifelong bachelorhood," pointing to then costars Thomas Ravenel, 59, and Whitney Sudler-Smith, 54.

"I'm not saying they're not happy, and I think the world of them both, but I don't want to be in their shoes when I'm their age," said Rose.

"I like to meet girls and have a good time, but my ladies man reputation sort of got exasperated," he added. "I never aspired to be like Warren Beatty or anything. I don't have a silver tongue, I'm not trying to make promises I can't meet. I'm always looking for someone who's unbelievable to blow my hair back, and I will happily forsake others if and when I find her."