Tamra Judge Says Ex Simon Barney's Cancer Battle Played Into Her 'RHOC' Exit: 'Nothing Else Matters'

Tamra Judge‘s priorities immediately changed after learning that her ex-husband Simon Barney was battling stage 3 throat cancer, so much so that she says she was ready to exit Real Housewives of Orange County.

In an exclusive interview with PEOPLE, the reality star — who shares son Spencer, 19, and daughters Sidney, 21, and Sophia, 14, with Barney — says that Barney’s condition, mixed with her husband Eddie Judge’s recent heart problems, really helped her see her priorities clearly.

“It made me reevaluate my life,” Tamra, 52, says.

“Simon had told me about his health back in November and I was completely destroyed,” she reveals. “I was a mess! This is the father of my children. Any ill will between the two us just vanished. My first thought was, ‘I don’t want to take my kids to their dad’s funeral.’ And my second was, ‘I don’t want to go back to the show.’ ”

“The reality of it is, after Eddie’s heart condition and now Simon with cancer, it’s just very hard for me to want to go on a show where people are screaming and yelling and arguing about things that don’t even matter,” Tamra adds. “They’ve taught me that life is precious, and you don’t get that time back.”

Tamra was just days away from filming the RHOC season 14 at the time. She said she initially planned to announce there she was leaving the reality show, but was talked out of it.

“I thought, I’ll be the first Housewife to go and be like, ‘This is my last year, I won’t be back.’ Because I was really done,” Tamra recalls. “But I ran it by a few people close to me who said, ‘Don’t do that right now because you’re emotional and you don’t know how you’re going to be feeling in a few months.’ So I agreed, I might have just been reacting too quickly.”

Still, Tamra says she was especially emotional during filming. “You can tell my behavior at the reunion was very subdued,” Tamra recalls. “I didn’t want to fight with anybody. My mind was not there at all. I hadn’t told production about Simon, so they didn’t fully understand what was wrong with me. But I told [costar Shannon Beador], ‘I feel like this is my last season.’ In my gut, I knew I wouldn’t come back.”

Days later, Tamra was in New York City for an appearance at BravoCon, the network’s first-ever fan convention. There, when Barney’s name was brought up during the RHOC cast panel, she almost immediately cried — telling the crowd, “He’s an amazing dad.”

“I was a wreck,” Tamra says, reflecting on that moment. “Everyone knew something was going on with me, but I wasn’t telling anyone because Simon had asked me not to. I just wanted to get away from the show and get away from the press and get home to my family.”

After BravoCon, Tamra returned home to Orange County to focus on Barney and her family.

She and Barney had been getting along — a first, after their 13-year marriage ended in 2011 — and were even able to repair the fractures in their family. “I’ve grown close with his fiancée [Catushia Ienni]. It makes me happy Simon has someone like her who can take care of him from day to day. And we’re all our talking with our kids again, after the estrangement. We’ve all rallied together to be there for Simon and make sure he has the support he needs,” Tamra says.

And then, in early January, RHOC producers came calling about season 15, in a routine check-in to see what would be happening in Tamra’s life if she returned to the show.

“They wanted to know what I’d be up to, and knowing Simon didn’t want me to talk about it, I told them I had a lot going on in my life that I wouldn’t be able to film,” Tamra remembers. “I said to them that it was going to be hard for me to film all the time, and that I didn’t know how much of a commitment I could put into filming. Because in my mind, I knew there was going to be a phone call during filming where someone needed me — where Simon was in the hospital, or one of my kids were upset — and I’d need to handle all that stuff, away from the cameras.”

Why didn’t she just tell them she didn’t want to return then? “I wanted to, but this was my job for the past 12 years. I was helping support my family with that income. And God forbid, if Simon doesn’t make it? I just wasn’t strong enough to just walk away.”

Instead, Tamra turned to prayer.

“It was very difficult for me,” she says. “For three weeks, I was a f—ing mess. I cried every single day. And then they called and scheduled a production meeting for a few weeks before filming, and I thought, ‘There’s no way I’m going to be able to go back to this show and keep this a secret. There’s no way I could just film as if nothing’s wrong.’ So I just prayed and prayed and prayed for divine intervention. I was like, ‘God, please make this decision for me.’ ”

On the evening of Jan. 24, days before that scheduled meeting, Tamra says she got a call from production telling her that the network wanted her back, but only for a limited role. She was offered three episodes (for a considerable amount of money, it turns out) to wrap up her storyline and move on.

Tamra declined the offer. “Those three episodes wouldn’t make any sense to me because I am very much a person who is very honest on the show,” Tamra says. “At the time, Simon did not want to come forward with it, so I could not talk about it. It didn’t make any sense for me to be like, ‘I’m sorry, I’ve been very busy now, gotta go,’ because it would be a lie. I just knew there would be no way to get around it. I couldn’t go back and be myself. I couldn’t be honest. I couldn’t be real.”

Barney has since come forward with his diagnosis, telling PEOPLE the news exclusively this week.

“I’m a private person, despite what my time on reality television might say,” the 55-year-old said with a laugh. “But I wanted to speak before someone else does and uses their words, not mine. And to speak out about it so others can see how I’ve taken this fight on with optimism, and the good this has ultimately brought my life.”

Tamra says she’s inspired by Simon’s attitude. “I like that he’s being positive, and he’s being super positive around the kids. I think he has a great outlook, and most people who beat cancer have a similar outlook, so I see that,” she says. “But he had said to me when I first saw him, ‘If it’s my time to go, it’s my time to go.’ And that just broke my heart. Like, no, that’s not happening. We’re going to get through this.”

As for the potential of a RHOC return one day, Tamra says that’s not on her horizon. “Andy has said he thinks I’ll be back one day, but right now, I can’t even think about that,” she explains. “I’m thankful for my time on Bravo, I really am. I’ve loved doing the show and love the people there. But I just want to be with my family. Nothing else matters.”