Billy Ray Cyrus Is 'More Relieved Every Day' That Marriage to Firerose Is Over, Felt 'Beyond Deceived,' Source Says (Exclusive)

Billy Ray Cyrus feels "relieved" to be moving on from his marriage to Firerose, a source tells PEOPLE.

The country star, 62, is "more relieved every day that he's done with his marriage" after filing for divorce from Firerose, 36, on May 23. The couple had been married for seven months.

"His instincts have been correct all along," the source says. "He believes she only married him for financial and other reasons. He's convinced that she didn't marry him for love."

In the May filing, Cyrus cited irreconcilable differences and inappropriate marital conduct as reasons for the split.

Firerose responded to the filing in a Friday, June 14 counter complaint, claiming that Cyrus filed for divorce just one day before she was set to undergo a preventative double mastectomy and accused him of "extreme verbal, emotional, and psychological abuse."

"Wife was subjected to Husband’s persistent drug use accompanied with the consumption of marijuana which made Husband unpredictable and volatile towards Wife," her attorneys wrote in the complaint obtained by PEOPLE.

Cyrus responded to the accusations by filing a supplemental complaint to his initial annulment on Monday, June 17. The complaint alleged that Firerose conducted a "campaign to isolate" him from his family and "blocked at least one of his daughters" from being able to contact him on his electronic devices.

With the new claims, Cyrus also alleged that Firerose spent $96,986 on 37 unauthorized charges on his credit cards, and claimed she wrote him a handwritten note professing her love to him.

“I miss you more than I can possibly explain in words Baby. Please let’s work this out and not become anything but closer out of this hellish experience. I need you. I love you. I am so sorry," the alleged note provided by Cyrus' lawyers, Rose Palermo and Jason Talley of Cheatham Palermo and Garret, and obtained by PEOPLE reads.

The attorneys told PEOPLE that Cyrus allegedly found it when he returned to his home two days after Firerose “was required, by court order, to vacate his residence of over 25 years."

Cyrus' lawyers also alleged that Firerose sent him a similar text message “shortly after she was required to vacate his premarital residence.”

“If Mr. Cyrus was truly guilty of the allegations that [Firerose] alleges in her pleadings, then it is mindboggling to try and explain why she would want to return to live with him. Furthermore, she was pleading with him to give her a chance to explain everything to him," his lawyers said in a statement to PEOPLE.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

The source tells PEOPLE that Cyrus "feels the back-and-forth now, very much also shows [Firerose's] true colors" and that he "feels beyond deceived and is ready to move on alone."

A representative for Firerose did not respond to PEOPLE’s request for comment about the alleged note, text message, or Cyrus' updated allegations.