Jimmie Allen Shares How Bond with His Late Dad Inspired New Single 'Down Home': 'Writing Helps'

When ACM Awards co-host Jimmie Allen took the stage earlier this month to sing his new single "Down Home" before a Las Vegas crowd that included both his wife and his mom, it was a career high.

But for the country star, who'd dreamed as a teenager of hosting the show as he watched from home with his late father, there was something missing.

"From the moment rehearsals started to the day before to the shooting of it, I kept picturing my dad sitting out there next to me wife and my mother," Allen, 36, tells PEOPLE. "It was a good moment, it was more happy thoughts than sad. [But] every now and again it'd be like, 'Damn, I wish he was here.' Or like, 'Is that him? Is that really him? OK, no it's not.'"

Allen lost his 65-year-old dad, James "Big Jim" Allen, in 2019, and his newest single "Down Home" pays tribute to the man the singer has described as "big-hearted, kind and patient."

The song's lyrics serve as a letter to James, and fill him in on all that he's missed back at home, like Allen's marriage to wife Alexis in 2020 and the birth of his daughters Naomi, 2, and Zara, 5 months.

Getting to a place where he was able to channel his immense grief into song took time, and a little coaxing from his co-writers Cameron Bedell and Rian Ball.

"They were pushing me for over a year to write a song about my dad and I was like, 'Yeah, nah. I ain't trying to do that,'" Allen says. "Emotionally, I wasn't ready and I didn't feel like it was the right time. Sometimes writing helps; it depends on the day. Sometimes fishing helps. Sometimes doing nothing helps. Sometimes, a lot of times, working helps. I kind of just take it day by day, moment by moment."

Eventually, though, the Dancing with the Stars alum was able to focus on the little things that reminded him of his dad, including James' love for the number 15, which the singer says is "the Allen family number," and one that his 7-year-old son Aadyn now wears on his jerseys.

A poignant music video shot entirely in black-and-white in Allen's hometown of Milton, Delaware also serves as a tribute to the patriarch, with scenes featuring Allen's childhood home and best friend, the field he used to play baseball on and his dad's favorite cheesesteak and fishing spots.

"It brought back a lot of memories," he says of heading home to film the video, which premieres on YouTube Friday at 6 p.m. ET. "I love Delaware. Every break I get, I go there. We fish all day and go to Moose Lodge and drink at night. Only place you can go get drunk and full for $9.28!"

Since losing his father, Allen — who is nominated for best new artist at the Grammys — says he's been able to place a renewed focus on the importance of the little things in life, including spending as much time with family as possible.

"It really re-emphasized the fact that life is short," he says. "Always make time for your family or friends, your people. We spend as much time as we can together, we talk a lot. That's really what I do, just make time for people I care about."

He's also been able to reap the wisdom imparted upon him by his parents' relationship, which the Bettie James singer says proved to him that love can be unconditional, even amid obstacles.

Though James and Allen's mom Angela split when the star was around 12 years old, they remained friends and had a "great" relationship until the day his dad died.

"My mom never moved on. She was still in love with my dad. And in a weird way, it was beautiful," he says. "I think a lot of times people have issues growing up in a single-parent home, or split parent. I didn't because the love never changed, and the love I saw them have for each other never changed."

He continues: "Not saying that they didn't argue, but I didn't see it — and I feel like that's how it should be. My son, me and his mom have disagreed before, but he'll never see it. He'll never hear me talk negative about it because that's his mother… [My parents] showed me about loving unconditionally and loving people even if it don't work out."

His mom was among the first to hear "Down Home" when he performed it live for the first time this past summer at Bettie James Fest, a festival he organized in his Delaware hometown.

"I remember I sang 'Down Home' there for the first time anybody heard it. My mom, I saw her backstage crying. I'm onstage crying. I could barely get through the song," he recalls. "The song should've been a three-minute song, but it ended up being like, six minutes because I couldn't keep myself together."

By the time he played the song at the ACM Awards, though, he'd managed to pull it together for his siblings, whom he say play the track often, the world, and of course, his mother.

"She got to see me sing a song I wrote about the man she was in love with her entire life on her birthday and co-host the ACMs," he says. "It was a night."

"Down Home" is out now, and the music video will be released on March 18.