Justin Bieber Kisses Wife Hailey in Cryptic Photos of Him Teasing a Potential Music Comeback

Justin Bieber might be teasing that new music on the horizon — and wife Hailey Bieber is supporting him along the way!

On Sunday, Oct. 20, the musician, 30, shared an Instagram carousel that featured cryptic photos of him seemingly working on new music in a studio.

The final picture in the batch featured him and Hailey, 27, going in for a kiss. Most of the photos showed the new dad making music on pianos, keyboards and sound boards. One photo showed Bieber sipping coffee.

Bieber captioned the post with only photographer Rory Kramer’s name.

Bieber hasn't released a solo studio album since Justice in 2021. A new album now would be his seventh since his 2010 debut, My World 2.0.

This potential new music from the two-time Grammy winner would be some of his first since 2022 and 2023, when he and Don Toliver released their joint singles "Honest" and "Private Landing."

On Saturday, Oct. 19, Bieber stopped by Toliver's show — with Hailey watching the show from the crowd — at the Crypto Arena as a surprise guest for a live rendition of "Private Landing."

Bieber appeared on the stage in a white beanie, black hoodie and black glasses, as seen in a clip shared by model Lori Harvey, 27, on her Instagram Stories. The "Love Yourself" singer stood on the opposite side of Toliver onstage as he danced to the song while holding his microphone in the air.

"Make some noise for f------ Justin B," Toliver said as Bieber entered.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

The new photos come about two months after he and his wife announced the arrival oftheir first child, son Jack Blues Bieber, on Aug. 23. Justin shared the news in sweet post on Instagram that featured an image of the newborn's foot.

"WELCOME HOME JACK BLUES BIEBER," he wrote in the caption. 

In September, a source exclusively told PEOPLE that the couple are "more in love" following the birth of their son and after celebrating the sixth anniversary of their September 2018 New York City courthouse wedding.

"They seem even more in love since the baby arrived," the source said.

The insider added that the Rhode founder is "feeling great" and that her husband is "encouraging her to catch up with friends." Hailey has "been out a few times without the baby," the source said. 

The insider also revealed that Bieber has "been in a happy bubble since baby Jack was born" and that the Grammy winner "wants to focus on being a great dad and husband."