Justin and Hailey Bieber Are 'Even More in Love' Since Welcoming Baby Jack: 'In a Happy Bubble' (Exclusive Source)

Love is in the air for the Biebers and their family of three!

Almost a month after Justin and Hailey Bieber announced the birth of their first child, a source tells PEOPLE that the couple appears "more in love" than ever now that they have son Jack Blues Bieber in their lives.

“They had a quiet wedding anniversary celebration," the insider says, referring to Justin and Hailey's recent 6th anniversary of their September 2018 New York City courthouse wedding.

"They seem even more in love since the baby arrived," the source adds.

The source also shares that the new mom and Rhode founder is "feeling great," and that her husband is "encouraging her to catch up with friends." The new mom has "been out a few times without the baby," the insider says.

Meanwhile, the source also notes that new dad Justin has "been in a happy bubble since baby Jack was born," adding that the 30-year-old Grammy winner "wants to focus on being a great dad and husband."

The proud parents announced the arrival of their son on Aug. 23 by sharing the news on Instagram, alongside an image of the newborn's foot.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

His name Jack also continues a family tradition for the Biebers — as the baby shares the initials ‘JB’ with his dad, his grandfather (Jeremy Bieber) and some of Justin's siblings.

Since Jack's birth, Justin and Hailey, 27, have provided just a few details on parenthood via social media, such as when Justin shared a peek at his cute "papa bear" coffee mug on Sept. 3 and Hailey also shared a glimpse of a "mom" ring with her followers the same day.

On Sept. 13, Hailey posted a touching tribute on her Instagram Stories in commemoration of the pair's latest wedding anniversary. She documented what appeared to be an at-home celebration by sharing images featuring balloons, candles and an adorable instant camera picture. The photo itself showed her and Justin sharing a kiss while posing in front of candles.

Other members of the Bieber family have shared insight into life now with baby Jack, as Hailey's dad Stephen Baldwin described his "cute" new grandson while attending the People + Chain 50th anniversary party in Los Angeles on Sept. 13.

"I don't want to say too much. He's unbelievably cute,” the actor said at the time, adding that “the world will see him soon.”

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

In late August, a source also told PEOPLE that the "Baby" singer was "already a great dad" and that the "pregnancy was something that they very much wished and prayed for."

"The day they found out that Hailey was pregnant, was the best ever for Justin," the source said, adding that he was "over the moon with excitement."

"It was a big celebration for them," the insider said.