Justin and Hailey Bieber Visit Church Service in Los Angeles After Welcoming Son Jack Blues

Justin and Hailey Bieber took a rare break from parenting duties for a couples' outing.

On Wednesday, Sept. 25, the new parents were spotted leaving a church service in Los Angeles, just over a month after announcing the arrival of their son, Jack Blues Bieber.

The pair were photographed exiting the church together, with Hailey, 27, leading the way dressed in a long leopard print jacket, black Intimissimi top, blue baggy jeans and red Adidas sneakers. The Rhode founder teamed the look with a matching leopard print handbag, which she carried over her shoulder. 

Justin, 30, meanwhile, matched his wife’s sneakers in an unzipped red hoodie, black pants, furry Louis Vuitton boots and a beige fedora. He also carried his "Papa Bear" mug, which he first showed off on Instagram earlier this month. The adorable coffee cup features a cute cartoon bear wearing a bowtie and holding up a heart with the words "Papa Bear" written above.

The couple’s church visit comes after a source close to Justin told PEOPLE that parenthood is something the duo “very much wished and prayed for.” 

"The day they found out that Hailey was pregnant was the best ever for Justin,” the source continued. “He was over the moon with excitement. It was a big celebration for them."

Last week, a source also told PEOPLE that the pair appear "more in love" than ever now that they are a family of three. 

Justin has "been in a happy bubble since baby Jack was born," the source said, adding that the musician "wants to focus on being a great dad and husband."

The source also shared that Hailey is "feeling great" and has "been out a few times without the baby.”

Justin and Hailey announced the birth of their first child on Aug. 23. 

"WELCOME HOME JACK BLUES BIEBER 🐻," the “Peaches” singer wrote on Instagram alongside a photo that showed his wife holding their baby boy’s foot. 

The model also posted the same shot on her Instagram Stories with her son’s name and added a teddy bear and blue heart emoji.

The PEOPLE Puzzler crossword is here! How quickly can you solve it? Play now!

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Just weeks later, the couple marked their sixth wedding anniversary on Sept. 13 with a low-key celebration. 

Sharing a glimpse into their special day, which marked six years since their 2018 New York City courthouse wedding, Hailey posted pictures of candles, flowers, heart-shaped balloons and a Polaroid snapshot of the couple. 

“6 years🤍💍,” Hailey wrote, adding, “Love you baby" over a photo of her hand holding an instant camera picture of herself and Justin kissing.