Justin Bieber and Hailey Baldwin Seen at New York Courthouse's Marriage Bureau

Justin Bieber and his fiancée Hailey Baldwin were seen at a courthouse where they issue marriage licenses on Thursday.

In a photo obtained by TMZ, the lovebirds were spotted walking hand-in-hand into the building. For the occasion, Baldwin, 22, opted for a white ensemble while Bieber, 24, wore a striped shirt and dark pants.

The outlet also reported that Baldwin and Bieber appeared to be extremely emotional with a courthouse insider saying Bieber told Baldwin, “I can’t wait to marry you, baby.”

A source also told TMZ Bieber told a court official, “Thanks for keeping it on the DL.”

While their courthouse appearance certainly raises eyebrows, a source close to the couple claims to PEOPLE that they didn’t get married on Thursday. A separate friend of Bieber tells PEOPLE a wedding is still in the works.

“They have hired a wedding planner and have been looking at venues,” ” the friend says. “The plan for now is to have a real wedding.”

The sighting comes as Baldwin discussed her wedding plans with The Cut.

“I just picture lights strung everywhere,” the model explained when asked about her upcoming nuptials.

“I think having it in the woods would be so beautiful,” Baldwin continued. Baldwin also told the magazine she has a great wedding planner and is thinking of having the ceremony on the West Coast.

Later in the interview, Baldwin addressed whether or not her family approves of her getting married at such a young age.

“My sister was 24 when she got married, and my parents also got married when they were young, too. I see so no reason to wait. When you know it’s right, it’s right,” Baldwin told The Cut.

When it comes to how well Baldwin’s parents and Bieber get along, she said, “I think they love him more than me!”

“I said to them, ‘You’d stop me if you didn’t think this was the right decision, right? And they didn’t.”

This week Bieber spent some time with his soon-to-be father in law, Stephen. The two were photographed in Grand Rapids, Michigan en route to a music festival.

The “What Do You Mean” singer popped the question to Baldwin in July while the two were on vacation in the Bahamas at a local restaurant.

Bieber later confirmed the news on Instagram writing, “So committed to spending my life getting to know every single part of you loving you patiently and kindLY.”

“I promise to lead our family with honor and integrity letting Jesus through his Holy Spirit guide us in everything we do and every decision we make.”