Watch Jack Antonoff Join Lana Del Rey to Play 'Margaret' — Her Tribute to His Wife Margaret Qualley

Lana Del Rey brought Jack Antonoff out on stage on Sunday to play a very special song live for the first time. 

At All Things Go Festival outside of Washington, D.C., the songstress, 38, invited her frequent production partner, 39, out to perform two songs during her headlining set. Fans captured footage of the duo playing their collaboration “Margaret,” off Del Rey’s recent album Did You Know That There's a Tunnel Under Ocean Blvd, live for the first time, as well as 2019’s “Venice Bitch.” 

There was even a special guest in the crowd. Antonoff’s wife and the subject of the song, actress Margaret Qualley, was seen sitting in the crowd just before the front of the stage. 

“He’s here with his beautiful wife Margaret,” the singer-songwriter said as the Bleachers frontman came out, surprising the crowd. “So, we’re going to do a little jam.”

The duo first played “Margaret,” marking the sweet song about Antonoff and Qualley’s romance it's onstage debut. Antonoff played acoustic guitar and sang his charming verse, while the “A&W” artist gave a sweet performance, frequently smirking throughout. 

“When you know, you know / When you know, you know / It kinda makes me laugh, runnin' down that path / When you're good, it's gold / 'Cause when you know, you know,” the singer crooned to the audience and her friends. 

The crowd was thrilled that Antonoff made an appearance, and screamed even louder as he and Del Rey performed a second song together as soon as “Margaret” wrapped. They then launched into a stripped-back, hypnotic rendition of the fan-favorite Norman F---ing Rockwell song “Venice Bitch.” 

The star also shared clips of her performance on Instagram, expressing her gratitude to the Maid star and “I Wanna Get Better” singer in the caption. “Thank you for the full moon show, Baltimore,” she wrote. “And to Jack and Margaret for coming. 🖤🎥.”

The Grammy winner has worked with the “Video Games” star since her sixth studio album. He previously joined her on stage earlier this year at Newport Folk Festival where they played her song “Mariners Apartment Complex” and a cover of Joni Mitchell’s “For Free.” 

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

In addition to a handful of recent festival appearances, alternative star is currently on a 10-date run of fall North American tour dates. 

In August, the Grammy-winning producer tied the knot with Qualley after nearly two years of dating. It was a star-studded affair in Antonoff’s native New Jersey and was attended by famous friends including Del Rey, Taylor Swift, Channing Tatum, Zoë Kravitz and Cara Delevingne. 

The rocker also recently released “Modern Girl,” the first single from his upcoming fourth studio album with his band Bleachers.