Jesse Tyler Ferguson Says He Doesn’t Want More Kids After Welcoming Second Baby: 'We're Busy'

Jesse Tyler Ferguson is happy with his family of four.

During an appearance on Wednesday's episode of The Jennifer Hudson Show, the Modern Family actor and Tony winner, 47, opened up about fatherhood and whether he wants to expand his family. 

Ferguson and husband Justin Mikita are dads to sons Beckett Mercer, 2, and Sullivan "Sully" Louis, 6 months.

When host Jennifer Hudson asked Ferguson, “Do you think you could have more?” he firmly responded with a “No.”

He explained on the syndicated talk show that he and Mikita, a Tony-winning producer, had their hands full with raising two children. The boys were their sole focus at the moment, Ferguson said.

“We’re busy,” he said. “Beckett’s potty training right now… but that’s going good! He’s so interesting, he has so many interests and he’s funny.”

“And [Sully] is sort of just like, you know, babies are blobs and they don't really do a lot, and so I think we’re all kind of like waiting… like ‘TikTok we’re ready to have fun with you,’ ” he joked. “'Like anytime you’re ready to get up and join the party.’ But it’s been fun. It’s been great.”

He noted that while being on Modern Family did teach him some lessons about fatherhood, it ultimately didn’t prepare him for the real life challenges.

“When I had the baby on Modern Family, they get fussy and then like three people come in and swoop the baby away and a second baby comes and you’re like, ‘Oh, this one’s calm,’ and that doesn’t happen in real life,” he said. “No one swoops in and gives you a non-fussy baby!”

Last month, Ferguson spoke with PEOPLE about watching his sons grow up and the lessons he's learned from parenting.

"It's really nice settling in and seeing his daily changes," Ferguson told PEOPLE. "I kind of forgot how quickly babies change. Every day, it's something completely new."

The Tony Award winner, who welcomed son Beckett in 2020 and Sully in 2022 via surrogate, noted that raising his two sons have been a different experience.

"We were hands-on all this time, kind of staring at him because we were all stuck together. Now because Beckett is very active and the world sort of come back, I'm working, and Justin is working, and we're using our in-laws and parents more to help us out, when before everyone sort of had to stay away," he explained.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

While it's "completely different," the screen and stage star said, "I enjoy doing it this way more."

"I definitely liked having the one-on-one time with Beckett, but I think doing it communally with a family and in a place where you can go out and enjoy the world, is a huge help," he added.