Why Steven Spielberg Says He's 'Very Happy' He Turned Down 'Harry Potter' Directing Job

Steven Spielberg is opening up about balancing work and family — and how he once "sacrificed a great franchise" to spend more time with his wife and children.

"There were several films I chose not to make. They offered me Harry Potter," the acclaimed filmmaker, 76, recalled in a recent chat with RRR director S.S. Rajamouli for Reliance Entertainment.

Spielberg explained he "chose to turn down the first Harry Potter" so he could "spend that next year and a half with my family, my young kids growing up."

"So I sacrificed a great franchise, which today looking back I'm very happy to have done, to be with my family," he added.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Back in 2010, Collider reported that Warner Bros.' then-president Alan Horn told the Los Angeles Times he "did think it would be worthwhile for Steven Spielberg to direct" the first Harry Potter film, 2001's Harry Potter and the Sorcerer's Stone, and that the studio had offered it to him.

"But one of the notions of Dreamworks' and Steven's was, 'Let's combine a couple of the books, let's make it animated,' and that was because of the [visual effects and] Pixar had demonstrated that animated movies could be extremely successful," continued Horn, per Collider.

"Because of the wizardry involved, they were very effects-laden. So I don't blame them. But I did not want to combine the movies, and I wanted it to be live-action," he added of the film, which was eventually directed by Chris Columbus.

Columbus, 64, also led 2002's follow-up Harry Potter and the Chamber of Secrets, while subsequent installments in the wildly successful boy-wizard franchise were directed by Alfonso Cuarón, David Yates and Mike Newell.

Spielberg's new semi-autobiographic film The Fabelmans, which he co-wrote, is up for seven awards at this year's Oscars, including Best Picture and Best Director.

In his conversation with Rajamouli, 49, Spielberg said the "personal meaning about [how the conflict between] art and family will tear you in half happened to me later, after I had already established myself as a filmmaker, as a working director."

"The choice I had to make in taking a job that would move me to another country for four or five months where I wouldn't see my family every day … that was really a rending, ripping kind of experience, the choices I had to make," he said.

While the iconic director is now a proud father of seven, he didn't always want children. In fact, it was his experience working on 1982's E.T. that opened him up to the idea of parenthood.

"I didn't want to have kids because it was not a kind of equation that made sense for me as I went from movie to movie to movie, script to script," Spielberg told Variety last year at a celebration for the iconic film's 40th anniversary. "It never occurred to me till halfway through E.T.: I was a parent on that film. I was literally feeling like I was very protective of Henry [Thomas] and [Robert MacNaughton] and my whole cast, and especially Drew [Barrymore], who was only 6 years old."

"And I started thinking, 'Well, maybe this could be my real life someday,' " he added. "It was the first time that it occurred to me that maybe I could be a dad. And maybe in a way, a director is a dad, or a mom … I really felt that that would be my big production."