Dwyane Wade Says It's 'Challenging' Not Living with Youngest Son Xavier, 8: 'Life Isn't Perfect'

Dwyane Wade is opening up about his desire to have a strong relationship with his youngest son in his new photographic memoir Dwyane.

In the 39-year-old NBA star's book, he writes about the occasional challenge that comes from parenting his 8-year-old son Xavier Zechariah, whom he shares with a woman from a previous relationship.

"Xavier doesn't live with me, so we don't get to see each other every day like I do with my other kids," Wade explains in his memoir. "It makes things challenging at times, but it's my job to make sure he never has to carry that weight."

Along with Xavier, the former Miami Heat player is father to son Zaire Blessing Dwyane, 19, daughters Zaya, 14, and Kaavia James, 3, and is raising his nephew Dahveon Morris, 20.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

"It's up to me to make sure he knows he's being raised with love even if we don't live under the same roof," Wade adds of Xavier, according to Dwyane. "And it's up to me to let him know that while distance may physically separate us at times, I'm always going to be the man he can depend on to love him and help lead him."

He continues, "What I've come to learn over my life is that fatherhood isn't a monolithic image that looks the same in every household. Each relationship is different. All I'm concerned about with Xavier is him always knowing he's loved."

Wade expresses his love of family throughout Dwyane, which features sweet photographs of all of his kids, his parents and wife Gabrielle Union, 49. The three-time NBA champion recently spoke to PEOPLE about his parenting style and allowing his children to be themselves.

"I think kids appreciate when you try to get to know them," Wade told PEOPLE. "And you know what I've always tried to do with my kids, from Zaire all the way down to Xavier, from Kaav and Zaya, I try to meet them where they're at, and I don't try to always bring them to where I want them to be, or my world."

He added, "So if Xavier is into Fortnite, then I'm going to try to learn as much about Fortnite as I can to meet him where he's at so we can have conversations about Fortnite, right?"

Wade's photo-centric memoir also delves into his personal journey, growing up in the South Side of Chicago and how his childhood taught him what he does and doesn't want for his own family.

"Because life isn't perfect, and every situation won't be perfect, it's the quality of the moments you spend with someone," he told PEOPLE. "It's the attention to detail with someone that means more than a parent that is home all day, every day."

Dwyane is on sale now.