AI and the Future of Healthcare

Key Insights from this Article

• How is AI and augmented reality a game changer in complex surgeries?

• How to take a human-centered approach to innovation?

• What is the next bound of innovation in the future of healthcare?

The Subject

The remarkable advancements in Holomedicine and mixed reality technology in medicine demonstrate the immense potential of this human-digital synergy. At National University Health System, we have witnessed how these cutting-edge technologies enhance surgical precision and improve patient outcomes. By augmenting our understanding of the human anatomy and facilitating better communication with patients, NUHS empowers individuals to actively participate in their healthcare decisions.

Holomedicine is already revolutionizing patient care. In one instance, a patient's CT and MRI scans were converted into 3D holograms, drastically enhancing our spatial awareness and anatomical understanding during a complex surgical procedure. This technological leverage facilitated the precise and successful removal of multiple tumours from a patient’s pelvis without significant loss of surrounding healthy tissues—a feat previously unattainable with conventional methods. In a typical surgery like that, the patient would have lost their urinary and reproductive functions. But thanks to AR technology, the patient has since enjoyed about a year and a half of being free from symptoms from the condition post-surgery.

However, even with such clear benefits in improving communications with patients, the doctors realised they must still keep human-centricity in mind when designing the user experience. For example, when NUHS first adopted 3D scans and holograms, the doctors would upload raw images straight from the scanners, including CT scans and MRI scans. They encountered a negative response in a patient requiring brain surgery. When they reconstructed the MRI brain scan into a 3D hologram, the realism of the 3D image was so profound that the patient felt unsettled seeing her own face looking back at her. This incident made them realize that what doctors and clinicians perceive as high quality might not be the same for patients. Higher resolution and detail is not inherently tied to an increase in patient welfare.

The next step of the vision... digital twins of our physical bodies.

Imagine surgeons employing these sophisticated virtual models for meticulous pre-operative planning, thereby elevating the precision of surgical interventions. Clinicians could forecast patient-specific responses to medications and therapies, tailoring treatment plans to optimize efficacy and minimize side effects.

The concept of a "digital twin" is often diluted in today's technological landscape, with many digital twins being simple replicas of real-world objects. When applied to healthcare, digital twins should signify something far more transformative: highly accurate virtual counterparts that replicate not just the form, but also the functionalities of the human body. In essence, these digital avatars serve as comprehensive, virtualized clones of individual patients.

The implications of high-fidelity digital twins in medicine are nothing short of revolutionary. Imagine surgeons employing these sophisticated virtual models for meticulous pre-operative planning, thereby elevating the precision of surgical interventions. Clinicians could forecast patient-specific responses to medications and therapies, tailoring treatment plans to optimize efficacy and minimize side effects. On a broader scale, doctors could simulate the health trajectories of entire populations based on current demographic data, guiding public health interventions in a proactive manner.

This vision is still years away, but NUHS is making steady progress towards this future. As doctors traverse this complex landscape, they must remain cognizant that at the end of the day, it is still a real human being that they are treating, and not the biological version of a digital twin.

Embracing innovation with a compassionate and thoughtful approach will ensure we thrive in a world where humans and technology coexist in synergy, fostering progress without losing sight of our humanity.

The above write-up was co-written by Dr Gao Yujia, together with Deloitte Center for the Edge.

The Guest

As Deloitte Center for the Edge was researching for its publication on The Metaverse in Asia back in 2022, we found out that Singapore was a world pioneer in holomedicine, which uses augmented reality in actual surgery.

We were super excited to meet the actual surgeons leading this effort from the National University Health System (NUHS), who spent a few hours with us sharing passionately about driving innovation in their hospital.

Dr. Gao Yu Jia, Assistant Chief Technology Officer of the National University Health System, is leading the holomedicine programme at NUHS, where he is using mixed reality in his liver transplants. To execute the vision, Dr Gao personally spent his nights and extra hours installing 5G routers in his hospital, modifying Hololens headsets, and travelling to 5G and tech conferences around the world scouring for the best technologies.

On top of all of that, he is also educating the next generation of doctors as a lecturer. His students tell me they want to join his class because they've never seen any doctor so passionate and down-to-earth. Now, he's also sharing his knowledge with other hospitals around the world, including projects in Uganda and other developing countries on remote health. He is creating a healthier future for all.

We filmed this interview almost a year ago, and there have been lots of developments since then, like an AI-diagnostic tool for doctors, AI-powered instant ultrasounds, and so on. But this interview provides you with the fundamentals of the next-generation of healthcare technology.

Edges of Southeast Asia

We featured NUHS in our latest publication on the Edges of Southeast Asia. Read more here:

If you found this example interesting, check out 23 more in the Edges of Southeast Asia that highlight 6 different ways you can innovate your business model:

The Interview

Q1 How do AR and AI actively enhance patient health and experience?

DR GAO: One of the cases that I always like to cite, and I think really demonstrates the impact of technology like this is for a very young patient with relapse of tumors in the pelvis.

So, she has about seven tumors in the pelvis and each of these tumor spots are actually less than one centimetre. So, we can barely make it out on CT scans and MRI scans and with the naked eye, it will be impossible to see where exactly these tumors are. So, if we were not able to pinpoint where the tumors are during the surgery itself, this could potentially have rendered her at a very young age incontinent, she wouldn’t be able to control her urine, her bowels, and would have affected her reproductive functions as well.

So, what we did was that we got the PET scan, and then we uploaded it onto our MRI device, which is the HoloLens tool, and we generated a 3D hologram, which we can then superimpose or overlay on her during the surgery itself. And this actually allowed us to pinpoint exactly where the seven spots were, and we were actually able to very accurately remove those seven tumors.

Q2 How did your patients react to that technology?

DR GAO: When we first started using this technology, the devices, we used to upload the raw images. But I remember the thing that shocked us the most was when we used it for a patient who was going for brain surgery. And to her, she actually got a little bit freaked out because the reconstruction of the 3D image was so realistic that it was literally her head looking back at her because she can see her own face and everything. And that case actually made us realise that what we perceive as a high-fidelity, high-quality image as doctors and clinicians may not be perceived the same way by the patients.

And to her, she actually got a little bit freaked out because the reconstruction of the 3D image was so realistic that it was literally her head looking back at her.

Q3 How do these technologies enhance body awareness?

So what we try to do nowadays is to have the patients wear the device so that they can have a look at their own scans, and for example, you know, counselling and consenting for procedures are done in, I guess, a three-dimensional world rather than a two-dimensional world.

 So, from the doctor’s perspective, it allows us to view the patient’s scans, CT scans, MRI scans, in three dimension with the 3D holograms. It allows us to appreciate the structures and the relationships a lot better compared to a two-dimensional screen which we use on a computer. And in that way, it translates to, of course, a better explanation and also better communication with the patients and their families as well.

Q4 How could the future Internet impact our physical and mental health negatively?

DR GAO: When the internet first came about, everyone was very apprehensive about it, we're worried that it's going to take over the world, it's going to cause a meltdown, the whole Y2K thing, but things turned out fine. Like the article that you wrote, pointed out, a lot of it, it's about finding the balance, right? Finding the balance between directed and empowering use of the Internet, finding the balance between meaning and utility. And also finding the balance between responsibility as well as regulations.

Progress with technology is always going to happen, right? We can't stop the progression of technology, whether or not it's going to bring harm or good, it really depends, perhaps in a very naive way, how we intend to use it and how we intend for people to consume and to accept it. But of course, if we abused technology, if we abused the internet, then they will definitely cause harm. But if let’s say we use it in the proper way, if let’s say we find the balance between the light and the dark side, if we can put it that way, then then of course, it’s going to bring tremendous amounts of benefits and advantages to us as humans.

Progress with technology is always going to happen, right? We can't stop the progression of technology, whether or not it's going to bring harm or good, it really depends, perhaps in a very naive way, how we intend to use it and how we intend for people to consume and to accept it.

Q5 What does being human in a digital world mean to you?

DR GAO: Being human in the digital world means recognising that we are human, and we will never be replaced by technology. Technology is there to assist us and to make our lives better, but not to replace us as humans and humanity.

Dr. Gao Yujia is a Consultant Surgeon in Liver, Pancreas, and Liver Transplant Surgery at the National University Hospital in Singapore. As the Assistant Group Chief Technology Officer for the National University Health System, Dr Gao is involved in the research and development of immersive technology and its application in clinical care and education. He works extensively with Mixed Reality (MR) technology, utilising MR devices to deliver cutting edge capabilities to clinicians including 3D holographic imaging, real-time computer-vision based imaged analysis, and multisource data integration. He is also responsible for the integration and implementation of 5G wireless technology for hospital infrastructure development and building of secured high-speed integrated data networks.

Michelle Khoo co-leads Deloitte's futures thinktank Center for the Edge.

This is part of a video series by Deloitte and Center for the Edge, where we interview thought leaders from across different sectors and demographics on "What it means to be human in a digital world" and how they are navigating an AI-driven future.