XR in Spatial Computing: How Consumers Will Interact within an Immersive World

Remember when people used two-dimensional (2D) mobile phones and computer screens to shop, communicate, recreate and get things done? 

In the next 10 years, consumers will increasingly live and work in a three-dimensional (3D), experiential world. They will interact and transact in this 3D realm—often called spatial computing—through Extended Reality (XR), using sophisticated augmented reality (AR) and virtual reality (VR) devices, which will bring the space to life through visuals, audio and even olfactory and haptic cues, guided by users’ behavioral data.  

This will bring about a new type of marketplace that will unlock many new opportunities for consumer-facing brands and related ecosystem players. The value of just one category of this market, global mobile AR revenue, will grow from US $3.98 billion in 2019 to US $21 billion in 2024, a 39 percent compound annual growth rate (CAGR).[i]Imagine, for example, using your high-end XR glasses to see and touch a location-based app store that hovers in mid-air. As you walk by retail stores or restaurants, the dynamic apps and promotions change based on your needs and preferences. Or think about being able to see a 3D digital map of your kitchen and interacting with your appliances through simple gestures. If your toaster stops working, you can use the XR glasses to access the user manual and order a replacement part. 

Reaching consumers in this completely different world—one in which any consumer activity that leads to a sale is powered by XR—will require brands to develop a new business and technology strategy, supported by advanced security for XR devices and cloud platforms, as well as protections for individuals’ data privacy. 

It will not be enough to convert consumer-facing interactions to VR or enhance mobile apps with AR graphic overlays. Instead companies must innovate with XR at speed and scale to become trusted brands in an experiential sphere where immersion is the norm. Retailers, hospitality and entertainment providers, consumer packaged goods companies and other brands must start building toward this future now. Ditto for ecosystem players providing the backbone of this XR world—network infrastructure, hardware devices, apps/software applications or advertising. 

Additionally, the global pandemic has increased the urgency: now XR innovation must become a C-level business priority for consumer-facing brands. Almost overnight people’s behaviors and commerce interactions fundamentally changed in terms of how they shop, communicate, view entertainment, collaborate and seek medical advice. Case in point: 60 percent of patients want to continue to use virtual care and nine of out 10 patients say the quality of care is as good or better than before, according to one recent Accenture survey.[ii]   

To succeed in this XR-powered future, brands and related ecosystem players must answer a range of questions: How will the consumer marketplace evolve with XR in the next decade? How will consumers use XR devices in their daily lives? How should brands enter the market and partner with ecosystem players? What type of consumer data will be necessary to deliver these individualized realities to every person? And what are the security and privacy implications?

Read on to gain Accenture’s perspective and recommendations for how to proceed with XR.

The XR market evolution

The spectrum of XR-enabled experiences has gained momentum in recent years, starting to transition from a nascent technology mostly dedicated to gaming and entertainment, into an enterprise tool for productivity and now an appealing way for brands to engage with consumers experientially. 

The market has gone from the early days of virtual reality (VR) gaming to today’s more advanced VR movies and immersive sales experiences. From beta versions of assisted reality glasses to augmented reality (AR) on mobile phones that enable location-based ads or product configuration. Now it is increasingly moving toward mixed reality(MR), which blends computer-generated and real content to help people interact with the real world. And as soon as 2021, the first generation of full-fledged XR glasses will emerge—providing new opportunities for brands to imagine and deliver XR applications and digital advertising.  

Figure 1: XR mediums provide different experiences. Copyright Accenture

That is just the beginning. Increasingly, consumers will ditch their mobile phones and conduct more of their daily lives and work via comfortable XR glasses with universal functionality. These glasses will switch seamlessly between full VR immersion and AR interaction with a digital twin replica of the world. Individualized information will appear based on where and what people are looking at, as well as what type of behavior or response they display during their interaction with specific elements of the world.

Does this future sound far-fetched? It is not. Just look at the mobile phone market growth of the last 10 years. Consumer demand for mobile pulled the plug on anything larger than a handheld screen. Phone usage morphed from a tool to make calls and text friends to an anytime-anywhere interaction channel for video, shopping, recreation and work. People brought their smartphones to the office, a tell-tale sign of how consumer use drives enterprise adoption; in time, many companies responded with bring-your-own-device policies, employee apps and on-the-go collaboration tools. We believe this is a good proxy for the consumer XR market. 

Technology tipping points driving this growth will be varied, based largely on how quickly XR ecosystem companies get products and services to market. In the near term, most consumer-facing opportunities will still come from advertising through smartphone-based experiences just as Snapchat, Instagram, Wayfair and other companies are doing now. Longer term so much depends on the XR devices themselves, including a form factor that is appealing to consumers and comfortable to wear for long periods of time. A handful of large companies are working to meet this demand with the first generation of XR glasses. We are starting to see these devices offered as a discounted companion product to the next generation of 5G mobile phones.

In the bigger picture, there will a progression toward this XR-powered future: First, consumer adoption and traction will most likely be driven through standardized technologies, including sharper XR displays and optics, location awareness and natural user interfaces such as gesture recognition. Next will come acceleration and maturation with the proliferation of 5G networks, edge computing and split rendering, all of which will make it possible to push content easily from cloud platforms to XR devices wherever people go. This stage will also depend on the availability of 3D assets at scale. Finally, advances in device power consumption, battery life and standalone computing will tip the scale toward mass adoption of XR in spatial computing—paired with a global scale 3D map and context-based access. 

Accenture Immersive Collaboration Platform 

In the time of COVID-19, technology innovation in XR is making it possible to continue living safe, productive lives until people can responsibly return to the office and travel.

Accenture is using the third generation of its teleportation solution, the Immersive Collaboration Platform, to keep global teamwork active among our workforce, as well as to meet with clients, share demos and workshops, and host both large-group and private meetings that foster greater interaction than calls and emails. 

The internal Accenture framework creates multi-user, photorealistic virtual environments for remote collaboration; it can also be used for training and education, virtual real estate and digital events solutions. The platform is unique in terms of its quality and rendering, but also accessibility. It supports a fully immersive, interactive experience through Oculus VR headsets. But unlike other approaches, it also supports seamless participation through iPads and desktops. This make real-time, “in person” immersive collaboration possible through existing devices, opening the experience up to more individuals. 

XR technology is also the foundation for the next evolution of work. In a post-pandemic world, many of the adaptations made to our ways of working will stay. Companies have seen firsthand just how much work can be done remotely, and how the flexibility of remote work can benefit employers and employees alike. There will always be a place for in-person work, but there is also a greater opportunity for remote collaboration—especially when it is immersive.

In this future, VR goggles or XR glasses will become a portal into the office. People will get the experience of being in an open space, and the feel of real interaction with colleagues who may be oceans and time zones away. Digital twin technology—another innovation Accenture is continuing to advance and apply within our Technology Innovation organization—can create a digital version of a real-world office environment. It can even support persistence capability: if you leave notes on a whiteboard in your “office,” they will be there next time you return. These touches help bring the “realness” of a physical work environment into the digital space, blending the best of both worlds to power the future of work. [iii]

Balancing consumer and brand expectations for XR

Brands spend billions of dollars annually to understand consumer trends—what kinds of products and services people want, what they do and do not value, how they want to interact. As a medium, XR provides a unique and compelling mode of interaction that mobile and online channels lack, through the detection and collection of behavioral data. However, like most technology adoption cycles, XR will also require a balancing act between what brands are looking to accomplish and what consumers are willing to accept. 

Today’s consumers want new and immersive ways to experience products and services that respond to both a practical and an emotional demand. For example, they want to be able to see and “fall in love” with a car by viewing it and interacting with it in 3D. Consumers also put a premium on customization and the ability to configure products to reflect their preferences, meaning they want to be able to compare features and move further along in the sales funnel from the comfort of their home. They prefer a multitude of solutions that enhance their daily experiences and individual productivity, with integrated social elements allowing them to connect with their personal network. And they want all this in affordable XR devices that are comfortable and stylish.

Meanwhile, brands are looking for ways to optimize sales and content distribution to deepen their relationships with consumers. This includes leveraging new immersive commerce platforms to sell existing goods and services, such as pre-sale visualization and customization along with post-sale interactive/augmented customer support. It also means new or improved 3D product design in real time. All the while, brands are seeking better ways to gather and analyze consented individual behavioral data for meaningful leads and more targeted marketing campaigns. 

Given these expectations, the balancing process continues: On the consumer side, personal privacy will be critical, and people will demand secure devices and secure cloud environments that protect personal information from leaks, particularly as behavioral data gets captured across devices. (For more on this topic, see the sidebar, “Implications for XR in Spatial Computing.”) For brands, the economics of business value will be at major factor along with regulatory compliance.

Eight functional areas to start

Given market projections, hardware offerings and consumer adoption trends, we see near-term XR use case opportunities in media and entertainment, fitness and recreation, gaming and education. This will be followed by mid-term opportunities in retail, productivity, infotainment, travel and hospitality, education, medical and healthcare. These use cases fall into eight major categories. Examples, including Accenture XR client work, are:

Product Design—XR provides tools through which consumer-facing brands can build virtual environments to create, test and optimize packaging, shelving and store designs before anything goes into physical production. Automotive companies, for instance, can design, create and visualize cars in VR.

Configuration—XR tools can be used by consumers to virtually try products before making a purchase. Skylight and window designer VELUX created MyDaylight, an XR experience that allows customers to explore the potential of VELUX roof windows when redesigning their homes. People can model their rooms in a few simple steps, using MyDaylight’s 360° light simulations to establish optimal lighting.

Commerce—XR is also being used as a direct sales channel. Thanks to services such as the Obsess AR commerce platform, brands can build entire 3D digital stores in augmented and virtual reality environments complete with e-commerce capabilities. Anything possible in a real-world bricks and mortar environment can now be replicated in virtual environments. Accenture helped clothing brand Diesel, and its parent company OTB, launch a 360-degree digital sales platform and showroom called Hyperoom, with an emphasis on creating a realistic and emotional connection with the buyer, making it possible to browse and use the interface to buy clothing, as if they were physically in the showroom.[iv]

Research & Analysis—Opportunities are ripe for brands to gather consumer data to provide individualized experiences and maximize sales XR-enabled research and analysis. One example is VR Merchandising, using mobile headsets with embedded eye-tracking, to provide more holistic data and a faster, lower-cost process compared to traditional merchandizing research. Recently, Accenture helped Kellogg’s employ such a solution to determine the placement, assortment and promotion of its new Pop Tarts Bites product. The VR testing revealed that when the Pop Tarts Bites were placed on a lower shelf—rather than higher up where consumers typically expect to find new products—testers paid more attention to other surrounding Pop Tarts products. That stimulated additional sales of Pop Tarts items, with an overall 18 percent brand sales increase during testing.

Marketing—XR is proving to be an excellent route to promote products through immersive experiences. Disney and Accenture Interactive unveiled a facial recognition photo booth to promote the studio's live-action remake of Dumbo, including an interactive movie poster that changed to reflect the expressions of people as they looked at the image.[v]Imagine a happy-face or silly-face elephant staring back as you walk by.

Pre-sale visualization and customization—XR can also help consumers imagine and configure what a product or service will be like before making the purchase. The event planning industry is a case in point. By extending the digital customer journey to include mobile, mobile AR, AR glasses and VR headsets, the Accenture’s XR Event Plannersolution allows hotel sales staff and event planners to visualize, customize and move through event spaces remotely and collaborate throughout the process. The approach can significantly improve the way hotels sell meeting space, while supporting sales of associated hotel rooms, and food and beverages.

Post-sales support and service—XR tools can be used to collaborate and connect with customers after a sale has been made by providing interactive user manuals, for example, or facilitating remote assistance. In these ways, XR is helping to extend the traditional digital consumer journey to create deeper, longer-lasting customer relationships while also gathering meaningful insights and leads for future business. Consumers will need to proactively decide which layers they want to see and when or they will rely on an AI-decision making tool to help determine what is needed. For example, looking at a printer could prompt the AI to bring up information on how to change a cartridge or order more ink.

New products and services—XR can help brands sell and engage in the future world of spatial computing by introducing experiences that do not exist today.

Implications for XR consumer apps in spatial computing

Security—In order to realize this future immersive world, companies must proceed responsibly and pay strict attention to designing and building secure XR devices and secure cloud platforms. For consumer XR solutions, brands will need to understand the cyber risks, especially when XR content is transferred over the cloud and AI recognition capabilities are on the cloud-as-a-service as well. These issues include location (secure stored personal location information); content masking (voice or gestural commands may be directed to the wrong object and generate a false activation); AI-trusted recognition (images transferred to a cloud service for object, voice or gestures recognition could be injected with synthetic information, which would confuse the response); content of images and environmental sound (source for data leakages, privacy and intellectual property). 

Since XR is a cloud-supported infrastructure, many of the same multi-tenant and shared environment security issues will be familiar to companies already managing security for their cloud environments, including identify verification and capabilities for monitoring, detecting and responding to XR-related cyberattacks. (For more information, see the Accenture Security Technology Vision 2020.)

To reinforce security for XR and other emerging technologies, Accenture is taking proactive actions now. Recently, for instance, Accenture Ventures announced strategic investments in Synadia to help provide digital systems, services and devices with highly scalable and secure communications technology;[vi] and in Quantexa, which provides a machine learning platform to help secure big data in financial services, insurance and government/public sector.[vii]

Privacy--When fully functional, XR devices will have multiple cameras and sensors with the capability to capture very private information. Think behavioral and perception data—such as pupil dilation, heart rate changes and facial expressions, or simply collecting images of your home. There are major ramifications around how the companies participating in this space are going to collect, store, use and manage this data. People will expect that gathered information is used purely to drive the experience, not to track physical motions throughout the day or deny privacy through a first-person view of events. Companies should also be prepared to build supplemental protection on the enterprise side to secure the XR application and associated data and to manage users’ personal data for GDPR compliance, data collection and data usage rights. 

Data Ownership--Particularly important to consumer brands, immersive experiences will be based on an individual’s data sharing preferences, as well as usage, location data and any other form of behavioral and perception data. In some cases, people may be able to choose the supplemental content they wish to see and interact with—or ignore entirely. It will be paramount for brands to gain permission to access individual consumers and then to build trust with them by delivering just the right amount of brand-based experiential content. Anything less and your company will miss valuable opportunities to convert sales and build loyalty; anything more and your brand could be labeled irrelevant or obtrusive. 

Steps to XR innovation now

Brands that want in on the action for the XR market and creators of the spatial computing world should consider the following: 

1.     Develop an XR business and technology strategy for where the consumer XR market will be within the larger spatial computing world. Determine what your company wants to control or participate in, and what will be required for the investment and execution plan. 

2.     Identify ecosystem partnerships to help carry out this strategy.

3.     Digitize your product library into 3D format to create your own XR commerce offerings

4.     Distribute your 3D assets for re-use across organizations in different functional areas, such as contextual advertisement campaigns or product design.

5.     Determine features and specs for XR activities. Conduct consumer surveys, compare identified needs and desires with your product features, create an execution plan and partner within the ecosystem to build the product and access the services needed to bring it to life.

6.     Develop a different understanding of your consumers based on their collected behavioral data, while protecting their privacy. 

7.     Decrease design and manufacturing costs, increase speed to market and deepen connections with consumers during the pre-sale cycle (providing immersion and configurability) and through interactive post-sale support.

8.     Be ready to triangulate XR offerings with a consumer’s location, what they are looking at, what app they are using, which digital map they are using and the behavioral data at the time of the offer.

The immersive future will not wait. Consumer brands must endeavor to understand the possibilities of XR technology and how it impacts their business model and consumer interaction strategies. Identifying the appropriate functional use cases and developing XR powered apps will enable your brand to deliver what consumers want—at scale.

Author

Raffaella Camera - Global Head of Innovation & Strategy, Accenture XR

[i] https://artillry.co/artillry-intelligence/mobile-ar-global-revenue-forecast-2019-2024/

[ii] https://newsroom.accenture.com/news/patients-want-to-continue-to-use-virtual-care-even-after-the-pandemic-ends-accenture-survey-finds.htm

[iii] https://www.linkedin.com/pulse/immersive-innovation-work-marc-carrel-billiard/

[iv] https://www.bizcommunity.com/Article/196/394/205555.html

[v] https://www.thedrum.com/news/2019/03/10/disneys-dumbo-and-accenture-interactive-collaborate-the-movie-poster-the-future

[vi] https://www.businesswire.com/news/home/20200709005108/en/Accenture-Strategic-Investment-Synadia-NATS.io-Open-Source#.XwcYVakFxKA.linkedin

[vii] https://www.businesswire.com/news/home/20200709005108/en/Accenture-Strategic-Investment-Synadia-NATS.io-Open-Source#.XwcYVakFxKA.linkedin