Consumers are increasingly shopping for products that aren’t physically in front of them. But the physical absence of products at the point of decision-making can hinder the customer experience, said Jonas Heller, former Research Student in the School of Marketing at UNSW Business School.

Mental imagery is a multi-sensory process that requires the representation of information in working memory, he explains. Consumers who are unable to generate mental images – for example, if they're unable to imagine a product in use at home while shopping online – could end up having a negative retail or service experience.

To overcome this challenge, companies such as Amazon, IKEA, L’Oréal, Nike, Akzo Nobel, Snapchat and Instagram are increasingly turning towards Augmented Reality (AR) to enhance frontline-customer interactions. But there has been little research into how exactly AR helps consumers offload their mental imagery processes onto this technology and whether AR can add tangibility to digitised service encounters, said Heller, who is currently an Assistant Professor at Maastricht University, based in the Netherlands. 

Last year, Heller received the Dean’s Award for Outstanding PhD Theses for his work on AR which seeks to bridge this gap. All chapters from his PhD dissertation Reality re-imagined: How augmented reality redefines decision processes and consumer behaviour in retailing, have been published in leading journals. 

Consumers choose products shown in AR vs 2D

Over several studies, Heller and his supervisors – UNSW Business School’s Associate Professor Mathew Chylinski and Adjunct Professor Ko de Ruyter – found consumers presented with holographic content through AR (holograms that require a display such as a phone or a tablet screen or AR glasses) chose the products presented in AR over those shown in traditional 2D printed images. Consumers were also willing to pay more for a product presented in AR than those that were 2D.

The research provides vital insights for managers who are interested in the type of AR application they can develop and implement at their retail-frontline.

Specifically, Heller’s paper Let me imagine that for you: Transforming the retail frontline through augmenting customer mental imagery ability showed AR applications must be interactive. For example, allowing consumers to inspect a product from multiple angles by rotating or moving it around or allowing customers to resize a hologram to fit into their real world.

“From a psychological perspective, we identified the reason that drives these behaviour changes are due to the fact that AR allows consumers to ‘offload’ part of their mental imagery processes onto the AR devices. Consumers feel more comfortable deciding on a product that they preview in AR (versus a traditional media such as a catalogue or a website),” said Heller.

Hand holding smart phone with AR application simulating a bedroom

Retailers such as Amazon, IKEA, L’Oréal, Nike, Akzo Nobel, Snapchat and Instagram are increasingly turning towards AR to enhance frontline-customer interactions. Photo: Shutterstock

Interactive products enhance customer experience

In the paper Touching the Untouchable: Exploring Multi-Sensory Augmented Reality in the Context of Online Retailing, Heller identified different forms of interaction with the same AR holograms could affect how consumers perceive the retailing experience. 

He also identified that retailers should offer hand gesture control over other forms of control (e.g. voice commands), as hand gesture controls most closely resemble how customers would interact with products if they were physically present.

“AR smart glasses allow consumers to control holograms in various ways, by using hand gestures, voice commands, or even gaze commands. While a voice-based search on computers and smartphones (think of “hey Siri”, “okay Google”, and “hey Cortana”) is on the rise, we found that for the case of AR, hand gesture control seems to be most effective for customers when it comes to assessing products,” he said.

In his third PhD project, Tangible Service Automation: Decomposing the Technology-Enabled Engagement Process (TEEP) for Augmented Reality, Heller investigated AR’s role in a world in which services become more and more digitised and thus less tangible for consumers.

“We wrote this manuscript a year before COVID-19, but it seems like it is more relevant than ever before given the ongoing pandemic. In particular, service automation removes physical aspects of services, which might result in reduced perceptions of service tangibility and adverse effects on customers’ service evaluations,” said Heller.

Overall, he said the use of AR can overcome issues with intangibility in automated services by bringing back a feeling of tangibility in the form of holograms.

“Even though holograms cannot be touched (in the end, they are pixels on a screen or lens), the feeling of customers is what matters. We show in a conceptual paper that AR allows for higher perceptions of value for customers, as well as an increase in engagement.”

Heller and his co-authors also found that consumers react differently to AR. Companies need to understand that different types of customers (e.g. consumers who are object vs. spatial visualisers, or consumers who are high or low in assessment orientation) will respond differently.

“For companies and retailers specifically, this means they need to learn more about their customers and offer an AR service to the right customers. AR is most beneficially employed for products that require visual assessment (e.g. makeup, beauty products, food) or products that require evaluation within a certain context, such as your own home (e.g. interior design, furniture, wall paint),” he said.

AR benefits business operations

AR technology also has enormous potential in the management of supply chains and retailers’ operations. For example, the employees of a supermarket can use AR glasses to restock shelves more efficiently, warehouse operations for online retailers can be improved, and last-mile delivery for logistics service providers offers plenty of opportunities for cost-reductions when using AR in various processes, explained Heller, who is currently exploring this area. 

“We currently live in a time where terms such as AR, VR, AI, chat- and voice-bots, robotics, and crypto are buzzwords that tech companies often use to convince retailers to buy their products. Retailers need to critically reflect on whether AR can add value to their customers”, said Heller.

Features like interactive and holographic face filters on Instagram, Snapchat, Tik Tok – these are no longer expensive. “You can hire freelance designers to design these, without any in-house expertise, at a very affordable price, even if you work with start-up budgets,” he said.

In sum, Heller said companies should “allow customers high customisation and interactive content to ensure visibility and customer engagement” to get the most out of AR technology.