A new twist on augmented reality (AR) packaging puts an intelligent holographic assistant on the package. Shoppers use their mobile phone to activate the hologram at the point of sale to see and hear the assistant, who appears to be inside the package.

Currently in prototype, the technology is from Melbourne, Australia-basedstart-upImmertia. This prototype completes the company’s second stage of development in linking an intelligent assistant to consumer goods packaging. In 2020, Immertia demonstrated the first prototype, which featured an animated voice assistant that consumers could talk with.

The first prototype of Immertia's AR technology, circa 2020, uses the package to link to an “intelligent voice assistant” — an animated character — with which consumers can converse using their voice.

In the second prototype, the holographic assistant speaks in the consumer’s choice of English or Spanish. She provides information such as product ingredients and benefits, presents images of other products, and plays short commercials and music. The consumer navigates through the content using clickable links that appear to hover within the hologram.

Content is streamed from the brand owner in real time, assuring that consumers receive the most recent information.

Immertia is currently developing a version of the technology that combines features of the two prototypes. The result will be an AR human concierge who will provide product information and with whom consumers will be able to converse, asking questions using spoken language and receiving voice responses.

The company expects to launch its first hologram product in 2022 and anticipates launching the conversational version in the second half of 2023.

Dave Chaffey, managing director of Immertia, answers Packaging Digest’s questions about the technology in this exclusive Q&A.

Do you have a customer for either of the prototypes yet?

Chaffey: We presented a prototype concept to Bayer Pharmaceuticals this year. That was the first time we unveiled the concept. Since, we’ve been contacted by several big pharma companies and fast-moving consumer goods (FMCG) manufacturers. It wouldn’t be appropriate to expand upon that at this stage.

When the time is right, we’ll be keen to involve companies for the first round of in-field trials of the technology.

The current, second-stage prototype, which was created for Bayer, appears in this short YouTube video. Click on the "Bayer Pharmaceuticals" link above to see the full-length demonstration video of the Bayer prototype.

Why would brands prefer a holographic, on-pack assistant to a human salesperson at the point of sale?

Chaffey: First, availability. We see the concept working alongside existing channels. Sales teams can be very powerful, but they’re up against it, trying to cater for everyone. And I guess that’s the big advantage here — accessibility. There’s a lot of value for brands knowing that they’re providing the answers to questions from potential customers in situations where there is no salesperson to help.

Second, reach. The prototype is based on label-recognition technology rather than quick response (QR) codes. It recognizes labels by sight, the same way you or I would, which fires up the hologram. That has big implications. When a label is added onto our virtual network, the hologram concept becomes live from that product, as well as from every identical product, globally. That system is already in use on our other, non-hologram AR platforms operating in the beverage industry.

Third, connection. It offers brands the opportunity to build deeper connections from every product. For brand reach and engagement, the potential is enormous, forming meaningful connections from products everywhere — in homes, supermarkets, warehouses.

Fourth, truth. There’s also a very powerful opportunity here for brands to look at packaging as a dynamic tool. For example, with the concept, the information that is provided to consumers comes from the manufacturer in real time and can be changed when needed. It takes a few minutes to flow through to the virtual network, and then bang! Every identical product package across the globe is updated with the new information.

We’ve all had that mobile search experience, wading through Google search results and endless pages with out-of-date information. Imagine wanting to know about related products or trending items and knowing the information is coming from the manufacturer, in real time. There are so many use cases, we haven’t stepped them all out yet. It becomes a powerful source of truth.

How would a brand owner indicate, on the package, that consumers can use their phone to access this holographic information?

Chaffey: That an interesting challenge that we’ve had to overcome for other concepts. It comes down to three things:

One, visibility of marketing collateral at the point of purchase;
Two, information and educational materials;
Three, a viral aspect — that is, a means for consumers to quickly share on social media.

It’s a challenge on day one, no doubt. Once there’s some interest, it can move quickly. There’s a lot of “social media cred” with new technology. Consumers are motivated to share, particularly if they’re the first to share. Social media will be a big factor in adoption of this concept. We’re factoring that in from day one.