AI / Apple · December 10, 2021

Apple is building a shopping assistant for you

Online shopping has spoiled us. You go to a physical store and you feel helpless.

When shopping online, you have a lot of info to work with. Customer ratings, reviews, easy product comparisons help you decide what to buy and what to avoid.

Physical stores can never provide such buying assistance . . . or can they?

Well, a patent application from Apple shows they can. Relying on object recognition technology, here is how Apple is planning to provide all the information to you need to make the right buying decision in a seamless manner.

As per the patent, you will enter a store with your Apple smart glasses on. You will reach to one of the shelves and pick a product that interests you.

Using computer vision, your smart glasses will identify the product you picked and pull its details from the cloud. For the sake of an example, let us say it’s a cereal box.

Now, as soon as this item is in front of you, you would see a “virtual label” hovering over the product, with all important details about it. Here is how it might look:

Drawing from Apple’s US Patent Application No. 2020/334,731

But that’s just the basic info. You could just as well read it from the label on the box, right? True. Good thing the story doesn’t end here:

Depending on your preferences (about which we will say more in a moment) you will also see more personalised ratings about the product. And here comes the shopping companion part. It might look something like the image below. Another virtual label hovering on top of the box has three rings, each showing how pricey the product is, how eco-friendly it is, and where it stands with respect to customer reviews.

Drawing from Apple’s US Patent Application No. 2020/334,731

Remember I said these will be personalised ratings and will depend on your preferences? Here is what I meant. You will have an option to set how much emphasis you give to things like price, eco-friendliness and customer opinions.

This means, for example, if you care a lot about climate change and want to limit your environment impact, products that create more waste will look unfavourable in the ratings you see. Your settings screen for specifying these preferences might look like this:

Drawing from Apple’s US Patent Application No. 2020/334,731

Notice an important detail in the above display. Soy is highlighted in “avoidances” — so you would be able to specify things you are allergic to. If you then come across a product that has something you avoid, you would see a clear warning sign, which I think is great!

Drawing from Apple’s US Patent Application No. 2020/334,731

Now comes the feature which I think is just epic because it will really help you in situation where you are in a buying dilemma.

If you pick two products of the same category, say two medicines for cold, and bring them physically next to each other, your device will sense that you are comparing these items and immediately show you a comparison report.

In my experience, doing such comparisons on shopping websites is useful but still feels clunky because you have to select those products separately. Here it happens almost naturally. And that’s the best thing about AR in my opinion — it obviates the need of carrying out so many explicit actions to do something and just “fits” into your day-to-day life with such elegance.

Drawing from Apple’s US Patent Application No. 2020/334,731

Do you think such a tech could bring back some of the lost charm of in-store shopping? Or will it be sidelined by something more elaborate, such as completely virtual reality based stores? Let us know your thoughts in the comments!

Liked this article? Do subscribe to our weekly newsletter to received a weekly digest of similar stuff.