Five years ago Amazon started working on a secret project: how to eliminate the checkout line in stores. Amazon executives reasoned that most other aspects of physical shopping had been pretty much perfected, except for one thing - nobody likes waiting in line.
The result of that project is Amazon Go, a futuristic convenience store in which shoppers are tracked by hundreds of cameras on the ceiling and a computer algorithm that analyses their every gesture, and then tallies up their receipt when they exit. Amazon calls this “just walk out” shopping, because there is no checkout counter and no checkout line, just a few turnstiles like those on the underground.
Opening today
0 of 3
The store, which has been in testing since December 2016 and opens to the public on Monday, represents Amazon’s most provocative effort yet to reshape the future of brick-and-mortar retail.
Located in a ground-floor retail spot at the centre of Amazon's campus in Seattle, the shop has posters in its windows advertising the "just walk out" shopping experience. To underscore the point, each receipt comes with a "trip timer" so shoppers see exactly how many seconds it took to grab their goods. The items for sale are similar to what might be found in a deli or a corner store, with an in-store kitchen that prepares fresh sandwiches and salads.
Although it was founded as an online-only bookstore, Amazon’s aggressive push into physical retail has surprised - and alarmed - many of its retail competitors. The company now operates 13 brick-and-mortar bookstores, several grocery pick-up points and, since its $13.7bn acquisition last year, hundreds of Whole Foods stores across the US.
Still experimenting
However the complexity of the Amazon Go store’s design means that it is more of an experimental concept at the moment, rather than a mature technology that can be easily and cheaply replicated. The store was initially supposed to open to the public in the spring of 2017 but was delayed until this month amid reports it was not quite working as expected.
Five years ago we said: can we push the boundaries of computer vision and machine learning to create this effortless experience for customers to come in, take what they want, and leave? - Dilip Kumar, Amazon
Dilip Kumar, who oversees the technology behind the Amazon Go store, explains that the store uses computer vision - the ceiling is dotted with hundred of video cameras - to determine what shoppers are picking out.
“Five years ago when we started this, we said: can we push the boundaries of computer vision and machine learning to create this effortless experience for customers to come in, take what they want, and leave?” he says.
Under surveilliance
Mr Kumar points to the cameras that nearly blanket the ceiling, and explains that the computer algorithm uses these to determine which customer is taking which products. There are also weight sensors on the store shelves, but these are less useful because different items can have the same weight, such as different flavours of yoghurt.
“The holy grail is video understanding,” he says. “To be able to understand and interpret and know exactly what is happening. Doing this at scale and getting transaction-level accuracy is what makes this challenging.”To help facilitate identification, certain items in the store such as sandwiches also have a special dot code on top, similar to a bar code but designed with circles and diamonds that make it easier for a camera to read from a distance
He explains that one of the most difficult things for the algorithm to master is when the store gets crowded. “It is much more convoluted when you have 50 people picking up multiple items or browsing,” he says. “They are occluded, they occlude each other, items are getting occluded, and the items are small.”
Over the past year, the tech team has been improving the algorithm so that it is trained to handle these scenarios, including when customers might partially cover an item with their hand when they take it from the shelf.
$800 billion - size of the US grocery business
To help facilitate identification, certain items in the store such as sandwiches also have a special dot code on top, similar to a bar code but designed with circles and diamonds that make it easier for a camera to read from a distance.
While Mr Kumar vowed that the technology was “very, very accurate”, he declined to specify when the company might open other Amazon Go stores. “We’d love to build more,” he said, without elaborating.
Future plans
Mr Kumar also said the company has “no plans” to introduce its checkout-free shopping in any Whole Foods stores, nor in the Amazon bookstores.
Nevertheless, in the year since Amazon first unveiled plans to reinvent the grocery store with Amazon Go, rivals have raced to invest in similar technologies as they battle for market share of the $800bn US grocery business.
Plans have ratcheted up since Amazon acquired Whole Foods last summer. In the ensuing months bankers have worked closely with their brick-and-mortar clients to help them come up with a tech-centric defence to the threat.
Some of those moves are already materialising as Kroger, the largest standalone US grocery chain, is rolling out its own take on the cashier-free experience this year. Visitors to a Kroger supermarket will be able to scan the barcodes of food on their smartphone as they walk through the aisles, and pay at a self-checkout machine upon leaving the store. The company says it will roll this technology out to 400 of its more than 2,700 stores in 2018.
Walmart, which derives more than half of its nearly $500bn annual sales from food, is testing a grocery shopping experience without cashiers in places including Dallas and Orlando. The company says that by the end of this month, 100 of its supermarkets will allow customers to buy groceries through a smartphone app - only interacting with a human when they show their digital receipt to an employee before walking out the door.
Rivals
Doug McMillon, chief executive of Walmart, has referred to himself as a “gadget-guy” and promised to mould Walmart into more of a tech company since he took the helm in 2014. At a conference last week, he boiled down the challenge, and the impetus, for retailers’ race to cut the queues.
Amazon Go: a new kind of store
Scan to enter: Shoppers enter the store using a smartphone app to scan a code that is linked to their Amazon account. If a family is shopping together, they can all scan in on the same code.
Pick up what you want: Merchandise is spaced out along shelves that have weight sensors and plastic dividers to keep the items distinct. Some, like the sandwich pictured above, have a dot code for easy camera identification.
You are being watched: Hundreds of cameras hang from the ceiling of the store, constantly monitoring shoppers' actions and recording which items a shopper selects.
Just walk out: When finished, shoppers simply exit the store. There is no need for them to use their smartphone again or scan anything. A receipt for their purchases will appear in the app.
– Copyright The Financial Times Limited 2018
