Running a restaurant can be chaotic to say the least. While employees are busy churning out orders, managing inventory and keeping the drive-thru moving, it can be helpful to have an extra, impassive set of eyes.

That’s where some believe computer vision could have an impact. More restaurants are beginning to explore the use of AI-powered cameras to help them operate more efficiently, often by helping staff make decisions in real time. It comes as costs continue to rise and experienced employees become harder to hire and keep.

In January, hot dog chain Portillo’s said it was testing computer vision to help speed up the drive-thru. The idea is to take some pressure off employees, especially when restaurants are very busy. Cameras trained on the drive-thru gather data and distill it into a real-time dashboard that can prompt staff to take action. The screen flashes red when a car has been waiting for longer than four minutes without service, for instance, alerting a worker to address the issue.

“It allows them to see what’s happening in the drive-thru in real time, and make decisions or manage based on that,” said Sara Wirth, Portillo’s director of public relations, in an email. “This may, for example, include deploying another runner or order taker to the drive-thru.”

That is just one of the many possible applications of computer vision in restaurants. Others are using it to help prevent theft or fraud, tell employees when to add more burger patties to the grill, or track inventory so that staff know when it’s time to restock a station.

However, restaurants are still in the early stages of adoption, and a dominant use case for computer vision has not emerged yet, said Adam Dumey, global vice president of retail at tech services company World Wide Technology. “You go from, ‘Does this technology work?’ to ‘Is there value in this technology?’” Dumey said. “What we’re seeing right now in QSR is more the proof of concept part of it.”

In its most basic form, computer vision uses cameras to identify patterns in the physical world and translate those patterns into usable data. That makes the possibilities for the technology almost endless.

Computer vision supplier Wobot, for instance, has developed more than 100 use cases for its software, each geared toward a specific business goal. These include identifying when a customer is waiting without an employee present, monitoring whether kitchen workers are wearing hairnets and gloves, and tracking employees’ on-the-clock cellphone usage. Almost anything a camera can capture can be tracked, said Will Kelso,
Wobot’s president of revenue and growth.

Most restaurants already have security cameras in place that can record these behaviors. But they don’t have the time or resources to analyze that footage and pick out the things they’re looking for, which is where Wobot’s software can help.

“It would take someone reviewing hours and hours’ worth of video to be able to capture what we’re formatting in a consumable way,” Kelso said. The question then becomes what operators do with the information they’re collecting. In the case of employee cellphone use, for instance, that data could be used to issue violations or to identify repeat abusers. Or it could be used to reward employees who stay off their phones while working. Even more use cases are unlocked when computer vision is integrated with the restaurant’s POS system. That allows the software to make decisions based on things like demand and staffing levels.

That is more or less the goal of PreciTaste, a company that offers a number of AI tools for restaurants. One of them, the Station Assistant, analyzes past sales data to predict demand while also using cameras to monitor inventory levels. This allows it to alert employees when an item needs to be restocked.

A similar product, the Planner Assistant, predicts demand throughout the day and tells employees when to start prepping certain items, like burgers, before they’ve even been ordered. This helps ensure that food is being delivered as fresh as possible.

The company says these tools can help restaurants reduce waste by up to 50% because kitchens are producing only as much as they will sell. They also cite higher revenue and customer satisfaction because food is served faster and fresher.

“Leveraging PreciTaste’s vision AI technology has given us insight and data into how we’re upholding our quality standards with cooked product hold time,” said the director of operations at a large burger chain in a statement. “Without this information, we’ve been completely in the dark about executing our promise of fresh food.”

PreciTaste’s vision tools are in use at thousands of restaurant locations, but its customers are under non-disclosure agreements because they are large chains that are in various stages of rolling out the technology. Demand for PreciTaste has been “stellar,” said VP of Operations Hauke Feddersen, so much so that the company has had to hire more employees to keep up with the various projects the company is working on.

Part of what is paving the way for more computer vision adoption is a desire among restaurants to operate more efficiently. In some cases, computer vision can help ease costly problems, like drive-thru pileups or food waste, or help automate time-consuming tasks, like stock keeping. But it’s just one possible solution.

“Not every customer is asking us for [computer vision],” Dumey of World Wide Technology stressed. “They’re more asking us how to solve a problem.” At the same time, computer vision has become a more realistic option for restaurants because it is becoming less expensive. Computer vision still requires a lot of horsepower to train and operate, said Feddersen, but that power has become a lot more affordable. “In the past, we
had racks of servers for something that can be done on a very small form factor PC right now in a restaurant,” he said.

Per location, PreciTaste’s vision system costs about $2,000 to install and then $399 a month after that. The cameras needed to operate computer vision have also come down in price from several thousands of dollars five years ago to a couple hundred bucks today, Dumey said. “The cost has absolutely gone down,” he said. 

And it’s no longer necessary to have a certain kind of camera to support computer vision. Wobot’s software, for instance, can integrate with a restaurant’s existing security cameras. The technology itself has also improved, which has added value and contributed to lower costs. When AI supplier Presto Automation was offering a drive-thru vision product two and a half years ago, for instance, it would install as many as eight cameras to capture all of the data it needed. Now, CEO Gee Lefevre said, the same information can be collected with a single camera thanks to advances in AI that allow it to reach conclusions with fewer
data points.

“Computer vision has probably become multiples cheaper as the result of the need for fewer cameras to achieve the same level of output,” he said. Presto has since dropped its vision product to focus solely on AI voice, which was growing faster at the time. But as computer vision improves, “We expect that to become more interesting very quickly,” Lefevre said. Still, a number of barriers remain to widespread adoption of computer vision in restaurants. 

As Dumey pointed out, the technology is most effective when it is integrated with systems that house a restaurant’s sales, labor, inventory and other data. Without that, computer vision is just another point solution operating in a vacuum. “This is the challenge that QSRs have,” Dumey said. “It’s inundated, like many retail segments, with stovepipe and siloed systems.”

Restaurants that want to implement computer vision need to have an integrated foundation in place first, he said. They also need to have a plan for the reams of data that computer vision generates. And it’s not only a matter of how to use the data, but also how to store and protect it, especially when it contains employees’ or customers’ personal information.

“That costs money and it increases your risk,” Dumey said. “So, is the value [of the data] worth the commensurate administrative burden?” On that note, the sources who participated in this story emphasized that computer vision is not to be confused with surveillance. The technology does not store personally identifiable information without customers’ or employees’ consent. In most cases, customers’ faces and other identifiable traits like license plates are blurred out. For systems that track employees, workers must be made fully aware what will be taking place.

Kelso of Wobot said the company has seen little pushback from restaurant employees over this new, high-tech method of observation. At the same time, he noted that restaurants are not necessarily all that eager to talk about it publicly. “I think sometimes, when it gets into the publicity side of it, it becomes … ‘Oh, could it be misinterpreted as we’re tracking things that we’re not even tracking?’” he said.

Despite those hurdles, experts do see computer vision continuing to grow in restaurants over the next several years. Restaurants’ tech strategies will become more sophisticated and integrated, they said, and artificial intelligence will be increasingly viewed as a must-have. “You’ll see it much more as part of the evaluation process, especially with large entities,” Kelso said. “It’s something that with the AI wave is just more
of the expectation now.”

Lefevre of Presto believes that computer vision will play a key role in the drive-thru of the future, along with voice AI and kitchen robotics. Cameras will not only be able to optimize the movement of cars through the line, but will also be used to scan license plates that are tied into loyalty programs, which will enable personalized offers and upselling at the menu board. That choreography will require some cooperation and perhaps some consolidation among the different players in the AI and robotics market.  “I actually don’t think it will be that far in the future before those systems begin to integrate in a way that would form the drive-thru of the future,” he said, adding that Presto sees itself as firmly in the mix. 

Feddersen of PreciTaste also sees computer vision as a central piece of a broader shift in restaurant tech. He believes that AI-powered forecasting will eventually replace the POS as the restaurant’s central hub. That forecast will tell the operator how much product to order that week and how many employees to schedule for a given shift. And cameras will have a big role in that ecosystem.

Dumey agreed that computer vision will likely have a hand in forecasting, specifically on inventory. He also noted that it could be applied to restaurant design and layout as chains expand or renovate locations in the coming years. “It’s going to be much more pronounced,” he said.

If you’re a subscriber, you can see the full article here in Restaurant Business.