Scancam uses ML.NET to prevent fuel theft

Scancam is a loss prevention company. One of their main products is an anti-fuel theft solution which caters to fuel retailers and provides an end-to-end solution for preventing and managing fuel theft, including debt recovery.

Business problem

Fuel theft costs the Australian fuel retailing sector millions of dollars every year. At fuel stations in Australia, customers fill up their fuel tanks first and pay afterwards. Some motorists drive off without paying for fuel, and some motorists realize they forgot their wallet or don't have enough money to pay for the fuel they've just dispensed. These drive-offs and failure to pay result in thousands of dollars of losses for fuel retailers.

Scancam's solution uses license plate recognition technology to detect vehicles at fuel station pumps and provides alerts for known offenders (people who owe money from any of the fuel stations in their network). It also includes a facility to report fuel thefts, automatic incident footage generation, online payments, and debt recovery.

Why ML.NET?

Scancam is primarily a .NET shop. Prior to ML.NET, the company ran object detection on a separate Docker container running Python code. When Custom Vision added exporting of object detection models, Scancam ported to ML.NET, which allowed the company to host machine learning components in the same codebase as their applications.

Scancam runs C# code from the smallest edge devices onsite right up to the cloud. They can use the same familiar C# language and tooling on any component in their solutions and can easily move code from the cloud to edge and vice versa (for example, from Azure Functions to ASP.NET Core container images running on ARM32 devices).

Impact of ML.NET

ML.NET gives Scancam the flexibility to easily move where their code executes, that is, on edge devices, mobile devices, or the cloud. The resulting ML.NET object detection functionality dropped Scancam's bandwidth requirements by up to 35% and consequently reduced their cloud compute requirements, while the anomaly detection components will proactively monitor hundreds of devices.

ML.NET allowed us to increase productivity by allowing us to code our ML components in the same language and tooling we use for everything else. ML.NET provided the easiest jumping point for our .NET developers to get started integrating machine learning to our applications."

Given the limited window of time available from the moment a vehicle is detected to when the motorist starts pumping fuel (5-7 seconds), ML.NET allowed Scancam to add object detection to their processing pipeline with hardly any impact on performance with predictions taking on average ~300 ms.

Solution architecture

Scancam's technical stack is comprised of ML.NET, ASP.NET Core, Xamarin on iOS, and Azure (App Service, Azure Functions, Azure Service Bus, Azure Storage, Azure Cosmos DB, SignalR, Azure SQL Database).

When a vehicle drives to a fuel pump, a camera triggers an event to an HTTP endpoint running on ASP.NET Core on Docker. ML.NET is then used to first detect the presence of vehicles and then detect the presence of license plates on those vehicles. They then use a separate license plate recognition technology to scan the license plate numbers and letters and hand it over to an Azure function to trigger the remaining Azure-based cloud processing pipeline, including cross checking the license plate with a database of known offenders and broadcasting detected license plates to iPad and TV displays via SignalR.

Client application

The main end-user application is a Xamarin app that is used by fuel station attendants on an iPad. This app displays all license plates detected on each pump and shows an alert if a license plate is in the known offender database. This allows the console operator to prevent dispensing of fuel and to ask the customer to prepay for fuel before enabling the pump.

Object detection

Most of the cameras that Scancam uses rely on motion detection to detect vehicles moving in and out of the fuel pump areas. They get hundreds of thousands of events from hundreds of cameras every day. Given how busy fuel stations are, Scancam can get tons of false triggers. These false triggers result in unwanted events that need to be processed for license plate recognition, which in turn consumes a lot of the limited available internet bandwidth at the fuel stations.

In order to reduce the number of events that needed to be sent to the cloud for processing, Scancam used Azure Custom Vision to train an object detection model for detecting the presence of a vehicle and the visibility of the license plate region in an image and exported the model in ONNX format. They then consume the model with ML.NET on edge devices to preprocess the camera events before sending to the cloud for further processing.

Anomaly detection

Scancam is also implementing ML.NET for anomaly detection to detect spikes (for example, incorrect motion trigger areas, trigger threshold too low) and dips (for example, camera has been knocked out of position, blocked, incorrect exposure settings) in camera events as part of monitoring the health of camera installations. This detection allows Scancam to easily pinpoint potential issues with their hundreds of cameras without having to manually search through logs.

Image classification

Scancam is also planning on adding vehicle make and model identification to their ML.NET solution.