Back to our work
Computer vision

Checking shelves against planograms with graph matching

As part of the client’s team, we built the 2D matching engine and backend. Cutting matching time for 80 products on one side of a fixture from 53 to 3-4 seconds made photo-based planogram checks practical on the shop floor.

Period
Nov 2019 - May 2020
Our scope
Compliance engine & backend
Engagement
Delivered by the aiLabs founders
Matching product positions against a planogram The expected layout and the products recognised in a photo are compared to identify missing items and other differences. The case measures matching for 80 products on one fixture side. Planogram Recognised in photo Missing item 2D matching · 80 products on one side
Explanatory diagram illustrating the engineering approach.

Key results

  • We reduced the time required to match 80 products on one side of a retail display from 53 seconds to 3-4 seconds by tuning the algorithm.
  • Works on any 2D fixture, not only shelves: products can be placed freely.
  • Every product is reported as in place, wrong, missing or extra, and the position of a missing product is restored.
  • Handed over to the client with all development agreements fulfilled.

The challenge

Once products in a photo are recognised, the system still has to decide which of them corresponds to which planogram position. Products can be missing, extra or misplaced, the photo can be skewed, and a fixture is not always a simple set of shelves. Matching alone initially took almost a minute for 80 products on one side of a fixture. Waiting that long after taking a photo would slow merchandisers down when checking multiple displays and call the value of the product into question.

Our contribution

  • Formulated compliance checking as graph matching between the planogram and the photographed fixture, moving from sequence alignment (Needleman-Wunsch) on single shelves to Reweighted Random Walks (RRWM) for free 2D layouts.
  • Implemented the matcher in C++: an affinity matrix of positions and edges, reweighted random walks with Sinkhorn bistochastic normalisation, and discretisation with our own Hungarian algorithm covered by 205 automated tests.
  • Turned the matching into merchandiser actions: matched, wrong product, missing with its position and size restored, and extra products.
  • Built the Go microservices around it: planogram and photo processing, compliance checking, storage and APIs for the mobile and desktop apps, with PostgreSQL and a Redis cluster.
  • Developed the desktop Planogram Creator in C++ and Qt, including recreating a planogram from a photo of a real fixture.
  • Prepared the infrastructure on AWS with Ansible and Docker.

What the work delivered

Faster matching made photo-based checks practical during routine store visits. Merchandisers could take a photo and get instructions on what to fix while working at the fixture. The team handed over the completed compliance engine and backend to the client.

More engineering stories

View all work
Platform engineering

Platform modernisation

Cloud-independent infrastructure, event-driven services and a faster, cheaper data layer for an established learning analytics platform.

Computer vision

Computer vision platform

A computer vision platform that takes a project from data storage and annotation through training to live inference. Our team built two applications to demonstrate the full workflow.

Applied ML

Predictive learning analytics

A service that trains a model on each institution’s data, explains its predictions and analyses text, topics and sentiment. It launched in 2021 and was extended in 2025-2026.

Let’s build something useful

Bring us the
hard part.

A new AI capability, a complex data problem or a platform that needs to evolve. Tell us what you’re working on.

Discuss your project

Start a conversation

info@ai-labs.tech

aiLabs Innovations FZ-LLC
Compass Building, Al Shohada Road, Al Hamra Industrial Zone-FZ, Ras Al Khaimah
United Arab Emirates
Licence No. 47014246

For an initial conversation, share your goal, the current system and the challenge you want to solve.