MBO Smart Buffet: edge computer vision to measure what is eaten and what is thrown away at a buffet

The client

MBO is an Andalusian startup that builds products for hospitality: technology that measures what is served, what is consumed and what is thrown away at hotel buffets and restaurants.

DPI Systems is the technical team behind MBO. We handle the complete development: selecting and installing the hardware and sensors, the computer vision models, the software that runs in the hotel, the cloud platform, the fieldwork and the testing with the buffet open to the public.

Vision at the edge

The video from each camera is decoded by the Jetson's dedicated hardware using NVIDIA DeepStream, without using CPU. One frame from each camera is grouped into a single batch and passed through a YOLO11 detector compiled with TensorRT in FP16 for the unit's specific GPU: a single inference for all the cameras.

Then several processes in parallel track each tray, person or utensil between frames with ByteTrack and apply the business rules that turn what is seen into events: a tray that goes out or comes back, a customer serving themselves, a bottle being poured, a tray being replaced, a person coming in or going out.

The same node covers five use cases depending on where the camera is placed:

• Pass: overhead camera and scale; records trays going out and coming back.
• Consumption counter: camera above the buffet; trays on display, servings, pours and tray changes.
• People counter: camera at the entrance; diners coming in and going out.
• Show cooking: servings of a fixed dish at a live cooking station.
• Waste weighing: scale only; kitchen and diner waste.

The pass, the center of measurement

All the buffet data starts at the pass. Each tray is weighed when it leaves the kitchen and when it comes back: that difference is the real consumption, what comes back is the waste and, with the recipe card, the cost. The dining room cameras add the detail of what happens in between, but the weight at the pass is the reference for everything.

That is why the pass comes in two versions. In installations with a Jetson, an overhead camera and a network scale record each tray. And for kitchens that need a dedicated station we developed the pass kiosk: a compact PC with Ubuntu, overhead camera, serial-port scale and touchscreen. The cook sets the tray down; the kiosk weighs it, photographs it, calculates its visual fingerprint and suggests the dish. If in doubt, the cook chooses between the candidates and the tray is recorded. In "back to kitchen" mode, the same station records the returns.

The kiosk's AI runs on CPU with OpenVINO, Intel's equivalent of TensorRT, and shares all the common logic with the Jetson: any improvement reaches both.

Recognizing the dish without retraining

The detector only learns what a tray is, a pair of tongs, a hand or a guest: 9 classes that do not change with the menu. To know which dish it is, Meta's DINOv3 model turns the photo of the tray into a fingerprint of 384 numbers that the backend searches for in Qdrant, with a similarity threshold configurable per hotel. If it does not recognize it, it records it as unknown to be labeled later.

When a tray comes back, the backend works out which of the trays that went out it is: it discards those that do not match by weight, compares by visual similarity and calculates consumed = served − returned. Adding a new dish means photographing it, not retraining.

Designed never to miss a service

A hotel is not a laboratory: the power goes out, the router restarts, the Internet goes down. The architecture starts from that reality:

• Outgoing connections only: each node initiates its calls over HTTPS with its own key. The cloud never enters the hotel network.
• Star topology, no hierarchy: the Jetson and the kiosk do not talk to each other. A node going down does not affect the others, and adding another one means registering it in the dashboard.
• Nothing is lost offline: each event is written to disk before it is sent, with a unique key and its real time. The queue holds 20,000 events or 7 days; an outage only delays the data.
• The cloud decides, the edge observes: the edge detects and measures; dish identification, matching and metrics are calculated in the backend, so the logic evolves without redeploying the hotels.
• When in doubt, capture: if the shift schedule is not known, the cameras stay on; if a tray that comes back finds no match, it is saved anyway.

The platform

MBO Platform is multi-company: each hotel has its own space, with administrator, manager and user roles, and subscribes to the buffet service, the restaurant service or both. The configuration of cameras, scales, use cases and shifts lives in the cloud and is changed from the dashboard, without touching the units. It includes a live kitchen view, a dish catalog with recipe cards, analytics on consumption, cost, waste, occupancy and peak hours, and a test environment to validate every change before production.

Fieldwork and testing

A vision system only works if it has been tested on the real site. We installed cameras and scales with the hotel in operation, set up the Jetson units and gave them secure remote access so they can be maintained without traveling.

• Real data: 5,033 labeled images from real buffets, from 23 cameras.
• On-site recordings: we record complete services and replay them in the lab to test every change against the same breakfast, frame by frame.
• Emulators: simulated cameras, scales and backend to test the complete system without occupying the hotel.
• Validation with the kitchen: buffet parameters and recipe cards agreed with the hotel team, and consumption checked against the real weighings.

From MVP to product

The first version, an MVP with a buffet monitoring dashboard, was built in six weeks. Today the product measures the complete cycle of each tray with cameras and scales, runs the vision on NVIDIA Jetson units inside the hotel and is in production at a real breakfast buffet.

The next step is to take MBO Smart Buffet to more hotels with an ever faster installation: the same equipment, the same software and a dish catalog that adapts to each kitchen.