Case Study

Automated Airplane Parking System

Key details

Automated Airplane Parking System

Challenge
The challenging part of the project was to build a robust system that can reliably detect the plane in all weathering conditions as well as all types of daylight conditions. Further, as the pilot needs real-time updates of how close the plane is to the parking line, the estimation of plane position should be done in real-time.
Approach
We developed an object detection and tracking model using tensorflow that was trained for airplanes. We carefully collected data from the videos collected by the airport and tried to include all possible variations of planes, lightning and weather conditions. We took a data-centric approach to build the model and iterated over the dataset a couple of times to get the most out of the data. We also used data augmentations like zoom-in/zoom-out, brightness changes, half-visible planes. The augmentations helped the model in learning different parts of the plane and was able to detect the plane even if only the nose was visible.
Goal
The goal of the project was to guide the plane to the parking line as soon as it enters camera view. The allowed tolerance in parking error was +-200cm for an average plane length of 4-6m, so the parking accuracy required was above 97%.
Outcome
This system is able to reliably track planes at speeds of 25 FPS or higher. It's equally skilled at tracking narrow and wide body planes, and it works well in any conditions – day or night with glares on the camera, with or without weather issues like snow or rain. The system once had to track a military plane with 18 wheels (a plane that it had not seen during training or wasn't tested as it was an unexpected occasion), and it still managed to successfully get it parked.

Automated Airplane Parking System

We developed an object detection and tracking model using tensorflow that was trained for airplanes. We carefully collected data from the videos collected by the airport and tried to include all possible variations of planes, lightning and weather conditions. We took a data-centric approach to build the model and iterated over the dataset a couple of times to get the most out of the data. We also used data augmentations like zoom-in/zoom-out, brightness changes, half-visible planes. The augmentations helped the model in learning different parts of the plane and was able to detect the plane even if only the nose was visible.

As a result, as soon as the plane entered the camera's frame, the model would immediately detect it and start tracking. We also developed a fallback system, to handle cases when the detection fails or there are frames where the plane is not completely visible or trackable.

For testing the model, we generated simulated videos and tested the model before taking it to production. Once it was tested and found ok for all possible edge cases, we deployed the model on GPU. We optimized the model using tensorrt to get the fastest inference time.

Technologies