AI-Based Embedded Platform Design
The AMD Advanced Surveillance Platform developed by Empa Elektronik presents a comprehensive demo showcasing a novel and versatile approach to AI-powered object tracking and two-axis motor control, implemented through the integrated operation of three AMD Kria boards (2× KV260 + 1× KD240).
The platform is built on a distributed system architecture consisting of three main components. The Image Processing Unit performs AI-based object detection and provides RTSP streaming, while the Handheld Terminal delivers a user interface with hardware-accelerated H.264/H.265 codec support. The Motor Control Unit is responsible for precise two-axis camera control.
The demo demonstrates the platform’s capabilities in real-time object detection, AI-assisted automatic tracking, modular algorithm replacement, manual control mode, and H.264/H.265 video codec performance. The platform addresses a wide range of applications, including security systems, traffic management, retail analytics, industrial automation, robotic systems, and military and defense solutions.
All source code for the project will be released as open source on GitHub, with the aim of showcasing the full potential of the AMD Kria platform and contributing to the advancement of edge AI applications. This Proof of Concept (PoC) demo highlights key advantages of AMD Kria boards, including FPGA acceleration, low power consumption, flexibility, scalability, and hardware-accelerated video codec support.
Designed to validate the potential of the AMD Kria platform in edge AI and robotic control applications, this project serves as a reference implementation for developing similar solutions across various industries.
AI Components of the Advanced Surveillance Platform
The AI components of this original application developed by Empa Elektronik were implemented using AMD’s AI development toolchain. For the advanced object tracking task, two distinct processing stages were designed: determining the two-axis spatial positions of objects within the visual frame and re-identifying detected objects across subsequent frames.
In the first stage, the DenseBox deep learning model architecture was used for detecting objects—specifically human faces—for tracking purposes. This pre-trained model, which provides facial bounding box annotations, was processed using the AMDVitis™ AI toolchain to run on the embedded Deep Learning Processing Unit (DPU) within the AMD Zynq UltraScale+ MPSoC architecture.
Source: DenseBox: Unifying Landmark Localization with End-to-End Object Detection (Huang et al., 2015)
During the processing flow, the pre-trained model originally in 32-bit floating-point (FP32) precision was first quantized to 8-bit integer (INT8) using the Vitis AI Quantizer, and then compiled into the DPU instruction set using the Vitis AI Compiler, optimized specifically for the target hardware.
Source: AMD Vitis™ AI User Guide
In the second stage, predictions obtained from the DPU for each frame were processed using a Re-Identification (ReID) method. In this approach, visual features extracted from detected face regions are used to correctly recognize the same individual across subsequent frames. The ReID algorithm performs high-accuracy matching by comparing vectors composed of discriminative visual attributes such as color, texture, and spatial features.
Source: Attention Driven Person Re-identification (Yang et al., 2018)
Through this architecture, an individual detected in a single frame can be automatically tracked by the system, while allowing manual target selection via the handheld terminal when required. All processing is executed using hardware acceleration, ensuring compliance with the platform’s real-time analysis and low-latency data transfer requirements. This real-world implementation demonstrates the real-time AI inference and low-power capabilities of AMD FPGA-based hybrid platforms.
For more detailed information about the application or to request support for your AI projects, please contact us at ai@empa.com