D-DeepOCSORT: Multi-Object Tracking Algorithm Based on LiDAR and Monocular Camera

Abstract:

Multi-object tracking is a fundamental problem in computer vision, and DeepOCSORT, due to its reliance on the 2D im-age position information of targets, tends to exhibit prediction drift. To enhance the tracking stability of DeepOCSORT, this paper proposes a novel multi-sensor data fusion-based multi-object tracking method. Specifically, we build upon the DeepOCSORT foundation and additionally integrate target velocity information directly measured by LiDAR. The intro-duction of this velocity information is conducted from three perspectives: firstly, during data association, a penalty term is constructed based on the differences in target velocities to constrain generating matches with consistent velocities; sec-ondly, using LiDAR velocity for initialization and online updating of the velocity state within the tracker, making tracking predictions more stable; thirdly, controlling the degree of dependence on velocity information by adjusting the process noise covariance matrix. Evaluation results on the KITTI dataset demonstrate that compared to the original DeepOCSORT, the proposed improved multi-source heterogeneous information fusion method significantly enhances tracking performance, with maximum improvements of 3.35, 3.26, and 3.71 on the HOTA, MOTA, and IDF1 metrics, respectively. This study provides an effective approach to building a more stable and accurate multi-object tracking sys-tem.

Key words:Multi-object Tracking, LiDAR–camera system, Multi-sensor fusion