Deep learning-based multi-cattle tracking in crowded livestock farming using video

Cattle monitoring is an essential aspect of precision farming, and recent advancements have greatly contributed to understanding cattle behavior using wearable devices like ear tags and collars, as well as contactless cameras for image-based detection. However, tracking multiple cattle in real farm conditions with cameras, particularly in crowded scenarios, poses significant challenges mainly due to scale variations, random motion, and occlusion. This paper proposes a deep learning-based framework with improved techniques for multi-cattle tracking using video, aiming to overcome these limitations. The proposed algorithm utilizes a detection-based tracking approach, leveraging a YOLO-v5 detector trained specifically for cattle detection to provide initial targets. The main contributions of our research primarily focus on implementing the tracking algorithm to address the aforementioned problems. Several improvements are introduced: first, to handle appearance and scale deformation, a wide residual network with SPP-Net is employed as the backbone to extract cattle appearance information. Second, an ensemble Kalman filter is utilized to adapt to unexpected movements. Additionally, the angle from the centered position of the individuals to the origin of the image is incorporated to predict their location. Third, to tackle occlusion, a novel bench-matching mechanism is designed, allowing for the retrieval of lost trajectories based on the assumption of a known number of cattle in the barn. To validate the performance of the proposed framework, experiments are conducted using video sequences from our Hanwoo cattle tracking dataset. Comparisons with other state-of-the-art trackers are also performed. Our method achieves an accuracy of 84.49 % in data association, which represents a significant improvement considering the challenges involved in precision livestock farming applications.