A Semantic Tree-Based Fast-Moving Object Trajectory Tracking Algorithm for Table Tennis

Introduction

In sports like table tennis, badminton, or baseball, tracking the fast-moving ball is a key technical problem for following analysis tasks. With the results of ball tracking, the analysis program could understand the process of a game, a round, or even a stoke. For example, with the tracking result, the analysis program could evaluate if the hit-point is proper in a training or competition. In addition, the tracking result can also help estimate the flying speed of the ball to evaluate the training effect of a professional athlete. In a number of professional situations, researches usually use radar or high-speed camera to track the fast-moving balls. However, these kinds of methods are difficult to widely use because of high cost and complexity. Towards this end, we aim to propose an algorithm to track the fast-moving table tennis ball based on images captured by commodity video camara, such as mobile phones. Such algorithms enable the development of web services and features for table tennis analysis, allowing users to enjoy accurate and professional analysis and guidance using mobile clients.

The basic idea of existing mainstream object tracking methods is finding matched objects after object detection in each frame. However, it is almost impossible to directly apply this kind of method for detecting the table tennis ball because it doesn’t have stable appearance and features, owing to its fast-moving state. On the other hand, the table tennis ball exhibits typical moving patterns like a shooting star in a video. As it is moving rapidly, the algorithm can preliminarily separate the table tennis ball, such as balls in Fig. 1, from the background by computing the difference of frame images (Rozumnyi et al., 2017). While not all the connected regions from separation are the table tennis ball, some of them are noises caused by movement of people or other objects. A proper algorithm should consider all the situations to distinguish the real trajectory of a ball.

To solve this problem, we propose a tree-based fast-moving object trajectory tracking (T-FORT) algorithm based on tree pruning and tracking strategies. The first step of the proposed algorithm is searching candidate regions based on differential images. Then, a tracking tree is formulated by the candidate regions. Because the candidate regions may contain a number of fake balls, we design a tree pruning process to filter fake objects to identify real balls. At last, the trajectory is restored based on a growing tracking tree process. To evaluate the effectiveness and performance of the proposed algorithm, we perform the proposed algorithm and other baseline methods on a self-built dataset and two open datasets. The experimental result demonstrates the excellent performance and robustness of the proposed algorithm. Our algorithm creatively combines tree structures and pruning algorithms to analyze all possible tracking results, greatly improving tracking recall and tracking performance.

The rest of this paper is structured as follows. Section 2 introduces the related work. In Section 3, we introduce the proposed method in details. In Section 4, we evaluate the proposed algorithm by experiments, based on a self-built dataset and two open source datasets, and explain the evaluation results. In Section 5, we conclude the work and point out future works.

Figure 1.

Example of table tennis ball to be tracked

Related Works

In 2017, the problem of tracking the “Fast Moving Object” in a table tennis scenario was first addressed in Rozumnyi et al. In this paper, the authors used binary differential images to search for possible candidate regions and then analyze each candidate region to identify table tennis balls. Their work can track table tennis balls in videos with a clean background. In this situation, the number of easily confused targets in differential images is relatively small. In addition, this algorithm cannot tolerate losing targets, which often happens in reality. In their following research works (Rozumnyi et al., 2021; Zita & Šroubek 2021), they try to introduce deep learning methods to improve the accuracy of detectors rather than improving the performance of tracking algorithms. To some extent, our work is inspired by the above works. However, we think that the fast-moving object tracking task, a very special object tracking problem, cannot be solved by the straightforward approach of applying deep learning method just using image features and semantic information.