Wiki Contents

Detection of Risky Tackles from Sports Video

Project Description

Centers for Disease Control (CDC) estimates that between 1.6 and 3.8 million sports-related concussions (SRC) are reported annually with American football showing the highest proportion of head injuries or concussions among all sports. Research shows that in youth football, on average, one player out of every 33 players may suffer a concussion during the season. In addition, the head impact may cause brain injuries such as hemorrhage, hematoma, and edema. Two-thirds of all football-related head injuries occur during practice and one-third during games, and 47% of all SRC occur as a result of head-to-head collisions. Learning proper tackle at an early age is an important developmental milestone for reducing unnecessary head impacts among youth football players.

The key rationale for visual analysis of sports videos is monitoring and then providing early warnings of potentially injurious practices. This would allow coaches to intervene to prevent injury and mitigate resulting risk and harm, whether physical, psychological, or financial, from improper tackling. Through this project, we introduce the problem of risky tackle detection directly from American football practice videos. We aim to solve the problem using modern AI-based computer vision techniques. We have built a tackle detection dataset by collecting practice videos from different practice fields. We formulate the risky tackle detection problem as a classification task of two classes - safe tackle and risky tackle.

To this goal, we have collected and annotated a data set of 545 videos by collaborating with experts from the Department of Athletic Training. The tackle occurs between a player and a blocking dummy. Coaches use blocking dummies to avoid player-to-player head impacts while teaching the skill to young players. They label the dataset such that 0 indicates the tackle is safe and 1 indicates the tackle is risky.

Keywords

risky tackle detection, tackle classification, american football, spatiotemporal pattern recognition, 3D convolution, pose estimation

Methods

Our initial approach to solving this problem considers segmenting regions in space and time (spatiotemporal event detection) and relates the 3D convolutional features of those detected "bounding boxes in space and time" to labeled annotations. Thus, we propose a multi-stage pipeline that identifies the tackle event-related video frames from the video and then extracts the bounding box of the tackle-performing player leveraging existing state-of-the-art human detection models. Finally, 3D convolution is applied to classify risky and safe tackles based on spatiotemporal features. Initially, we experimented with a smaller dataset of 178 videos. Our empirical results demonstrate that our proposed method outperforms state-of-the-art video classification and anomaly detection approaches applied directly to untrimmed tackle videos. We refer interested readers to our MLDM 2022 publication.

We are expanding our work to detect mask segmentation of the players and the dummies. We believe this will aid automated risky tackle detection as well as the manual analysis of human judges. Further, we are working on estimating the pose of the tackle-performing player and leveraging skeleton-based activity detection approaches to achieve better classification results. In the future, we plan to use multi-modal approaches to detect risky tackles that combine RGB, mask segmentation, and pose.

Current Team Members

  • Nasik Muhammad Nafi - Team Leader & Graduate Research Assistant (Ph.D. Candidate)
  • Ahsan Zaidi - Ph.D. Student
  • Sean Chin Loy - Undergraduate Student
  • William H. Hsu - Professor, Computer Science, Kansas State University
  • Derek Christensen - M.S. Operations Research, Kansas State University

Affiliates

  • Scott Dietrich - Assistant Professor, Athletic Training, Barry University

Alumni

Data Sets

This project has a confidential dataset kept on our data server. This may only be visible to KDD researchers who have access to Volare.

Currently, we have a new dataset consisting of 545 tackle videos. There are 371 safe videos and 174 risky videos. Earlier, we worked on a smaller dataset that had 178 tackle practice videos. Our MLDM 2022 publication is based on this smaller dataset. Data has been collected from different practice fields at different times thus providing a variety in terms of background and players present.

Trello Board

Source Code

To be posted.

References

  • Nonaka, N., Fujihira, R., Nishio, M., Murakami, H., Tajima, T., Yamada, M., Maeda, A. & Seita, J. (2022). End-to-End High-Risk Tackle Detection System for Rugby. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3550-3559), New Orleans, LA, USA, June 19-24, 2022.

  • Martin, Z., Hendricks, S., & Patel, A. (2021). Automated tackle injury risk assessment in contact-based sports-a rugby union example. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4594-4603), Nashville, TN, USA, June 19-25, 2021.

KDD Lab Publications

  • Nafi, Nasik Muhammad, Scott Dietrich, and William Hsu. "Risky Tackle Detection from American Football Practice Videos using 3D Convolutional Networks."

  • Hu, Y. (2021). Deep vision with generative adversarial networks to augment and classify tackle images in American youth football. MS Report, Department of Computer Science, Kansas State University, 2021.

Back to KSU KDD Lab Computer Vision (CV) research overview

Last updated by ahsanzaidi on May 24, 2024