Introduction
Cardiopulmonary Resuscitation (CPR) is an essential skill in emergency treatment. Currently, the assessment of CPR skills mainly depends on dummies and trainers, leading to high training costs and low efficiency. For the first time, we constructed a vision-based system to complete error action recognition and skill assessment in CPR. Specifically, we define 13 types of single-error actions and 74 types of composite error actions during external cardiac compression and then develop a video dataset named CPR-Coach. By taking the CPR-Coach as a benchmark, we thoroughly investigated and compared the performance of existing action recognition models based on different data modalities. To solve the unavoidable Single-class Training & Multi-class Testing problem, we propose a human-cognition-inspired framework named ImagineNet to improve the model’s multi-error recognition performance under restricted supervision. We hope this work could advance research toward fine-grained medical action analysis and skill assessment.
Statistics
The CPR-Coach dataset contains around 4.5K videos and 2.2M frames in total. The storage size of the entire dataset is 449GB. The CPR-Coach also provides optical flow images generated by the TV-L1 algorithm and 2D skeletons of the rescuer obtained by the Alphapose algorithm. The following table provides a detailed statistics of the dataset.
| Item | Data |
|---|---|
| Perspectives | 4 |
| FPS | 25 |
| Video Resolution | 4096×2160 (4K) |
| Number of Participants | 12 |
| Classes of Single-class Actions | 1+13=14 |
| Classes of Composite Error Actions | 59+10+5=74 |
| Frames (RGB) | 2,217,756 |
| Frames (RGB+Flow) | 6,644,596 |
| Videos | 4,544 |
| Avg. Len. of Videos | 19.52s |
| Storage Size | 449GB |
Download
Science the paper has not been officially accepted, we have only uploaded some example videos of the CPR-Coach Dataset. You can download these examples from Here. If you want to obtain the entire dataset, please contact us via E-mail (slwang19@fudan.edu.cn). Please indicate your work unit and purpose in E-mail.
Code of the ImagineNet
We propose a concise framework named ImagineNet to handle the intractable Single-class Training & Multi-class Testing problem properly. The essence of the ImagineNet is a human-inspired feature combination training strategy. As its name implies, it can Imagine composite error features based on restricted single-class error actions and achieves high performance in the unseen composite error recognition task. The code of the ImagineNet is available at Here.
System Demonstration
Our composite error action recognition system was received as a Demo by ICCV-2023. The detailed system demonstration video is available at Here.
Acknowledgements
This work was supported by the National Key R&D Program of China (2021ZD0113502) and the Shanghai Municipal Science and Technology Major Project (2021SHZDZX0103).