TY - GEN
T1 - An Auxiliary Tasks Based Framework for Automated Medical Skill Assessment with Limited Data
AU - Zhao, Shang
AU - Zhang, Xiaoke
AU - Jin, Fang
AU - Hahn, James
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Automated medical skill assessment facilitates medical education by merging varying clinical experiences across instructors for standardizing medical training. However, medical datasets for training such automated assessment rarely have satisfactory sizes due to the cost of data collection, safety concerns and privacy restrictions. Current medical training relies on evaluation rubrics that usually include multiple auxiliary labels to support the overall evaluation from varying aspects of the procedure. In this paper, we explore machine learning algorithms to design a generalizable auxiliary task-based framework for medical skill assessment to address training automated systems with limited data. Our framework exhaustively mines valid auxiliary information in the evaluation rubric to pre-train the feature extractor before training the skill assessment classifier. Notably, a new regression-based multitask weighting method is the key to pre-train a meaningful feature representation comprehensively, ensuring the evaluation rubric is well imitated in the final model. The overall evaluation task can be fine-tuned based on the pre-trained rubric-based feature representation. Our experimental results on two medical skill datasets show that our work can significantly improve performance, achieving 85.9% and 97.4% accuracy in the intubation dataset and surgical skill dataset, respectively.
AB - Automated medical skill assessment facilitates medical education by merging varying clinical experiences across instructors for standardizing medical training. However, medical datasets for training such automated assessment rarely have satisfactory sizes due to the cost of data collection, safety concerns and privacy restrictions. Current medical training relies on evaluation rubrics that usually include multiple auxiliary labels to support the overall evaluation from varying aspects of the procedure. In this paper, we explore machine learning algorithms to design a generalizable auxiliary task-based framework for medical skill assessment to address training automated systems with limited data. Our framework exhaustively mines valid auxiliary information in the evaluation rubric to pre-train the feature extractor before training the skill assessment classifier. Notably, a new regression-based multitask weighting method is the key to pre-train a meaningful feature representation comprehensively, ensuring the evaluation rubric is well imitated in the final model. The overall evaluation task can be fine-tuned based on the pre-trained rubric-based feature representation. Our experimental results on two medical skill datasets show that our work can significantly improve performance, achieving 85.9% and 97.4% accuracy in the intubation dataset and surgical skill dataset, respectively.
UR - http://www.scopus.com/inward/record.url?scp=85122549232&partnerID=8YFLogxK
U2 - 10.1109/EMBC46164.2021.9630498
DO - 10.1109/EMBC46164.2021.9630498
M3 - Conference contribution
C2 - 34891594
AN - SCOPUS:85122549232
T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
SP - 1613
EP - 1617
BT - 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2021
Y2 - 1 November 2021 through 5 November 2021
ER -