Natural Language Processing-Based Triage of Superficial Soft Tissue Ultrasound Reports in Orthopedic Practice

Abstract

Background/Objectives: Natural language processing (NLP) has emerged as a promising approach for extracting clinically meaningful information from unstructured radiology reports. While most artificial intelligence applications in musculoskeletal imaging focus on image-based analysis, the potential of NLP for urgency assessment in superficial soft tissue ultrasound reports remains underexplored. This study aimed to develop and evaluate an NLP-based triage model to classify superficial soft tissue ultrasound reports according to clinical urgency in orthopedic practice. Methods: A curated dataset of superficial soft tissue ultrasound reports requested for palpable soft tissue masses and subcutaneous swellings was retrospectively collected from routine orthopedic outpatient practice. Reports were manually annotated into three triage categories: non-pathological (GREEN), non-urgent pathological (YELLOW), and urgent or potentially urgent findings (RED). A pretrained Turkish BERT model was fine-tuned for three-class classification. Model performance was evaluated using accuracy, macro-averaged F1 score, per-class precision and recall, and confusion matrices. An independent dataset of previously unseen reports was additionally used to assess robustness under real-world conditions. Results: After preprocessing and deduplication, 394 unique report segments were included. The baseline BERT model achieved an accuracy of 92.5% and a macro-averaged F1 score of 0.9106 on the test set. High classification performance was observed across all classes, with particularly reliable detection of RED reports representing urgent clinical conditions. External evaluation on independent reports demonstrated high agreement with physician annotations, with discrepancies mainly occurring in borderline or indeterminate cases. Conclusions: This study demonstrates that NLP-based analysis of superficial soft tissue ultrasound reports can effectively support urgency assessment in orthopedic practice. The proposed approach offers a practical, scalable, and image-independent solution for triage, with potential to improve workflow efficiency and facilitate timely clinical decision-making in musculoskeletal imaging.