Information Extraction from Polish Radiology Reports using Language Models

Overview

Radiology reports are critical for patient care but are typically written as free text, which introduces ambiguity, inconsistency, and omission risks. Structured Reporting (SR) provides standardization but is rarely adopted due to workflow burdens. We propose an automatic parametrization model for Polish radiology reports using deep language models. Trained on 1,200 annotated chest CT reports labeled with 44 observation tags, our model achieves an F1 score of 81%, bridging free-text flexibility with structured interpretability.

Problem Framing

Structured Reporting is endorsed by RSNA and ESR and improves accuracy, consistency, and integration with classification systems such as CO-RADS, yet adoption remains low due to perceived rigidity and workflow overhead. We frame the task as sequence labeling under the information extraction paradigm — not simple NER — since radiological findings are contextual and span-dependent.

Dataset

We collected 1,200 anonymized chest reports from the University Clinical Centre in Gdańsk, Poland, double-annotated by clinical experts with 44 standardized observation tags. Prior Polish systems were rule-based, and no medical-domain Polish language models existed before this work.

Models

We evaluated several general-domain Polish language models — Polish RoBERTa-base-v2, Polish DistilRoBERTa, Polish Longformer, HerBERT, and the multilingual entity-aware mLUKE — adapting them to assign standardized tags to free-text radiological observations.

Why It Matters

The system reconciles the expressiveness of free text with the machine-readability of structured data, facilitating clinical integration, data reuse, and downstream analytics for Polish-language radiology.