Radiomics from ovarian ultrasound: research demonstration

Ovarian masses seen on ultrasound range from harmless cysts to cancer, and telling them apart can be difficult. Radiomics turns images into hundreds of numbers describing shape, brightness and texture, which a model can use for classification. This project tests whether such features can separate eight types of ovarian findings in a public dataset, as a demonstration of method rather than a diagnostic tool.

Can quantitative radiomic features extracted from segmented ovarian ultrasound images help distinguish the eight official MMOTU diagnostic categories?

This is a research demonstration and does not establish clinical validity or diagnostic utility.

MMOTU is a public collection of ovarian ultrasound images, each labelled with one of eight diagnostic categories and paired with a segmentation mask. Some images had to be excluded because files were missing or broken. The categories are unbalanced: some have hundreds of images and others only a few dozen, which makes the rarer ones harder to learn.

MMOTU OTU_2d contains 1,469 labeled images in eight official categories. 1,269 images have usable image and segmentation-mask files; 200 observations were excluded because files or masks were missing or invalid. The label lists identify images by filename and do not provide reliable patient identifiers.

The table below shows the existing class distribution.

Segmentation means outlining the lesion on the image. The resulting mask (white for the lesion, black for the background) tells the software exactly which pixels to analyse, so features describe the lesion rather than the surrounding tissue.

The mask defines the region of interest used for radiomic extraction. This example displays one existing image, its binary mask, and the mask overlay.

PyRadiomics is a widely used open-source tool that computes standard radiomic features. First-order features describe the distribution of pixel brightness; texture features (GLCM, GLRLM) describe how brightness patterns repeat across the image; shape features describe the outline of the lesion. Without the physical size of each pixel, shape values are in pixels rather than millimetres.

PyRadiomics was applied to segmented ultrasound regions. The saved feature configuration contains 67 columns across first-order, GLCM, GLRLM, and 2D shape families. Physical pixel spacing is unavailable; shape values are in pixel units and have no physical interpretation.

This summary reads the recorded extractor configuration; it does not rerun feature extraction.

Data leakage happens when information from test data slips into model building, making results look better than they are. To prevent it, every preparation step (filling missing values, removing uninformative features, scaling, selecting the most useful features) is learned on the training data only and then applied unchanged to the data used for testing.

The model pipeline uses imputation, variance filtering, scaling, and mutual-information feature selection. These steps are fit only on each training fold, then applied to that fold's validation data. No feature selection or preprocessing is fit on the held-out official validation partition.

Validation is how a model’s performance is estimated on data it has not seen. Five-fold cross-validation splits the training data into five parts, each taking a turn as the test set. A separate official validation set is kept aside until the very end. Because the dataset has no patient identifiers, images from the same patient could appear in both training and testing, which limits how far results can be generalised.

Five-fold cross-validation was performed on the official training partition, with the official validation partition held out until final evaluation. The official validation partition was not used for tuning. The random seed was 42. A duplicate-grouped cross-validation was also performed as a robustness analysis.

No patient-level independence can be claimed because reliable patient identifiers are unavailable.

Four models are compared, including a “dummy” baseline that always guesses the most common class, the score any real model must beat. Accuracy is the share of correct predictions; macro F1 and balanced accuracy give each of the eight categories equal weight, so rare categories count as much as common ones. A large gap between training and validation scores is a sign of overfitting: the model memorises rather than learns.

The main CV table below reports the existing standard image-level CV results. The official validation metrics are shown separately. These are reported as model comparisons, not as clinical interpretations.

Logistic Regression has the smallest train–validation gap. Random Forest and XGBoost show larger gaps, consistent with greater overfitting; the more complex models achieve somewhat higher CV AUCs. Recall varies substantially across classes.

A confusion matrix shows, for each true category, how often the model predicted each category, making it clear which types get mixed up. ROC curves show the trade-off between catching a category and raising false alarms. Calibration checks whether predicted probabilities are honest: if the model says 70%, it should be right about 70% of the time.

These are the existing figures for the official validation partition. Recall varies substantially across the eight categories; calibration and discrimination summaries are also reported in the tables above.

Dummy baseline

Logistic Regression

Random Forest

XGBoost

Robustness checks test whether results hold up when the analysis is done a slightly different way. Here the main concern is identical images appearing more than once: if a copy lands in both training and testing, the model is effectively tested on something it has already seen. Cross-validation was repeated keeping duplicates together to see whether scores changed.

The exact-duplicate audit found no duplicated modeling observations and no exact-duplicate groups crossing CV folds or the official train/validation boundary. Duplicate-grouped CV produced essentially unchanged results. The official validation set required no duplicate-based exclusions, and its metrics remained unchanged.

The existing macro-F1 chart summarizes duplicate-grouped CV by model. The table in the next section compares those values with standard image-level CV.

The dataset contains many duplicated image files. The audit compares files byte by byte to find exact copies and checks that no copy is split across training and testing. This guards against a common and often overlooked source of inflated results in public imaging datasets.

The audit inspected 1,886 physical source files and identified 617 sets of duplicate physical source files. Each labeled modeling observation was used only once. Zero duplicate groups crossed CV folds or the official train/validation boundary. The table compares existing original and duplicate-grouped CV values; no new model fitting is performed here.

This section lists the rules followed to keep the analysis honest: no information from test data used during training, a fixed random seed so results can be reproduced exactly, and no invented data such as patient identifiers, clinical variables or pixel sizes that the dataset does not provide.

• Imputation, variance filtering, scaling, and feature selection are fit within training folds.
• The held-out official validation partition is not used for tuning.
• Random seed = 42.
• No patient identifiers or clinical covariates are fabricated.
• No physical spacing is invented.
• The official eight-class labels are retained; no benign/malignant remapping is imposed.

Every study has limits, and stating them clearly is part of the result. These define what the findings can and cannot support, and what would be needed, such as patient-level data or images from other hospitals, before any clinical claim could be made.

1. No reliable patient identifiers are available.
2. Patient-level independent validation cannot be claimed.
3. The evidence comes from one public dataset and acquisition context.
4. No external-center validation was performed.
5. Physical pixel spacing is unavailable.
6. Radiomic features may depend on annotation, display, and compression characteristics.
7. The results do not establish clinical validity or diagnostic utility.

This closing section summarises what the experiment shows: a reproducible radiomics and machine-learning workflow, how the models compared, and what the checks for leakage and duplicates found. It should be read as a methodological demonstration, not as a validated diagnostic model.

This demonstration shows that a reproducible radiomics-to-ML workflow can be built on MMOTU ovarian ultrasound data, including leakage-aware preprocessing, model comparison, calibration, and robustness checks. More complex models achieved somewhat higher discrimination but also showed larger train–validation gaps. Exact duplicate leakage was investigated and did not materially affect performance. The experiment should be interpreted as a methodological demonstration rather than a clinically validated diagnostic model.