Online citations, reference lists, and bibliographies.
← Back to Search

Quality Of Science And Reporting Of Radiomics In Oncologic Studies: Room For Improvement According To Radiomics Quality Score And TRIPOD Statement

J. Park, D. Kim, H. S. Kim, S. Park, J. Kim, S. J. Cho, Jae Ho Shin, J. Kim
Published 2019 · Medicine

Cite This
Download PDF
Analyze on Scholarcy
Share
Objectives To evaluate radiomics studies according to radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to provide objective measurement of radiomics research. Materials and methods PubMed and Embase were searched for studies published in high clinical imaging journals until December 2018 using the terms “radiomics” and “radiogenomics.” Studies were scored against the items in the RQS and TRIPOD guidelines. Subgroup analyses were performed for journal type (clinical vs. imaging), intended use (diagnostic vs. prognostic), and imaging modality (CT vs. MRI), and articles were compared using Fisher’s exact test and Mann-Whitney analysis. Results Seventy-seven articles were included. The mean RQS score was 26.1% of the maximum (9.4 out of 36). The RQS was low in demonstration of clinical utility (19.5%), test-retest analysis (6.5%), prospective study (3.9%), and open science (3.9%). None of the studies conducted a phantom or cost-effectiveness analysis. The adherence rate for TRIPOD was 57.8% (mean) and was particularly low in reporting title (2.6%), stating study objective in abstract and introduction (7.8% and 16.9%), blind assessment of outcome (14.3%), sample size (6.5%), and missing data (11.7%) categories. Studies in clinical journals scored higher and more frequently adopted external validation than imaging journals. Conclusions The overall scientific quality and reporting of radiomics studies is insufficient. Scientific improvements need to be made to feature reproducibility, analysis of clinical utility, and open science categories. Reporting of study objectives, blind assessment, sample size, and missing data is deemed to be necessary. Key Points • The overall scientific quality and reporting of radiomics studies is insufficient. • The RQS was low in demonstration of clinical utility, test-retest analysis, prospective study, and open science. • Room for improvement was shown in TRIPOD in stating study objective in abstract and introduction, blind assessment of outcome, sample size, and missing data categories.
This paper references
10.1007/s00330-018-5830-3
Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging
Y. Park (2018)
10.1093/neuonc/now135
Radiogenomics to characterize regional genetic heterogeneity in glioblastoma
L. Hu (2017)
10.1148/radiol.2018180179
Radiomic Phenotypes of Mammographic Parenchymal Complexity: Toward Augmenting Breast Density in Breast Cancer Risk Assessment.
D. Kontos (2019)
10.1148/radiol.2015151169
Radiomics: Images Are More than Pictures, They Are Data
R. Gillies (2016)
10.1186/s12916-018-1099-2
Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement
P. Heus (2018)
10.1016/j.jtho.2016.07.002
Predicting Malignant Nodules from Screening CT Scans
Samuel H. Hawkins (2016)
10.1148/radiol.2018173064
Radiomic Machine Learning for Characterization of Prostate Lesions with MRI: Comparison to ADC Values.
D. Bonekamp (2018)
10.1007/s00330-018-5395-1
Identification of high-risk plaque features in intracranial atherosclerosis: initial experience using a radiomic approach
Z. Shi (2018)
10.1016/j.radonc.2018.03.033
Tracking tumor biology with radiomics: A systematic review utilizing a radiomics quality score.
S. Sanduleanu (2018)
10.1093/neuonc/now270
Magnetic resonance perfusion image features uncover an angiogenic subgroup of glioblastoma patients with poor survival and better response to antiangiogenic treatment
T. Liu (2017)
10.1007/s00330-018-5763-x
Preoperative radiomic signature based on multiparametric magnetic resonance imaging for noninvasive evaluation of biological characteristics in rectal cancer
Xiaochun Meng (2018)
10.1007/s00330-017-4855-3
Comparison of a radiomic biomarker with volumetric analysis for decoding tumour phenotypes of lung adenocarcinoma with different disease-specific survival
Mei Yuan (2017)
10.3348/kjr.2016.17.5.706
Does the Reporting Quality of Diagnostic Test Accuracy Studies, as Defined by STARD 2015, Affect Citation?
Y. Choi (2016)
10.1007/s00330-017-5005-7
Preoperative prediction of sentinel lymph node metastasis in breast cancer based on radiomics of T2-weighted fat-suppression and diffusion-weighted MRI
Y. Dong (2017)
10.1093/JNCI/DJI237
Reporting recommendations for tumor marker prognostic studies (REMARK).
L. McShane (2005)
10.1007/s00330-018-5629-2
Radiomics nomogram for predicting the malignant potential of gastrointestinal stromal tumours preoperatively
T. Chen (2018)
10.1007/s00330-017-5221-1
Radiomic signature as a diagnostic factor for histologic subtype classification of non-small cell lung cancer
Xinzhong Zhu (2017)
10.1158/1078-0432.CCR-17-1038
Radiomics Analysis for Evaluation of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer
Z. Liu (2017)
10.1148/RADIOL.2016161382
Radiogenomics of Glioblastoma: Machine Learning-based Classification of Molecular Characteristics by Using Multiparametric and Multiregional MR Imaging Features.
P. Kickingereder (2016)
10.1016/j.ejca.2011.11.037
Qualification of imaging biomarkers for oncology drug development.
J. Waterton (2012)
10.1148/radiol.2016160845
Radiomic Profiling of Glioblastoma: Identifying an Imaging Predictor of Patient Survival with Improved Performance over Established Clinical and Radiologic Risk Models.
P. Kickingereder (2016)
10.1007/s00330-018-5747-x
Radiomics features on non-contrast-enhanced CT scan can precisely classify AVM-related hematomas from other spontaneous intraparenchymal hematoma types
Y. Zhang (2018)
10.1148/radiol.2018180200
Radiomic MRI Phenotyping of Glioblastoma: Improving Survival Prediction.
S. Bae (2018)
10.1158/1078-0432.CCR-16-2910
Radiomics Features of Multiparametric MRI as Novel Prognostic Factors in Advanced Nasopharyngeal Carcinoma
Bin Zhang (2017)
10.1007/s00330-017-5267-0
Genotype prediction of ATRX mutation in lower-grade gliomas using an MRI radiomics signature
Y. Li (2017)
10.1158/1078-0432.CCR-17-1510
A Radiomics Nomogram for the Preoperative Prediction of Lymph Node Metastasis in Bladder Cancer
Shaoxu Wu (2017)
10.1007/s00330-016-4638-2
Radiation injury vs. recurrent brain metastasis: combining textural feature radiomics analysis and standard parameters may increase 18F-FET PET accuracy without dynamic scans
P. Lohmann (2016)
10.1007/s00330-018-5725-3
Preoperative prediction of cavernous sinus invasion by pituitary adenomas using a radiomics method based on magnetic resonance images
Jianxing Niu (2018)
10.1007/s00330-018-5368-4
Primary central nervous system lymphoma and atypical glioblastoma: Differentiation using radiomics approach
Hie Bum Suh (2018)
10.1148/RADIOL.2016152110
MR Imaging Radiomics Signatures for Predicting the Risk of Breast Cancer Recurrence as Given by Research Versions of MammaPrint, Oncotype DX, and PAM50 Gene Assays.
H. Li (2016)
10.1093/neuonc/now256
MRI features predict survival and molecular markers in diffuse lower-grade gliomas
H. Zhou (2017)
10.1158/1078-0432.CCR-17-3445
Machine Learning–Based Radiomics for Molecular Subtyping of Gliomas
Chia-Feng Lu (2018)
Radiomics features on noncontrast-enhanced CT scan can precisely classify AVM-related hematomas from other spontaneous intraparenchymal hematoma
Y Zhang (2018)
10.1148/RADIOL.2016152234
Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non-Small Cell Lung Cancer.
Yanqi Huang (2016)
10.1007/s00330-017-5270-5
Imaging prediction of nonalcoholic steatohepatitis using computed tomography texture analysis
S. Naganawa (2017)
10.1007/s00330-018-5706-6
Differentiation between pilocytic astrocytoma and glioblastoma: a decision tree model using contrast-enhanced magnetic resonance imaging-derived quantitative radiomic features
Fei Dong (2018)
10.1038/nrclinonc.2017.141
Radiomics: the bridge between medical imaging and personalized medicine
P. Lambin (2017)
10.1007/s00330-018-5824-1
Radiomics model of contrast-enhanced computed tomography for predicting the recurrence of acute pancreatitis
Yong Chen (2018)
10.1016/j.jtho.2016.11.2230
Imaging Phenotyping Using Radiomics to Predict Micropapillary Pattern within Lung Adenocarcinoma
S. H. Song (2017)
10.1093/neuonc/nox092
Quantitative imaging biomarkers for risk stratification of patients with recurrent glioblastoma treated with bevacizumab
P. Grossmann (2017)
10.1148/radiol.2018181352
Radiomic versus Convolutional Neural Networks Analysis for Classification of Contrast-enhancing Lesions at Multiparametric Breast MRI.
D. Truhn (2019)
10.1136/eb-2013-101637
Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD
D. Korevaar (2013)
10.1158/1078-0432.CCR-16-0702
Large-scale Radiomic Profiling of Recurrent Glioblastoma Identifies an Imaging Predictor for Stratifying Anti-Angiogenic Treatment Response
P. Kickingereder (2016)
10.7326/M14-0698
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
K. Moons (2015)
10.1158/0008-5472.CAN-17-0122
Somatic Mutations Drive Distinct Imaging Phenotypes in Lung Cancer.
E. Rios Velazquez (2017)
10.1007/s00330-018-5463-6
Classifying brain metastases by their primary site of origin using a radiomics approach based on texture analysis: a feasibility study
Rafael Ortiz-Ramón (2018)
10.1148/radiol.2018172229
Prediction of Response to Neoadjuvant Chemotherapy and Radiation Therapy with Baseline and Restaging 18F-FDG PET Imaging Biomarkers in Patients with Esophageal Cancer.
R. J. Beukinga (2018)
10.1007/s00330-017-5180-6
Non-invasive radiomics approach potentially predicts non-functioning pituitary adenomas subtypes before surgery
Shuaitong Zhang (2017)
10.1093/neuonc/noy033
In vivo evaluation of EGFRvIII mutation in primary glioblastoma patients via complex multiparametric MRI signature
H. Akbari (2018)
10.1007/s00330-017-5154-8
A predictive model for distinguishing radiation necrosis from tumour progression after gamma knife radiosurgery based on radiomic features from MR images
Zijian Zhang (2017)
10.1148/radiol.14141160
Reporting diagnostic accuracy studies: some improvements after 10 years of STARD.
D. Korevaar (2015)
10.1007/s00330-018-5343-0
Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT
Wenbing Lv (2018)
10.1148/radiol.2018172300
MR Imaging of Rectal Cancer: Radiomics Analysis to Assess Treatment Response after Neoadjuvant Therapy.
Natally Horvat (2018)
10.1093/neuonc/noy021
Diffusion radiomics as a diagnostic model for atypical manifestation of primary central nervous system lymphoma: development and multicenter external validation
Daesung Kang (2018)
10.1007/s00330-018-5509-9
The predictive value of CT-based radiomics in differentiating indolent from invasive lung adenocarcinoma in patients with pulmonary nodules
Yunlang She (2018)
a revised tool for the quality assessment of diagnostic accuracy studies
PF Whiting (2020)
10.1136/gutjnl-2018-316204
Deep learning Radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study
K. Wang (2018)
10.1007/s00330-018-5730-6
Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features
P. Yin (2018)
10.1158/1078-0432.CCR-15-2997
Rectal Cancer: Assessment of Neoadjuvant Chemoradiation Outcome based on Radiomics of Multiparametric MRI
K. Nie (2016)
10.1148/radiol.2017170226
Psychoradiologic Utility of MR Imaging for Diagnosis of Attention Deficit Hyperactivity Disorder: A Radiomics Analysis.
Huaiqiang Sun (2018)
10.1007/s00330-018-5683-9
Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer
Yanfen Cui (2018)
10.1007/s00330-018-5797-0
Ultrasound-based radiomics score: a potential biomarker for the prediction of microvascular invasion in hepatocellular carcinoma
Hang-Tong Hu (2018)
10.1007/s00330-018-5802-7
MRI radiomics analysis for predicting preoperative synchronous distant metastasis in patients with rectal cancer
H. Liu (2018)
10.1093/neuonc/nox188
Radiomic subtyping improves disease stratification beyond key molecular, clinical, and standard imaging characteristics in patients with glioblastoma
P. Kickingereder (2018)
10.1007/s00330-018-5539-3
A clinical-radiomics nomogram for the preoperative prediction of lung metastasis in colorectal cancer patients with indeterminate pulmonary nodules
Tingdan Hu (2018)
10.1158/1078-0432.CCR-18-1305
A Combined Nomogram Model to Preoperatively Predict Histologic Grade in Pancreatic Neuroendocrine Tumors
W. Liang (2018)
10.1177/0962280214537333
The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions
L. Kessler (2015)
10.1158/1078-0432.CCR-17-3420
A Coclinical Radiogenomic Validation Study: Conserved Magnetic Resonance Radiomic Appearance of Periostin-Expressing Glioblastoma in Patients and Xenograft Models
P. Zinn (2018)
10.1007/s00330-018-5787-2
Predicting the grade of hepatocellular carcinoma based on non-contrast-enhanced MRI radiomics signature
M. Wu (2018)
10.1007/s00330-017-4800-5
Machine learning-based analysis of MR radiomics can help to improve the diagnostic performance of PI-RADS v2 in clinically relevant prostate cancer
Jing Wang (2017)
10.1038/nrclinonc.2016.162
Imaging biomarker roadmap for cancer studies
J. O’Connor (2017)
10.1007/s00330-018-5581-1
Radiomics nomogram outperforms size criteria in discriminating lymph node metastasis in resectable esophageal squamous cell carcinoma
Xianzheng Tan (2018)
10.1007/s00330-018-5575-z
A multi-sequence and habitat-based MRI radiomics signature for preoperative prediction of MGMT promoter methylation in astrocytomas with prognostic implication
J. Wei (2018)
10.7326/0003-4819-155-8-201110180-00009
QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies
P. Whiting (2011)
10.1148/radiol.2017170273
Radiomics Based on Adapted Diffusion Kurtosis Imaging Helps to Clarify Most Mammographic Findings Suspicious for Cancer.
S. Bickelhaupt (2018)
10.1007/s00330-017-5146-8
Can CT-based radiomics signature predict KRAS/NRAS/BRAF mutations in colorectal cancer?
L. Yang (2017)
10.1016/S1470-2045(18)30413-3
A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study.
R. Sun (2018)
10.1001/JAMA.289.10.1278
Central challenges facing the national clinical research enterprise.
N. Sung (2003)
10.1148/radiol.2018181408
Biliary Tract Cancer at CT: A Radiomics-based Model to Predict Lymph Node Metastasis and Survival Outcomes.
Gu-Wei Ji (2019)
10.1200/JCO.2015.65.9128
Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer.
Yanqi Huang (2016)
10.1007/s00330-018-5639-0
Prognostic value of radiomic analysis of iodine overlay maps from dual-energy computed tomography in patients with resectable lung cancer
J. Choe (2018)
10.1038/ncomms5006
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
H. Aerts (2014)
10.1007/s00330-017-5302-1
Multiregional radiomics features from multiparametric MRI for prediction of MGMT methylation status in glioblastoma multiforme: A multicentre study
Zhi-Cheng Li (2017)
10.1007/s00330-018-5704-8
Radiomics based on multicontrast MRI can precisely differentiate among glioma subtypes and predict tumour-proliferative behaviour
Changliang Su (2018)
10.1093/neuonc/noy133
Incorporating diffusion- and perfusion-weighted MRI into a radiomics model improves diagnostic performance for pseudoprogression in glioblastoma patients
J. Kim (2019)
10.1148/radiol.2018180946
Radiomics of Brain MRI: Utility in Prediction of Metastatic Tumor Type.
H. Kniep (2019)
10.1016/j.jtho.2016.11.2226
Radiomic‐Based Pathological Response Prediction from Primary Tumors and Lymph Nodes in NSCLC
T. Coroller (2017)
Radiomics signature onmagnetic resonance imaging: association with disease-free survival in patients with invasive breast cancer
ParkH (2018)
10.1007/s00330-016-4653-3
Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma
Jinhua Yu (2016)
10.1007/s00330-018-5381-7
MR-based radiomics signature in differentiating ocular adnexal lymphoma from idiopathic orbital inflammation
J. Guo (2018)
10.1007/s00330-017-4964-z
MRI features can predict EGFR expression in lower grade gliomas: A voxel-based radiomic analysis
Y. Li (2017)
10.1007/s00330-018-5583-z
The MR radiomic signature can predict preoperative lymph node metastasis in patients with esophageal cancer
Jinrong Qu (2018)
10.1158/1078-0432.CCR-17-3783
Radiomics Signature on Magnetic Resonance Imaging: Association with Disease-Free Survival in Patients with Invasive Breast Cancer
Hyunjin Park (2018)



This paper is referenced by
10.1186/s13244-020-00887-2
Radiomics in medical imaging—“how-to” guide and critical reflection
Janita E van Timmeren (2020)
10.1007/s11547-020-01188-w
Computed tomography (CT)-derived radiomic features differentiate prevascular mediastinum masses as thymic neoplasms versus lymphomas
M. Kirienko (2020)
10.1007/s00259-019-04625-9
Radiomics of 18F-FDG PET/CT images predicts clinical benefit of advanced NSCLC patients to checkpoint blockade immunotherapy
W. Mu (2019)
10.1055/s-0039-3400268
Quantification in Musculoskeletal Imaging Using Computational Analysis and Machine Learning: Segmentation and Radiomics.
M. Bach Cuadra (2020)
10.1016/j.canrad.2020.07.005
Use of radiomics in the radiation oncology setting: Where do we stand and what do we need?
U. Schick (2020)
10.1371/journal.pone.0234800
Preliminary study on discriminating HER2 2+ amplification status of breast cancers based on texture features semi-automatically derived from pre-, post-contrast, and subtraction images of DCE-MRI
Lirong Song (2020)
10.1016/j.mehy.2019.109415
Radiotherapy dose painting by circadian rhythm based radiomics.
H. Abdollahi (2019)
10.1093/pcmedi/pbaa028
The application of artificial intelligence and radiomics in lung cancer
Yaojie Zhou (2020)
10.1007/s00330-019-06564-3
Deep learning: definition and perspectives for thoracic imaging
G. Chassagnon (2019)
10.21037/atm-20-4589
Radiomics: an overview in lung cancer management-a narrative review
Bojiang Chen (2020)
10.1186/s41824-020-00078-8
Methodological framework for radiomics applications in Hodgkin’s lymphoma
M. Sollini (2020)
10.1016/j.breast.2019.10.018
Overview of radiomics in breast cancer diagnosis and prognostication
A. Tagliafico (2019)
10.1016/j.critrevonc.2020.103068
Radiomics and "radi-…omics" in cancer immunotherapy: a guide for clinicians.
M. Porcu (2020)
10.1016/j.ejrad.2020.109095
Prostate MRI radiomics: A systematic review and radiomic quality score assessment.
A. Stanzione (2020)
10.3389/fcvm.2020.591368
Radiomics Signatures of Cardiovascular Risk Factors in Cardiac MRI: Results From the UK Biobank
I. Cetin (2020)
10.1016/j.ymeth.2020.01.007
PET/CT radiomics in breast cancer: mind the step.
M. Sollini (2020)
10.1007/s00330-020-07221-w
A systematic review of radiomics in osteosarcoma: utilizing radiomics quality score as a tool promoting clinical translation
Jingyu Zhong (2020)
10.3748/wjg.v26.i32.4729
New advances in radiomics of gastrointestinal stromal tumors
R. Cannella (2020)
10.1007/s00330-020-06927-1
Current status and quality of radiomics studies in lymphoma: a systematic review
Hongxi Wang (2020)
10.21037/cco.2019.12.02
Role of image-guided biopsy and radiomics in the age of precision medicine.
L. Tselikas (2019)
10.1016/j.jbo.2019.100263
Radiomics signature extracted from diffusion-weighted magnetic resonance imaging predicts outcomes in osteosarcoma
Shu-Liang Zhao (2019)
10.1007/s00330-020-07108-w
A decade of radiomics research: are images really data or just patterns in the noise?
Daniel Pinto dos Santos (2020)
10.3348/kjr.2019.0847
Radiomics and Deep Learning from Research to Clinical Workflow: Neuro-Oncologic Imaging
J. Park (2020)
10.3390/diagnostics10060359
Imaging-Based Prediction of Molecular Therapy Targets in NSCLC by Radiogenomics and AI Approaches: A Systematic Review
G. Ninatti (2020)
10.1186/s13244-020-00866-7
Artificial intelligence abstracts from the European Congress of Radiology: analysis of topics and compliance with the STARD for abstracts checklist
Thomas Dratsch (2020)
10.1016/j.ejrad.2020.109283
Systematic review of sarcomas radiomics studies: Bridging the gap between concepts and clinical applications?
A. Crombé (2020)
10.3390/cancers12102881
Radiomics of Liver Metastases: A Systematic Review
Francesco Fiz (2020)
10.1016/j.remnie.2019.11.002
Interdisciplinarity: An essential requirement for translation of radiomics research into clinical practice -a systematic review focused on thoracic oncology.
M. Sollini (2020)
10.1007/s00330-020-06666-3
Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma—a systematic review and meta-analysis
Stephan Ursprung (2020)
10.1016/j.remn.2019.10.003
Interdisciplinaridad: un requerimiento esencial para la traslación de investigación en radiómica a la práctica clínica
M. Sollini (2020)
10.3389/fonc.2020.01301
Progress and Future Trends in PET/CT and PET/MRI Molecular Imaging Approaches for Breast Cancer
Yue Ming (2020)
10.1007/s00330-020-07370-y
The influence of cardiac motion on radiomics features: radiomics features of non-enhanced CMR cine images greatly vary through the cardiac cycle
Deniz Alis (2020)
See more
Semantic Scholar Logo Some data provided by SemanticScholar