Next Article in Journal
Patients’ Experiences of a Sarcoma Diagnosis: A Process Mapping Exercise of Diagnostic Pathways
Next Article in Special Issue
Post-Chemoradiation Metastatic, Persistent and Resistant Nodes in Locally Advanced Rectal Cancer: Metrics and Their Impact on Long-Term Outcome
Previous Article in Journal
MicroRNA-371a-3p—The Novel Serum Biomarker in Testicular Germ Cell Tumors
Previous Article in Special Issue
Preoperative Radiotherapy with a Simultaneous Integrated Boost Compared to Chemoradiotherapy for cT3-4 Rectal Cancer: Long-Term Results of a Multicenter Randomized Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 15
Issue 15
10.3390/cancers15153945
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Validated Pretreatment Prediction Models for Response to Neoadjuvant Therapy in Patients with Rectal Cancer: A Systematic Review and Critical Appraisal

by
Max D. Tanaka
1,*,
Barbara M. Geubels
2,3,4,
Brechtje A. Grotenhuis
2,
Corrie A. M. Marijnen
1,5,
Femke P. Peters
1,
Stevie van der Mierden
6,
Monique Maas
4,7 and
Alice M. Couwenberg
1
1
Department of Radiation Oncology, The Netherlands Cancer Institute, 1066 CX Amsterdam, The Netherlands
2
Department of Surgery, The Netherlands Cancer Institute, 1066 CX Amsterdam, The Netherlands
3
Department of Surgery, Catharina Hospital, 5602 ZA Eindhoven, The Netherlands
4
GROW School for Oncology and Reproduction, Maastricht University, 6200 MD Maastricht, The Netherlands
5
Department of Radiation Oncology, Leiden University Medical Centre, 2333 ZA Leiden, The Netherlands
6
Scientific Information Service, The Netherlands Cancer Institute, 1066 CX Amsterdam, The Netherlands
7
Department of Radiology, The Netherlands Cancer Institute, 1066 CX Amsterdam, The Netherlands
*
Author to whom correspondence should be addressed.
Cancers 2023, 15(15), 3945; https://doi.org/10.3390/cancers15153945
Submission received: 15 June 2023 / Revised: 27 July 2023 / Accepted: 27 July 2023 / Published: 3 August 2023
(This article belongs to the Special Issue Advances in Radiotherapy and Prognosis of Rectal Cancer)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Organ preservation strategies can be offered to patients with rectal cancer that show a strong response to preoperative treatment in order to avoid major surgery. Prediction models may help identify these patients before the start of preoperative treatment, when treatment can still be adapted. We systematically reviewed validated pretreatment prediction models for response to preoperative treatment in patients with rectal cancer. Sixteen studies were included in this review. All studies were considered to have a high risk of bias and external validation was missing. Nevertheless, some studies show promising results, which could serve as a foundation for future research. Our systematic review provides a comprehensive overview of the current state of the literature regarding pretreatment prediction models for response to preoperative treatment in patients with rectal cancer.

Abstract

Pretreatment response prediction is crucial to select those patients with rectal cancer who will benefit from organ preservation strategies following (intensified) neoadjuvant therapy and to avoid unnecessary toxicity in those who will not. The combination of individual predictors in multivariable prediction models might improve predictive accuracy. The aim of this systematic review was to summarize and critically appraise validated pretreatment prediction models (other than radiomics-based models or image-based deep learning models) for response to neoadjuvant therapy in patients with rectal cancer and provide evidence-based recommendations for future research. MEDLINE via Ovid, Embase.com, and Scopus were searched for eligible studies published up to November 2022. A total of 5006 studies were screened and 16 were included for data extraction and risk of bias assessment using Prediction model Risk Of Bias Assessment Tool (PROBAST). All selected models were unique and grouped into five predictor categories: clinical, combined, genetics, metabolites, and pathology. Studies generally included patients with intermediate or advanced tumor stages who were treated with neoadjuvant chemoradiotherapy. Evaluated outcomes were pathological complete response and pathological tumor response. All studies were considered to have a high risk of bias and none of the models were externally validated in an independent study. Discriminative performances, estimated with the area under the curve (AUC), ranged per predictor category from 0.60 to 0.70 (clinical), 0.78 to 0.81 (combined), 0.66 to 0.91 (genetics), 0.54 to 0.80 (metabolites), and 0.71 to 0.91 (pathology). Model calibration outcomes were reported in five studies. Two collagen feature-based models showed the best predictive performance (AUCs 0.83–0.91 and good calibration). In conclusion, some pretreatment models for response prediction in rectal cancer show encouraging predictive potential but, given the high risk of bias in these studies, their value should be evaluated in future, well-designed studies.

1. Introduction

Due to optimization of rectal cancer diagnosis and treatment, most notably by the introduction of total mesorectal excision (TME) and neoadjuvant therapy, local recurrence rates have been significantly reduced [1,2]. Neoadjuvant therapy is usually given for more advanced tumors, as either long-course chemoradiotherapy (CRT) or short-course radiotherapy (SCRT), to obtain downsizing of the tumor. More recently, total neoadjuvant treatment (TNT) has emerged as an alternative to decrease the number of distant metastases without compromising local control [2,3,4,5]. The positive effect of neoadjuvant therapy followed by TME on oncological outcomes has been widely recognized, although it is also associated with significant acute and long-term gastro-intestinal, urinary, and sexual morbidity [6,7].
A selected group of patients with rectal cancer shows no residual viable tumor cells in the resected specimen after neoadjuvant therapy, known as a pathological complete response (pCR) [8]. Preoperatively, a clinical complete response (cCR) can be defined with high accuracy based on digital rectal examination, rectoscopy, and MRI [9]. Patients with a cCR can be offered a “watch and wait” (W&W) strategy to avoid major surgery. Some patients with a near-cCR (ncCR) can be treated with additional treatment to avoid TME, such as local excision or contact X-ray brachytherapy (Papillion) [10,11,12], followed by a W&W approach if a complete response is achieved. Accumulating evidence shows that a W&W strategy might be considered as a safe alternative to TME surgery [6,13,14,15]. Furthermore, the W&W strategy is associated with favorable functional outcomes, good quality of life [16,17], and avoids a permanent stoma in patients with a very low tumor [18,19]. For some patients, the W&W strategy may be the optimal treatment option, and carefully comparing the advantages and challenges of this strategy versus standard TME should be part of a shared decision-making process.
The chance of achieving a complete response after neoadjuvant therapy is partly dependent on the neoadjuvant treatment regimen. In patients with locally advanced rectal cancer (LARC), CRT results in pCR rates of 10–30% [3], whereas TNT increases this chance [2,3,4,5], with better outcomes after the use of consolidation therapy versus induction therapy [20,21]. Nevertheless, TNT is associated with a higher risk of toxicity [20], which may impair patients’ quality of life. Careful selection of patients for type of neoadjuvant regimen is therefore essential.
Eligibility for a W&W strategy is assessed after completion of neoadjuvant therapy. Response assessment is generally recommended 12 weeks after the start of neoadjuvant CRT or SCRT and, in case of a ncCR, again at 16–20 weeks [22]. For TNT, timing of response assessment depends on the duration of the treatment and could take up to 38 weeks [22]. Ideally, selection of eligible patients for a W&W strategy is performed before the start of neoadjuvant therapy. Pretreatment response prediction would allow for personalized neoadjuvant approaches based on individual patient characteristics. This is particularly important now that neoadjuvant therapy is intensified and more often used in patients with early tumor stages, with the intention of achieving a higher rate of organ preservation [23,24,25,26,27,28,29]. In order to select patients who will benefit the most from (intensified) neoadjuvant therapy and avoid unnecessary toxicity in those who are ineligible for W&W but still require TME surgery, pretreatment response prediction is critical.
However, pretreatment prediction of response is highly challenging, as rectal cancer is a heterogeneous disease and response to neoadjuvant therapy can vary greatly, indicating a complex relationship between tumor characteristics and treatment response [30,31]. As a result, no individual pretreatment predictor currently has the ability to accurately select all eligible patients for organ preservation [31]. To improve predictive accuracy, individual pretreatment predictors can be combined in multivariable prediction models. Recently, several publications have shown the predictive potential of radiomics-based models (which extract numerous quantitative features from medical images [32]) and imaged-based deep learning models [32,33,34]. Other promising candidate predictors for pretreatment models include conventional clinical factors that are commonly measured during routine diagnostic work-up, like cTNM-stage and carcinoembryonic antigen (CEA) [31,35], or more novel predictors such as mismatch-repair (MMR) status now that immunotherapy shows encouraging results in MMR-deficient tumors [21]. At the moment, an overview of pretreatment prediction models that combine predictors other than image-based features is lacking.
In this systematic review, we aim to summarize and critically appraise validated pretreatment prediction models, other than radiomics-based models or image-based deep learning models, for response to neoadjuvant therapy in patients with rectal cancer and provide evidence-based recommendations for future research.

2. Materials and Methods

2.1. Reporting

This systematic review is reported according to the “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) statement [36] and the PRISMA extension for searching (PRISMA-S) [37]. The research protocol was registered in PROSPERO (CRD42023385057).

2.2. Search Strategy

MEDLINE via Ovid, Embase.com, and Scopus were searched by an information specialist (SvdM). The schematic search was as follows: Rectal Cancer AND Response AND (Neoadjuvant therapy OR CRT/SCRT/TNT). Free text terms including synonyms were used. In addition, thesaurus terms were used for MEDLINE (MeSH) and Embase (Emtree). See Supplementary Table S1 for the complete search. No filters, restrictions, or limits were used, except for the exclusion of references published prior to 1 January 2012. The search was not based on previous work and reviewed by a second information specialist. It was originally executed on 9 June 2022 and updated on 18 November 2022. All searches were performed separately on their respective platforms (i.e., no federated searches). The results were deduplicated in EndNote 20 [38], first based on PMID, second on DOI, and finally a manual check was performed. Forward and backward citation chasing (“snowballing”) was performed for the included articles. No other databases, registries, or alternative methods were used.

2.3. Eligibility Criteria

2.3.1. General Inclusion and Exclusion Criteria

Studies investigating a multivariable (at least two predictors) pretreatment prediction model for response to neoadjuvant therapy were selected if they met the following criteria: (1) rectal cancer patients without metastases (cT1-4N0-2M0), (2) adenocarcinoma, (3) neoadjuvant radiotherapy with or without additional systemic therapy, (4) outcome defined as either pCR, pathological tumor response, cCR, or ncCR followed by additional successful organ-preserving treatment, (5) at least one development or validation cohort with a sample size of 50 or more, and (6) any form of internal or external validation. The following study types were excluded: studies that included patients with mucinous, signet-ring cell or neuroendocrine tumors, studies including radiomics-based models or image-based deep learning models, non-English studies, studies published in non-peer-reviewed journals, conference abstracts, case reports, preclinical studies, (systematic) reviews, and meta-analyses.

2.3.2. Development and Validation Studies

Studies with type 1b-4 models, according to the “transparent reporting of a multivariable prediction model for individual prognosis or diagnosis” (TRIPOD) statement, were selected (Table 1) [39]. Type 1a studies do not include any form of validation and were excluded from this review. Type 1b-2 studies use a single dataset for model development and validation. Type 1b and 2a are internal validation studies. Type 1b studies use resampling methods (e.g., cross-validation or bootstrapping) and type 2a studies randomly split the data into a development and validation cohort. Type 2b studies nonrandomly split the data into a development and validation cohort, for instance, based on location (e.g., different hospital) or time (different moment of inclusion). Because of the nonrandom variation, type 2b can be seen as intermediary between internal and external validation, according to TRIPOD [39]. Type 3 and 4 are external validation studies. Type 3 studies use separate development and validation cohorts; type 4 studies do not develop a model but only validate an existing one.

2.4. Article Selection and Data Extraction

Using the eligibility criteria of Section 2.3, two independent reviewers (MDT and BMG) screened titles and abstracts, using Rayyan [40], to identify potential articles. The same independent reviewers performed subsequent full-text eligibility assessment. Reasons for exclusion of ineligible articles were recorded. Included articles had to be approved by both reviewers. Disagreements were initially resolved by discussion between the reviewers and, if necessary, with help from a third reviewer (AMC). The data extraction form was based on the TRIPOD statement [39], the 11 domains of the “Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies” (CHARMS) checklist [41], and the “Prediction model Risk Of Bias Assessment Tool” (PROBAST) [42]. Completed data extraction forms of the first five articles were discussed by MDT, BMG, and AMC to ensure that no relevant data were missed. Data extraction of the remaining articles was performed by MDT. Authors were not contacted in case of missing data. The data extraction form and extracted data have not been made publicly available.

2.5. Risk of Bias Assessment

PROBAST was used for risk of bias (ROB) assessment and evaluation of concerns regarding the applicability of a model [42,43]. ROB is assessed across four domains: participants, predictors, outcome, and analysis. Applicability is assessed across the first three domains. Each domain contains specific signaling questions to help with the final ROB and applicability rating; some questions are only related to model development studies. As suggested by PROBAST, a prediction model was considered to have high risk of bias or high concerns regarding applicability if at least one domain was judged as high. An unclear rating was given if at least one domain was judged as unclear and none of the other domains were judged as high. Consequently, a prediction model was only judged as low risk if all domains were graded as such. PROBAST forms of included articles were completed by MDT and discussed by MDT, BMG, and AMC.

2.6. Data Synthesis

For each prediction model, relevant information and ROB assessment was summarized in a descriptive synthesis, supported by tables and figures. Predictive performance measures that were reported included model discrimination and calibration. Discrimination is often quantified and reported with the area under the curve (AUC). The apparent AUC (discrimination of the model on the development cohort before internal validation) and validated AUCs were reported. An AUC of 1.0 indicates perfect discrimination. As a general rule of thumb, AUC values can be considered as poor (0.5–0.7), moderate (0.7–0.8), good (0.8–0.9), or excellent (0.9–1.0) [44]. Calibration is commonly evaluated with a goodness-of-fit test, such as the Hosmer–Lemeshow test (a p-value < 0.05 indicates miscalibration [45]), or graphically with a calibration curve, which can be quantified by the calibration slope and intercept. If a study included multiple prediction models, results of the model with the best discrimination were reported. Prediction models were categorized according to type of included predictors. No meta-analysis was performed due to expected heterogeneity.

3. Results

3.1. Search

In total, 16.129 studies were identified by our literature search. After removal of conference abstracts and deduplication, titles and abstracts of 5006 studies were screened. In total, 4881 studies were excluded, and 125 studies were assessed for eligibility through full text review, resulting in the inclusion of 15 studies. Forward and backward citation chasing of included studies identified one additional study [46]. Sixteen studies were included for data extraction and ROB assessment [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. See the PRISMA flow diagram (Figure 1) for an overview of the search process.

3.2. Overview of Study Populations

Table 2 shows the cohort characteristics of selected studies. Studies were published between 2014 and 2022, 14 in the last five years [46,48,49,51,52,53,54,55,56,57,58,59,60,61]. Patients were treated between 1998 and 2021. Sample sizes of development and validation cohorts ranged from 14 to 666 participants. Four studies consisted of at least one prospective cohort [46,50,52,55], four studies consisted of one or more multicenter cohorts [52,56,57,58], while nine studies were single-center retrospective cohort studies [47,48,49,51,53,54,59,60,61]. Tumor stages were generally intermediate or advanced (TNM stage cT3-4N0-2/stage II-III) and the neoadjuvant treatment regimen was CRT in all studies. In one study, CRT was preceded by induction chemotherapy [59] and, in three studies, CRT was followed by consolidation chemotherapy [46,55,60]. In all but one study, a standard total radiotherapy dose of 45.0–50.4 Gy was used. In the study of Joye et al., patients received a total dose of 36–55.8 Gy [47]. In most studies, capecitabine or 5-FU was administered during CRT. Almost all patients underwent TME; only in two studies a small subset of patients was treated with a local excision [51,54]. Time between the end of CRT and surgery ranged from 4 to 12 weeks.
There were 14 (87.5%) model development studies (TRIPOD type 1b–3) [46,47,48,49,50,51,52,54,55,56,57,58,59,60]. None of the model development studies were externally validated in an independent study. Eleven studies performed internal validation using resampling methods (Type 1b) [46,47,48,49,50,51,52,54,55,56,57] and fives studies by randomly splitting the data (Type 2a) [48,56,58,59,60]. An intermediary (i.e., between internal and external) form of validation was performed in one study by nonrandomly splitting the data (Type 2b) [52]. Three development studies externally validated the model in the same study (Type 3) [52,57,58]. There were two (12.5%) model-validation-only studies (Type 4) [53,61]. The two validation-only studies validated models that were originally developed for a different outcome, which did not meet our inclusion criteria.
Table 3 shows the model and outcome characteristics of the selected studies. Supplementary Table S2 describes the final predictors per model in more detail. None of the studies looked at cCR or ncCR. Model outcomes were pCR (four studies) [49,52,57,58], pathological tumor response (10 studies) [46,48,51,53,54,55,56,59,60,61], or both (two studies) [47,50]. All studies dichotomized pathological tumor response into good versus poor response or synonyms of these terms. This dichotomous outcome is hereafter referred to as good response (GR). In all but one study [50], which used ypTN stage, dichotomization was conducted using one of the following tumor regression grade (TRG) systems [62,63,64,65,66,67]: American Joint Committee on Cancer (AJCC) [46,48,55,56], Mandard [53,54], Song [59,60], Dworak [61], Quirke [47], and Gastrointestinal Pathology Study Group of the Korean Society of Pathologists [51].
Nine studies used logistic regression as a modelling method during model development, sometimes in combination with other methods [46,47,49,50,51,56,57,60,61]. Other types of modelling methods were ridge regression (RR), support vector machine (SVM), lasso and elastic-net regularized generalized linear models (R package “glmnet”), partial least squares (PLS) analysis, repeated measure analysis of variance (RM-ANOVA), multilevel partial least squares discriminant analysis (ML-PLS-DA), multi-instance learning (MIL), convolutional neural network (CNN), and graph neural network (GNN).

3.3. PROBAST Risk of Bias Assessment

Results of ROB assessment using PROBAST are shown in Table 4 and Figure 2. All studies were classified as having a high ROB, mainly because all studies had problems arising in the analysis domain. Reasons for a high ROB in the analysis domain were: 11 of 14 (79%) model development studies did not account for overfitting appropriately, mostly because resampling methods were not used for all model development procedures, including predictor selection. In 12 studies (75%), the low number of participants with the outcome (pCR/GR) compared to the number of candidate predictors could lead to a high ROB. Four of fourteen (29%) model development studies did not avoid predictor selection based on univariable analysis and three (19%) studies did not handle continuous and/or categorical predictors appropriately. Moreover, there was no information about handling of missing data and calibration in 13 (81%) and 11 (69%) studies, respectively, and the complete final model formula was not given in 13 (93%) of 14 model development studies. A high risk of bias in the predictors domain was seen in three studies because of reported heterogeneity in quality of biopsies.
There were only minor concerns with regard to applicability of the included studies: one study had high concerns and two studies had unclear concerns regarding applicability.

3.4. Model Results

3.4.1. General

AUCs of all studies ranged from poor to excellent (0.51–0.93) (Figure 3). AUCs were between 0.60 and 0.91 for the outcome pCR and 0.51 and 0.93 for the outcome GR. Apparent performance (discrimination of the model on the development cohort before internal validation) was reported in six of fourteen model development studies [51,54,56,57,59,60]. Five studies [49,54,56,57,60] described model calibration using the Hosmer–Lemeshow goodness-of-fit test, a calibration curve, or both. Calibration slopes and intercepts were not reported. Based on the predictors included in the final model, studies were grouped into the following five categories: clinical, combined (clinical, serological, and imaging), genetics, metabolites, and pathology. Besides the combined prediction model from Buijsen et al. [50], all models used predictors from a single category or combined those predictors only with standard clinical predictors.

3.4.2. Clinical Prediction Models

Three studies developed a prediction model using clinical predictors only [47,48,49], with poor–moderate AUCs, ranging from 0.60 to 0.70. Joye et al. [47] developed and internally validated a model in one cohort (n = 620) for the outcomes pCR and good response (GR). Following backward predictor selection, the pCR model included six (age, American Society of Anesthesiologists (ASA) score, CEA, cN stage, gender, and hemoglobin (Hb)) and the GR model four (CEA, cT stage/mesorectal fascia (MRF), cN stage, and Hb) clinical predictors. Apparent AUCs were not reported. After bootstrapping, the pCR model had an AUC of 0.60 (range 0.56–0.62) and the GR model 0.63 (range 0.62–0.64). Kim et al. [48] evaluated a model on GR and randomly split the data into a development (n = 190), tuning (n = 41), and validation (n = 41) set. The clinical model included 10 predictors: age, alcohol, ASA, BMI, diabetes, distance from anal verge, gender, hypertension, smoking, and tumor grade. AUCs for the prediction of GR were 0.65 (mean ± std: 0.02), 0.53 (mean ± std 0.08), and 0.51 (mean ± std 0.08) in the training, tuning, and validation cohort. Kim et al. also developed models with additional serological predictors measured during CRT, of which the best model had slightly better discriminative performance than the pretreatment model. The study by Ren et al. [49] evaluated a model on pCR after CRT with mFOLFOX in 126 patients that included two predictors after forward predictor selection: MRF and tumor length. The apparent AUC was not reported; after bootstrapping the AUC was 0.70 (95% CI 0.61–0.79). Model calibration performance was evaluated with a calibration curve, which seemed to indicate miscalibration in the 0.2–0.4 predicted probability range.

3.4.3. Combined Clinical, Serological and Imaging Prediction Model

Only one study (Buijsen et al.) [50] combined pretreatment predictors of more than two different categories in a model. Four clinical (CEA, cT stage, cN stage, and tumor length), three serological (interleukin (IL)-6, IL-8, and osteopontin), and one imaging (maximal standardized uptake value (SUVmax)) predictor(s) were selected. Data of 276 patients were used for development and internal validation of the model, predicting both pCR and GR. Apparent AUCs were not reported; bootstrapped AUCs for pCR and GR were good and moderate: 0.81 (95% CI 0.73–0.88) and 0.78 (95% CI 0.71–0.85), respectively.

3.4.4. Genetics Prediction Models

Four models evaluated gene-based prediction models [51,52,53,54]. AUCs ranged widely, from poor to excellent (range 0.66–0.91). None of the genes were used in more than one model. The studies had small sample sizes (n = 24–184); only two studies used a cohort that consisted of more than 100 patients. Cho et al. [51] developed an eight-gene mRNA radio-response prediction index for GR using stepwise predictor selection in a cohort that included 184 patients, with an apparent AUC of 0.85 (95% CI 0.80–0.90) and AUCs after cross-validation that ranged from 0.81 (95% CI 0.71–0.91) to 0.91 (95% CI 0.84–0.98). Emons et al. [52] developed a 21-transcript signature for pCR prediction in a cohort of 64 patients using hill-climbing predictor selection. The apparent AUC was not reported. Several cohorts (n = 14–161) were used for internal and external validation, with AUCs between 0.70 and 0.81. The transient receptor potential channels (TRPC) score of Wang et al. [53], which reflects the expression of eight TRPG related genes, was developed using patients with colorectal cancer for the outcome overall survival. However, in the same article, the score was externally validated in two separate rectal cancer cohorts (n = 80 and n = 85) for the outcome GR, with AUCs of 0.66 and 0.71, respectively. In the study by Wei et al. [54], four immune-related differentially expressed genes were selected using least absolute shrinkage and selection operator (LASSO) regression and combined with two clinical predictors (age and gender) in a prediction nomogram. The analyzed cohort consisted of 59 patients that received chemotherapy as either 5-FU or capecitabine with or without oxaliplatin. The apparent AUC for GR was 0.82 and after bootstrapping 0.75. The graphically displayed calibration curve seemed to demonstrate slight miscalibration, especially in the higher predicted probability ranges.

3.4.5. Metabolites Prediction Models

Two studies evaluated metabolite-based models, Jia et al. (2018) and Lv et al. (2022) [46,55]. Both cohorts (n = 105 and n = 106, respectively) were from the same hospital which treated patients with one additional consolidation cycle of chemotherapy after CRT. The studies used the same methods for metabolite analysis but included different metabolites in the final models. Both used the outcome GR. Apparent AUCs were not reported. The 15-metabolite panel from Jia et al. [55] had a good AUC of 0.80 (95% CI 0.67–0.91) after bootstrapping and the eight-metabolite panel from Lv et al. [46] had a poor AUC of 0.54 (95% CI 0.43–0.65) after cross-validation. Lv et al. measured and evaluated metabolites at multiple time points; the last measurement was within two days before surgery. The use of multiple measurements improved the cross-validated discriminative performance slightly.

3.4.6. Pathology Prediction Models

There were six pathology-based prediction models [56,57,58,59,60,61]. AUCs were between 0.71 (good) and 0.91 (excellent) [56,57,58,59,60]. Jiang et al. published two collagen feature models, in 2021 [56] on GR and 2022 on pCR [57]. Both studies used data from patients in the same three hospitals, collected between 2010 and 2018. The collagen features were analyzed in the same manner using multiphoton images. One collagen feature was used in both models. In the study published in 2021, data were randomly split in a development (n = 299) and validation (n = 129) cohort. The authors first combined three collagen features in a Collagen Features Support Vector Machine (CFs-SVM) classifier, after which the CFs-SVM classifier was combined with three clinical predictors (CEA, cT stage, and tumor differentiation) in a nomogram for GR. Without the addition of clinical predictors, the CFs-SVM classifier had an AUC of 0.78 (95% CI 0.72–0.83) in the development and 0.77 (95% CI 0.68–0.85) in the validation cohort. The apparent AUC of the combined nomogram was 0.83 (95% CI 0.79–0.88). The nomogram was validated in the development and validation cohorts with AUCs of 0.84 and 0.85 (95% CI 0.79–0.92). Then, the nomogram was validated in a cohort that used all patients, from the development as well as the validation cohort, with an AUC of 0.84. Calibration curves were good and the Hosmer–Lemeshow (H-L) goodness-of-fit test results, with p-values of 0.096 (development) and 0.496 (validation), suggested no miscalibration. The dataset of the study published in 2022 consisted of a development (n = 353) and external validation (n = 163) cohort. The pCR model combined four collagen features (selected with LASSO regression) with four clinical predictors (CEA, cT stage, tumor differentiation, and tumor dimension (length)). The collagen signature, before combination with the clinical predictors, had an AUC of 0.84 (95% CI 0.79–0.90) in the development and 0.84 (95% CI 0.75–0.92) in the validation cohort. The apparent AUC of the final model that included clinical predictors was 0.89 (95% CI 0.85–0.94), with validated AUCs between 0.89 and 0.91 (95% CI 0.86–0.96). Calibration curves showed good calibration and there was no suggestion for miscalibration based on the p-values from the H-L goodness-of-fit test, which were 0.314 in the development and 0.670 in the validation cohort.
Three models evaluated the predictive value of pathological whole slide images (WSIs). Lou et al. [58] developed a deep learning model with 666, 117, and 102 participants in the training, testing, and external validation cohort, respectively. The apparent AUC was not reported. AUCs for pCR in the testing and external validation cohorts were 0.71 (95% CI 0.60–0.81) and 0.72 (95% CI 0.59–0.84). Wang et al. published a digital-pathology-based deep learning model [59]. Patients in the development (n = 55) and randomly split (n = 14) cohorts were treated with induction chemotherapy (FOLFOX or CAPOX) and CRT. The evaluated model used a maximum of 500 samples from collected WSIs, which were processed and analyzed using a convolutional neural network and a graph neural network. The model showed an apparent AUC of 0.78 and an AUC of 0.73 in the randomly split cohort. Zhang et al. [60] developed a pathology signature with 17 predictors, selected from WSIs using LASSO, trained for the outcome GR. A total of 151 patients were randomly split in a development (n = 120) and validation (n = 31) dataset. A subset of 78 (52%) patients received two additional cycles of consolidation chemotherapy (capecitabine or 5-FU/leucovorin) after CRT. It was unclear how many patients with GR received consolidation chemotherapy. The apparent AUC was 0.93 (95% CI 0.88–0.97) and the AUC of the nonrandomly split cohort 0.88 (95% CI 0.72–0.97). Calibration curves showed slight miscalibration in the lower and higher predicted probability ranges, although this was not seen in the p-values of the H-L goodness-of-fit test (0.332 and 0.213).
One study (Huang et al.) [61] externally validated the immunoscore, a prediction model that uses tumor-infiltrating lymphocytes (TILs). They evaluated the score using CD3+ and CD8+ TILs in a small cohort of rectal cancer patients (n = 55). The AUC for GR was 0.72. It is important to note that the immunoscore was originally not developed to predict the response to neoadjuvant therapy but, rather, to evaluate prognostic outcomes in colorectal patients who underwent surgery without neoadjuvant therapy [68,69].

4. Discussion

In this systematic review we summarized and critically appraised 16 validated pretreatment prediction models for response to neoadjuvant therapy in patients with rectal cancer. The models were grouped into five categories and included several promising predictors, with some models showing encouraging predictive potential. However, calibration outcomes were only reported in five studies and PROBAST indicated that all studies had a high risk of bias (ROB), mainly in the analysis domain. In addition, the majority of studies used small sample sizes, and external validation in independent studies was lacking. Based on these findings, we propose some recommendations for future research.
The two collagen feature-based models from Jiang et al. [56,57] showed the most promising results of all studies. The models reported good to excellent discriminative performance (AUCs 0.83–0.91) with relatively small confidence intervals, good calibration, and consistency across cohorts. The discriminative performance of both collagen-only signatures improved with the addition of clinical predictors, suggesting that the collagen features may be complementary to standard clinical predictors, rather than explain similar aspects of the tumor. Furthermore, multiphoton imaging for collagen feature analysis offers some advantages, such as rapid analysis using routinely collected pretreatment biopsies and a quick learning curve for newly trained pathologists [56,57,70,71]. In addition, comparable pretreatment collagen feature models showed good and excellent discrimination for prognostic outcomes in colon cancer (n = 882, disease-free survival/overall survival) and gastric cancer (n = 375, lymph node metastasis), respectively [71,72]. Thus, collagen feature-based prediction models are good potential candidates for further (external) validation, despite the high ROB of the included studies.
The pathological whole slide image (WSI)-based models [58,59,60], immunoscore [61], and metabolite-based models [46,55] reported poor to excellent discriminative performance (AUCs 0.54–0.93). Due to the heterogeneous results, small sample sizes, and high ROB, it is challenging to determine their exact predictive value. Consequently, we cannot confidently recommend these models for further validation. Nevertheless, the included predictors remain interesting for future research. Digital WSIs of pretreatment biopsies can be used to quantify pathological features [59,73]. WSI analysis has the potential to be easily incorporated in routine clinical practice, since standard rectal cancer biopsies can be used and the annotation workload for the pathologist is significantly reduced with new analysis and modelling methods [59,60,74]. Metabolomics is defined as the analysis of intermediate or end products of metabolic processes (metabolites) [75]. It is an easily obtained and minimally invasive method for response prediction, with serum as the preferred sample for analysis, but the literature about its role for the prediction of treatment response in patients with rectal cancer is scarce [76]. The immunoscore was originally developed [68] and later validated [69] for prognostic outcomes in colorectal patients undergoing surgery without neoadjuvant therapy. The score combines the densities of two tumor-infiltrating lymphocytes (TILs) populations at the center of the tumor and at the invasive margin. In addition to the included study from Huang et al. [61], El Sissy et al. [77] validated the pretreatment immunoscore in a cohort of 249 rectal cancer patients and showed a positive correlation of the immunoscore with tumor response and significantly higher pCR rates in patients with a high immunoscore. This study was excluded for data extraction and ROB assessment in the present review because tumors other than adenocarcinoma were included. The immunoscore has strong interobserver reproducibility [69] but the exact value of the immunoscore for pretreatment response prediction remains to be determined.
The four gene-based models [51,52,53,54] showed widely varying discriminative performances, ranging from poor to excellent (AUCs 0.66–0.91). Small sample sizes were used, a common issue in rectal cancer gene-based prediction model studies [30,31,78,79]. A second frequently occurring problem is the lack of overlap between genes in different studies [51,80]. None of the genes from the models included in this review were used in more than one model. Moreover, Cho et al. [51] compared their eight-mRNA gene model with 11 other gene expression studies and found that only one of their eight mRNA genes showed predictive value in other studies. Furthermore, a systematic review by Izzotti et al. identified 674 mRNAs and 77 microRNAs with potential predictive value for response to neoadjuvant therapy in colorectal cancer patients and only 19 mRNAs and 6 microRNAs were differentially expressed in more than one study [81]. The results of the gene-based models included in this review confirm the need for larger sample sizes and genes that show consistent predictive value across multiple prediction models.
We did not include radiomics-based and image-based deep learning models in this review because of the already available recent reviews. Two systematic reviews evaluated the predictive value of pretreatment radiomics-based models. Staal et al. evaluated 24 studies [32] and Di Re et al. 9 studies [34]. AUCs ranged from 0.47 to 0.99. A meta-analysis from 2022 by Jia et al. [33] evaluated 21 MRI radiomics-based models and image-based deep learning models. The pooled AUC of the validation cohorts was excellent (0.92 with 95% CI: 0.88–0.93), although a small number of studies also included models with both pre- and post-treatment predictors. As shown by these systematic reviews, the number of radiomics-based and image-based deep learning models is high and good–excellent AUCs were often reported. However, the models suffer from similar problems as those included in the current review, such as a high ROB, reproducibility issues, and a lack of external validation [32,34]. Consequently, radiomics-based models and image-based deep learning models are not ready for clinical practice, pending external validation in large multicenter prospective trials.
PROBAST indicated a high ROB in all studies, mainly because all studies had a high ROB in the analysis domain. We obtained these results despite only selecting studies that reported some form of internal or external validation. The results are in line with other systematic reviews that use PROBAST, as is shown in a recent meta-review of 50 systematic reviews that used PROBAST. Included studies were often considered to have a high or unclear ROB, mostly because of problems in the analysis domain [82]. The negative effect of a high ROB is quantified by Venema et al. [83], who rated 102 prediction models with a shortened form that included six PROBAST items. The median change in discrimination (AUC) between development and validation cohorts was significantly higher in high ROB models (−11.7%) compared to low ROB models (−0.9%). Large sample sizes that include a sufficient number of participants with the outcome are critical to minimize ROB. Twelve of sixteen (75%) included studies had a low number of participants with the outcome (pCR/GR) compared to the number of candidate predictors. A sample size calculation can be helpful to identify the maximum number of candidate predictors or to identify a dataset that is too small for a particular research question [84]. A problem such as the effect of not properly accounting for overfitting, which was seen in most included development studies, might even be negligible if a dataset is used with a sufficient number of participants with the outcome [83]. Unfortunately, even in large clinical rectal cancer trials, it is difficult to include a reasonable number of patients with pCR or GR, let alone patients with a (near) cCR. Alternatively, data collection in prospective observational cohort studies may improve sample size numbers.
Our study has some limitations. We could have missed interesting studies by excluding those that included patients with mucinous, signet-ring cell, or neuroendocrine tumors. However, these tumors may respond differently to neoadjuvant therapy than standard adenocarcinomas because of distinct tumor biology [85,86,87,88], which would have led to more heterogeneous results. Moreover, by excluding radiomics-based and image-based deep learning models, we might have missed potential studies that combine these models with predictors from other categories. Even so, we believe that most relevant models were included in the reviews of Di Re, Staal, and Jia [32,33,34], since these reviews also included models that added predictors from different categories.
The substantial ROB of selected studies in this review prevents the implementation of these models in a clinical setting. However, our findings indicate that some pretreatment prediction models, in particular the collagen-feature models and perhaps the genetics-, metabolites-, and pathology-based models, could have important clinical value in the future. Several recommendations can be given to guide future research on the additional clinical value of these models. The models should be further developed and externally validated in independent cohorts with a sufficient sample size, and appropriate statistical plans should be designed to minimize bias. Furthermore, easily obtainable predictors should be used that are measured with standardized methods to allow reproducibility of the results. In addition, the models need to be validated for organ preservation outcomes instead of postoperative outcomes given the conflicting results that exist regarding the concordance between cCR and pCR [9,89,90]. Finally, to improve generalizability, investigated patient populations should also include patients with early tumor stages and neoadjuvant treatment regimens other than CRT. These steps might enable us to select the most promising models that should be evaluated in the clinic and discard those models that are not worthy of further investigation.

5. Conclusions

This systematic review aimed to summarize and critically appraise validated pretreatment prediction models for response to neoadjuvant therapy in patients with rectal cancer and provide evidence-based recommendations for future research. Results were heterogeneous, all studies were considered to have a high risk of bias, and external validation in independent studies was lacking. Despite these limitations, several promising predictors were identified, and some models demonstrated encouraging predictive potential, in particular the collagen feature-based models. These studies could form the basis for future research, which should focus on the reduction of bias, evaluate reproducible predictors, and use outcomes specifically tailored to organ preservation. Furthermore, it is crucial that these models undergo external validation in independent studies in order to reach clinical value.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers15153945/s1, Table S1: Complete search. Table S2: Overview of final predictors per model.

Author Contributions

Conceptualization, M.D.T., B.M.G., B.A.G., C.A.M.M., F.P.P., S.v.d.M., M.M. and A.M.C.; methodology, M.D.T., B.M.G., B.A.G., C.A.M.M., F.P.P., S.v.d.M., M.M. and A.M.C.; formal analysis, M.D.T. and B.M.G.; investigation, M.D.T., B.M.G., S.v.d.M. and A.M.C.; data curation, M.D.T. and B.M.G.; writing—original draft preparation, M.D.T.; writing—review and editing, B.M.G., B.A.G., C.A.M.M., F.P.P., S.v.d.M., M.M. and A.M.C.; visualization, M.D.T.; supervision, B.A.G., C.A.M.M., F.P.P., M.M. and A.M.C.; project administration, M.D.T. and B.M.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Gao, Z.; Gu, J. Surgical treatment of locally recurrent rectal cancer: A narrative review. Ann. Transl. Med. 2021, 9, 1026. [Google Scholar] [CrossRef] [PubMed]
  2. Kong, J.C.; Soucisse, M.; Michael, M.; Tie, J.; Ngan, S.Y.; Leong, T.; McCormick, J.; Warrier, S.K.; Heriot, A.G. Total Neoadjuvant Therapy in Locally Advanced Rectal Cancer: A Systematic Review and Metaanalysis of Oncological and Operative Outcomes. Ann. Surg. Oncol. 2021, 28, 7476–7486. [Google Scholar] [CrossRef] [PubMed]
  3. Liu, S.; Jiang, T.; Xiao, L.; Yang, S.; Liu, Q.; Gao, Y.; Chen, G.; Xiao, W. Total Neoadjuvant Therapy (TNT) versus Standard Neoadjuvant Chemoradiotherapy for Locally Advanced Rectal Cancer: A Systematic Review and Meta-Analysis. Oncologist 2021, 26, e1555–e1566. [Google Scholar] [CrossRef]
  4. Zhang, X.; Ma, S.; Guo, Y.; Luo, Y.; Li, L. Total neoadjuvant therapy versus standard therapy in locally advanced rectal cancer: A systematic review and meta-analysis of 15 trials. PLoS ONE 2022, 17, e0276599. [Google Scholar] [CrossRef]
  5. Donnelly, M.; Ryan, O.K.; Ryan, É.J.; Creavin, B.; O’Reilly, M.; McDermott, R.; Kennelly, R.; Hanly, A.; Martin, S.T.; Winter, D.C. Total neoadjuvant therapy versus standard neoadjuvant treatment strategies for the management of locally advanced rectal cancer: Network meta-analysis of randomized clinical trials. Br. J. Surg. 2023, znad177. [Google Scholar] [CrossRef] [PubMed]
  6. van der Valk, M.J.M.; Hilling, D.E.; Bastiaannet, E.; Meershoek-Klein Kranenbarg, E.; Beets, G.L.; Figueiredo, N.L.; Habr-Gama, A.; Perez, R.O.; Renehan, A.G.; van de Velde, C.J.H. Long-term outcomes of clinical complete responders after neoadjuvant treatment for rectal cancer in the International Watch & Wait Database (IWWD): An international multicentre registry study. Lancet 2018, 391, 2537–2545. [Google Scholar] [CrossRef] [Green Version]
  7. Birgisson, H.; Påhlman, L.; Gunnarsson, U.; Glimelius, B. Late adverse effects of radiation therapy for rectal cancer—A systematic overview. Acta Oncol. 2007, 46, 504–516. [Google Scholar] [CrossRef]
  8. Maas, M.; Nelemans, P.J.; Valentini, V.; Das, P.; Rödel, C.; Kuo, L.J.; Calvo, F.A.; García-Aguilar, J.; Glynne-Jones, R.; Haustermans, K.; et al. Long-term outcome in patients with a pathological complete response after chemoradiation for rectal cancer: A pooled analysis of individual patient data. Lancet Oncol. 2010, 11, 835–844. [Google Scholar] [CrossRef]
  9. Maas, M.; Lambregts, D.M.; Nelemans, P.J.; Heijnen, L.A.; Martens, M.H.; Leijtens, J.W.; Sosef, M.; Hulsewé, K.W.; Hoff, C.; Breukink, S.O.; et al. Assessment of Clinical Complete Response After Chemoradiation for Rectal Cancer with Digital Rectal Examination, Endoscopy, and MRI: Selection for Organ-Saving Treatment. Ann. Surg. Oncol. 2015, 22, 3873–3880. [Google Scholar] [CrossRef] [Green Version]
  10. Garcia-Aguilar, J.; Renfro, L.A.; Chow, O.S.; Shi, Q.; Carrero, X.W.; Lynn, P.B.; Thomas, C.R., Jr.; Chan, E.; Cataldo, P.A.; Marcet, J.E.; et al. Organ preservation for clinical T2N0 distal rectal cancer using neoadjuvant chemoradiotherapy and local excision (ACOSOG Z6041): Results of an open-label, single-arm, multi-institutional, phase 2 trial. Lancet Oncol. 2015, 16, 1537–1546. [Google Scholar] [CrossRef] [Green Version]
  11. Gerard, J.P.; Myint, A.S.; Barbet, N.; Dejean, C.; Thamphya, B.; Gal, J.; Montagne, L.; Vuong, T. Targeted Radiotherapy Using Contact X-ray Brachytherapy 50 kV. Cancers 2022, 14, 1313. [Google Scholar] [CrossRef] [PubMed]
  12. NCT05772923: Organ Preservation in Rectal Cancer: Contact X-ray Brachytherapy vs. Extending the Waiting Interval and Local Excision (OPAXX). Available online: ClinicalTrials.gov (accessed on 17 April 2023).
  13. Dossa, F.; Chesney, T.R.; Acuna, S.A.; Baxter, N.N. A watch-and-wait approach for locally advanced rectal cancer after a clinical complete response following neoadjuvant chemoradiation: A systematic review and meta-analysis. Lancet Gastroenterol. Hepatol. 2017, 2, 501–513. [Google Scholar] [CrossRef] [PubMed]
  14. Temmink, S.J.D.; Peeters, K.C.M.J.; Bahadoer, R.R.; Kranenbarg, E.M.-K.; Roodvoets, A.G.H.; Melenhorst, J.; Burger, J.W.A.; Wolthuis, A.; Renehan, A.G.; Figueiredo, N.L.; et al. Watch and wait after neoadjuvant treatment in rectal cancer: Comparison of outcomes in patients with and without a complete response at first reassessment in the International Watch &Wait Database (IWWD). Br. J. Surg. 2023, 110, 676–684. [Google Scholar] [CrossRef]
  15. Renehan, A.G.; Malcomson, L.; Emsley, R.; Gollins, S.; Maw, A.; Myint, A.S.; Rooney, P.S.; Susnerwala, S.; Blower, A.; Saunders, M.P.; et al. Watch-and-wait approach versus surgical resection after chemoradiotherapy for patients with rectal cancer (the OnCoRe project): A propensity-score matched cohort analysis. Lancet Oncol. 2016, 17, 174–183. [Google Scholar] [CrossRef]
  16. Hupkens, B.J.P.; Martens, M.H.; Stoot, J.H.; Berbee, M.; Melenhorst, J.; Beets-Tan, R.G.; Beets, G.L.; Breukink, S.O. Quality of Life in Rectal Cancer Patients After Chemoradiation: Watch-and-Wait Policy Versus Standard Resection—A Matched-Controlled Study. Dis. Colon. Rectum. 2017, 60, 1032–1040. [Google Scholar] [CrossRef]
  17. Custers, P.A.; van der Sande, M.E.; Grotenhuis, B.A.; Peters, F.P.; van Kuijk, S.M.J.; Beets, G.L.; Breukink, S.O.; Consortium, D.W.-A.-W. Long-term Quality of Life and Functional Outcome of Patients With Rectal Cancer Following a Watch-and-Wait Approach. JAMA Surg. 2023, 158, e230146. [Google Scholar] [CrossRef] [PubMed]
  18. van der Valk, M.J.M.; van der Sande, M.E.; Toebes, R.E.; Breukink, S.O.; Bröker, M.E.E.; Doornebosch, P.G.; Maliko, N.; Neijenhuis, P.A.; Marinelli, A.; Peters, F.P.; et al. Importance of patient reported and clinical outcomes for patients with locally advanced rectal cancer and their treating physicians. Do clinicians know what patients want? Eur. J. Surg. Oncol. 2020, 46, 1634–1641. [Google Scholar] [CrossRef] [PubMed]
  19. Wrenn, S.M.; Cepeda-Benito, A.; Ramos-Valadez, D.I.; Cataldo, P.A. Patient Perceptions and Quality of Life After Colon and Rectal Surgery: What Do Patients Really Want? Dis. Colon. Rectum. 2018, 61, 971–978. [Google Scholar] [CrossRef]
  20. Aschele, C.; Glynne-Jones, R. Selecting a TNT Schedule in Locally Advanced Rectal Cancer: Can We Predict Who Actually Benefits? Cancers 2023, 15, 2567. [Google Scholar] [CrossRef] [PubMed]
  21. Couwenberg, A.M.; Varvoglis, D.N.; Grieb, B.C.; Marijnen, C.A.M.; Ciombor, K.K.; Guillem, J.G. New Opportunities for Minimizing Toxicity in Rectal Cancer Management. Am. Soc. Clin. Oncol. Educ. Book 2023, 43, e389558. [Google Scholar] [CrossRef]
  22. Fokas, E.; Appelt, A.; Glynne-Jones, R.; Beets, G.; Perez, R.; Garcia-Aguilar, J.; Rullier, E.; Joshua Smith, J.; Marijnen, C.; Peters, F.P.; et al. International consensus recommendations on key outcome measures for organ preservation after (chemo)radiotherapy in patients with rectal cancer. Nat. Rev. Clin. Oncol. 2021, 18, 805–816. [Google Scholar] [CrossRef] [PubMed]
  23. Bach, S.P. Can we Save the rectum by watchful waiting or TransAnal surgery following (chemo)Radiotherapy versus Total mesorectal excision for early REctal Cancer (STAR-TREC)? Protocol for the international, multicentre, rolling phase II/III partially randomized patient preference trial evaluating long-course concurrent chemoradiotherapy versus short-course radiotherapy organ preservation approaches. Color. Dis. 2022, 24, 639–651. [Google Scholar] [CrossRef]
  24. van den Berg, K.; Schaap, D.P.; Voogt, E.L.K.; Buffart, T.E.; Verheul, H.M.W.; de Groot, J.W.B.; Verhoef, C.; Melenhorst, J.; Roodhart, J.M.L.; de Wilt, J.H.W.; et al. Neoadjuvant FOLFOXIRI prior to chemoradiotherapy for high-risk (“ugly”) locally advanced rectal cancer: Study protocol of a single-arm, multicentre, open-label, phase II trial (MEND-IT). BMC Cancer 2022, 22, 957. [Google Scholar] [CrossRef] [PubMed]
  25. Bach, S.P.; Gilbert, A.; Brock, K.; Korsgen, S.; Geh, I.; Hill, J.; Gill, T.; Hainsworth, P.; Tutton, M.G.; Khan, J.; et al. Radical surgery versus organ preservation via short-course radiotherapy followed by transanal endoscopic microsurgery for early-stage rectal cancer (TREC): A randomised, open-label feasibility study. Lancet Gastroenterol. Hepatol. 2021, 6, 92–105. [Google Scholar] [CrossRef]
  26. Bökkerink, G.M.; de Graaf, E.J.; Punt, C.J.; Nagtegaal, I.D.; Rütten, H.; Nuyttens, J.J.; van Meerten, E.; Doornebosch, P.G.; Tanis, P.J.; Derksen, E.J.; et al. The CARTS study: Chemoradiation therapy for rectal cancer in the distal rectum followed by organ-sparing transanal endoscopic microsurgery. BMC Surg. 2011, 11, 34. [Google Scholar] [CrossRef] [Green Version]
  27. Garcia-Aguilar, J.; Patil, S.; Gollub, M.J.; Kim, J.K.; Yuval, J.B.; Thompson, H.M.; Verheij, F.S.; Omer, D.M.; Lee, M.; Dunne, R.F.; et al. Organ Preservation in Patients With Rectal Adenocarcinoma Treated With Total Neoadjuvant Therapy. J. Clin. Oncol. 2022, 40, 2546–2556. [Google Scholar] [CrossRef]
  28. Gerard, J.-P.; Barbet, N.; Schiappa, R.; Magné, N.; Martel, I.; Mineur, L.; Deberne, M.; Zilli, T.; Dhadda, A.; Myint, A.S. Neoadjuvant chemoradiotherapy with radiation dose escalation with contact x-ray brachytherapy boost or external beam radiotherapy boost for organ preservation in early cT2–cT3 rectal adenocarcinoma (OPERA): A phase 3, randomised controlled trial. Lancet Gastroenterol. Hepatol. 2023, 8, 356–367. [Google Scholar] [CrossRef]
  29. NCT04246684: Short RT Versus RCT, Followed by Chemo. and Organ Preservation for Interm and High-Risk Rectal Cancer Patients. Available online: ClinicalTrials.gov (accessed on 23 April 2023).
  30. Ryan, J.E.; Warrier, S.K.; Lynch, A.C.; Ramsay, R.G.; Phillips, W.A.; Heriot, A.G. Predicting pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: A systematic review. Color. Dis. 2016, 18, 234–246. [Google Scholar] [CrossRef]
  31. Fischer, J.; Eglinton, T.W.; Richards, S.J.; Frizelle, F.A. Predicting pathological response to chemoradiotherapy for rectal cancer: A systematic review. Expert Rev. Anticancer Ther. 2021, 21, 489–500. [Google Scholar] [CrossRef]
  32. Staal, F.C.R.; van der Reijd, D.J.; Taghavi, M.; Lambregts, D.M.J.; Beets-Tan, R.G.H.; Maas, M. Radiomics for the Prediction of Treatment Outcome and Survival in Patients With Colorectal Cancer: A Systematic Review. Clin. Color. Cancer 2021, 20, 52–71. [Google Scholar] [CrossRef]
  33. Jia, L.L.; Zheng, Q.Y.; Tian, J.H.; He, D.L.; Zhao, J.X.; Zhao, L.P.; Huang, G. Artificial intelligence with magnetic resonance imaging for prediction of pathological complete response to neoadjuvant chemoradiotherapy in rectal cancer: A systematic review and meta-analysis. Front. Oncol. 2022, 12, 1026216. [Google Scholar] [CrossRef] [PubMed]
  34. Di Re, A.M.; Sun, Y.; Sundaresan, P.; Hau, E.; Toh, J.W.T.; Gee, H.; Or, M.; Haworth, A. MRI radiomics in the prediction of therapeutic response to neoadjuvant therapy for locoregionally advanced rectal cancer: A systematic review. Expert Rev. Anticancer Ther. 2021, 21, 425–449. [Google Scholar] [CrossRef] [PubMed]
  35. Huang, Y.; Lee, D.; Young, C. Predictors for complete pathological response for stage II and III rectal cancer following neoadjuvant therapy—A systematic review and meta-analysis. Am. J. Surg. 2020, 220, 300–308. [Google Scholar] [CrossRef] [PubMed]
  36. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef] [PubMed]
  37. Rethlefsen, M.L.; Kirtley, S.; Waffenschmidt, S.; Ayala, A.P.; Moher, D.; Page, M.J.; Koffel, J.B. PRISMA-S: An extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst. Rev. 2021, 10, 39. [Google Scholar] [CrossRef]
  38. The EndNote Team. EndNote, EndNote 20; Clarivate: Philadelphia, PA, USA, 2013. [Google Scholar]
  39. Collins, G.S.; Reitsma, J.B.; Altman, D.G.; Moons, K.G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMJ 2015, 350, g7594. [Google Scholar] [CrossRef] [Green Version]
  40. Ouzzani, M.; Hammady, H.; Fedorowicz, Z.; Elmagarmid, A. Rayyan—A web and mobile app for systematic reviews. Syst. Rev. 2016, 5, 210. [Google Scholar] [CrossRef] [Green Version]
  41. Moons, K.G.; de Groot, J.A.; Bouwmeester, W.; Vergouwe, Y.; Mallett, S.; Altman, D.G.; Reitsma, J.B.; Collins, G.S. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The CHARMS checklist. PLoS Med. 2014, 11, e1001744. [Google Scholar] [CrossRef] [Green Version]
  42. Wolff, R.F.; Moons, K.G.M.; Riley, R.D.; Whiting, P.F.; Westwood, M.; Collins, G.S.; Reitsma, J.B.; Kleijnen, J.; Mallett, S. PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Ann. Intern. Med. 2019, 170, 51–58. [Google Scholar] [CrossRef] [Green Version]
  43. Moons, K.G.M.; Wolff, R.F.; Riley, R.D.; Whiting, P.F.; Westwood, M.; Collins, G.S.; Reitsma, J.B.; Kleijnen, J.; Mallett, S. PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. Ann. Intern. Med. 2019, 170, W1–W33. [Google Scholar] [CrossRef] [Green Version]
  44. Nahm, F.S. Receiver operating characteristic curve: Overview and practical use for clinicians. Korean J. Anesth. 2022, 75, 25–36. [Google Scholar] [CrossRef] [PubMed]
  45. Huang, Y.; Li, W.; Macheret, F.; Gabriel, R.A.; Ohno-Machado, L. A tutorial on calibration measurements and calibration models for clinical prediction models. J. Am. Med. Inform. Assoc. 2020, 27, 621–633. [Google Scholar] [CrossRef]
  46. Lv, J.; Jia, H.; Mo, M.; Yuan, J.; Wu, Z.; Zhang, S.; Zhe, F.; Gu, B.; Fan, B.; Li, C.; et al. Changes of serum metabolites levels during neoadjuvant chemoradiation and prediction of the pathological response in locally advanced rectal cancer. Metabolomics 2022, 18, 99. [Google Scholar] [CrossRef] [PubMed]
  47. Joye, I.; Debucquoy, A.; Fieuws, S.; Wolthuis, A.; Sagaert, X.; D’Hoore, A.; Haustermans, K. Can clinical factors be used as a selection tool for an organ-preserving strategy in rectal cancer? Acta Oncol. 2016, 55, 1047–1052. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Kim, J.; Sohn, K.A.; Kwak, J.H.; Kim, M.J.; Ryoo, S.B.; Jeong, S.Y.; Park, K.J.; Kang, H.C.; Chie, E.K.; Jung, S.H.; et al. A Novel Scoring System for Response of Preoperative Chemoradiotherapy in Locally Advanced Rectal Cancer Using Early-Treatment Blood Features Derived From Machine Learning. Front. Oncol. 2021, 11, 790894. [Google Scholar] [CrossRef]
  49. Ren, D.L.; Li, J.; Yu, H.C.; Peng, S.Y.; Lin, W.D.; Wang, X.L.; Ghoorun, R.A.; Luo, Y.X. Nomograms for predicting pathological response to neoadjuvant treatments in patients with rectal cancer. World J. Gastroenterol. 2019, 25, 118–137. [Google Scholar] [CrossRef]
  50. Buijsen, J.; van Stiphout, R.G.; Menheere, P.P.; Lammering, G.; Lambin, P. Blood biomarkers are helpful in the prediction of response to chemoradiation in rectal cancer: A prospective, hypothesis driven study on patients with locally advanced rectal cancer. Radiother. Oncol. 2014, 111, 237–242. [Google Scholar] [CrossRef]
  51. Cho, E.; Park, I.J.; Yeom, S.S.; Hong, S.M.; Lee, J.B.; Kim, Y.W.; Kim, M.J.; Lim, H.M.; Lim, S.B.; Yu, C.S.; et al. A Multigene Model for Predicting Tumor Responsiveness After Preoperative Chemoradiotherapy for Rectal Cancer. Int. J. Radiat. Oncol. Biol. Phys. 2019, 105, 834–842. [Google Scholar] [CrossRef]
  52. Emons, G.; Auslander, N.; Jo, P.; Kitz, J.; Azizian, A.; Hu, Y.; Hess, C.F.; Roedel, C.; Sax, U.; Salinas, G.; et al. Gene-expression profiles of pretreatment biopsies predict complete response of rectal cancer patients to preoperative chemoradiotherapy. Br. J. Cancer 2022, 127, 766–775. [Google Scholar] [CrossRef]
  53. Wang, L.; Chen, X.; Zhang, H.; Hong, L.; Wang, J.; Shao, L.; Chen, G.; Wu, J. Comprehensive analysis of transient receptor potential channels-related signature for prognosis, tumor immune microenvironment, and treatment response of colorectal cancer. Front. Immunol. 2022, 13, 1014834. [Google Scholar] [CrossRef]
  54. Wei, F.Z.; Mei, S.W.; Wang, Z.J.; Chen, J.N.; Shen, H.Y.; Zhao, F.Q.; Li, J.; Xiao, T.X.; Liu, Q. Development and Validation of a Nomogram and a Comprehensive Prognostic Analysis of an LncRNA-Associated Competitive Endogenous RNA Network Based on Immune-Related Genes for Locally Advanced Rectal Cancer With Neoadjuvant Therapy. Front. Oncol. 2021, 11, 697948. [Google Scholar] [CrossRef] [PubMed]
  55. Jia, H.; Shen, X.; Guan, Y.; Xu, M.; Tu, J.; Mo, M.; Xie, L.; Yuan, J.; Zhang, Z.; Cai, S.; et al. Predicting the pathological response to neoadjuvant chemoradiation using untargeted metabolomics in locally advanced rectal cancer. Radiother. Oncol. 2018, 128, 548–556. [Google Scholar] [CrossRef] [PubMed]
  56. Jiang, W.; Li, M.; Tan, J.; Feng, M.; Zheng, J.; Chen, D.; Liu, Z.; Yan, B.; Wang, G.; Xu, S.; et al. A Nomogram Based on a Collagen Feature Support Vector Machine for Predicting the Treatment Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer Patients. Ann. Surg. Oncol. 2021, 28, 6408–6421. [Google Scholar] [CrossRef] [PubMed]
  57. Jiang, W.; Wang, S.; Wan, J.; Zheng, J.; Dong, X.; Liu, Z.; Wang, G.; Xu, S.; Xiao, W.; Gao, Y.; et al. Association of the collagen signature with pathological complete response in rectal cancer patients. Cancer Sci. 2022, 113, 2409–2424. [Google Scholar] [CrossRef]
  58. Lou, X.; Zhou, N.; Feng, L.; Li, Z.; Fang, Y.; Fan, X.; Ling, Y.; Liu, H.; Zou, X.; Wang, J.; et al. Deep Learning Model for Predicting the Pathological Complete Response to Neoadjuvant Chemoradiotherapy of Locally Advanced Rectal Cancer. Front. Oncol. 2022, 12, 807264. [Google Scholar] [CrossRef]
  59. Wang, A.; Ding, R.; Zhang, J.; Zhang, B.; Huang, X.; Zhou, H. Machine Learning of Histomorphological Features Predict Response to Neoadjuvant Therapy in Locally Advanced Rectal Cancer. J. Gastrointest. Surg. 2022, 27, 162–165. [Google Scholar] [CrossRef]
  60. Zhang, F.; Yao, S.; Li, Z.; Liang, C.; Zhao, K.; Huang, Y.; Gao, Y.; Qu, J.; Li, Z.; Liu, Z. Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features. Clin. Transl. Med. 2020, 10, e110. [Google Scholar] [CrossRef] [Green Version]
  61. Huang, A.; Xiao, Y.; Peng, C.; Liu, T.; Lin, Z.; Yang, Q.; Zhang, T.; Liu, J.; Ma, H. 53BP1 expression and immunoscore are associated with the efficacy of neoadjuvant chemoradiotherapy for rectal cancer. Strahlenther. Onkol. 2020, 196, 465–473. [Google Scholar] [CrossRef]
  62. Amin, M.B.; Greene, F.L.; Edge, S.B.; Compton, C.C.; Gershenwald, J.E.; Brookland, R.K.; Meyer, L.; Gress, D.M.; Byrd, D.R.; Winchester, D.P. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more “personalized” approach to cancer staging. CA Cancer J. Clin. 2017, 67, 93–99. [Google Scholar] [CrossRef]
  63. Mandard, A.M.; Dalibard, F.; Mandard, J.C.; Marnay, J.; Henry-Amar, M.; Petiot, J.F.; Roussel, A.; Jacob, J.H.; Segol, P.; Samama, G.; et al. Pathologic assessment of tumor regression after preoperative chemoradiotherapy of esophageal carcinoma. Clinicopathologic correlations. Cancer 1994, 73, 2680–2686. [Google Scholar] [CrossRef]
  64. Song, C.; Chung, J.H.; Kang, S.B.; Kim, D.W.; Oh, H.K.; Lee, H.S.; Kim, J.W.; Lee, K.W.; Kim, J.H.; Kim, J.S. Impact of Tumor Regression Grade as a Major Prognostic Factor in Locally Advanced Rectal Cancer after Neoadjuvant Chemoradiotherapy: A Proposal for a Modified Staging System. Cancers 2018, 10, 319. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Dworak, O.; Keilholz, L.; Hoffmann, A. Pathological features of rectal cancer after preoperative radiochemotherapy. Int. J. Colorectal. Dis. 1997, 12, 19–23. [Google Scholar] [CrossRef] [PubMed]
  66. Quirke, P.; Durdey, P.; Dixon, M.F.; Williams, N.S. Local recurrence of rectal adenocarcinoma due to inadequate surgical resection. Histopathological study of lateral tumour spread and surgical excision. Lancet 1986, 2, 996–999. [Google Scholar] [CrossRef]
  67. Chang, H.J.; Park, C.K.; Kim, W.H.; Kim, Y.B.; Kim, Y.W.; Kim, H.G.; Bae, H.I.; Song, K.S.; Chang, M.S.; Chang, H.K.; et al. A Standardized Pathology Report for Colorectal Cancer. Korean J. Pathol. 2006, 40, 193–203. [Google Scholar]
  68. Pagès, F.; Kirilovsky, A.; Mlecnik, B.; Asslaber, M.; Tosolini, M.; Bindea, G.; Lagorce, C.; Wind, P.; Marliot, F.; Bruneval, P.; et al. In situ cytotoxic and memory T cells predict outcome in patients with early-stage colorectal cancer. J. Clin. Oncol. 2009, 27, 5944–5951. [Google Scholar] [CrossRef] [PubMed]
  69. Pagès, F.; Mlecnik, B.; Marliot, F.; Bindea, G.; Ou, F.S.; Bifulco, C.; Lugli, A.; Zlobec, I.; Rau, T.T.; Berger, M.D.; et al. International validation of the consensus Immunoscore for the classification of colon cancer: A prognostic and accuracy study. Lancet 2018, 391, 2128–2139. [Google Scholar] [CrossRef]
  70. Yashima-Abo, A.; Otsuka, K.; Nishizuka, S.S. Editorial Comment to “A nomogram Based on a Collagen Feature Support Vector Machine for Predicting the Treatment Response to Neoadjuvant Chemoradiotherapy in Rectal Cancer Patients”. Ann. Surg. Oncol. 2021, 28, 5818–5819. [Google Scholar] [CrossRef]
  71. Chen, D.; Chen, G.; Jiang, W.; Fu, M.; Liu, W.; Sui, J.; Xu, S.; Liu, Z.; Zheng, X.; Chi, L.; et al. Association of the Collagen Signature in the Tumor Microenvironment With Lymph Node Metastasis in Early Gastric Cancer. JAMA Surg. 2019, 154, e185249. [Google Scholar] [CrossRef]
  72. Jiang, W.; Wang, H.; Chen, W.; Zhao, Y.; Yan, B.; Chen, D.; Dong, X.; Cheng, J.; Lin, Z.; Zhuo, S.; et al. Association of collagen deep learning classifier with prognosis and chemotherapy benefits in stage II-III colon cancer. Bioeng. Transl. Med. 2023, 8, e10526. [Google Scholar] [CrossRef]
  73. Aeffner, F.; Zarella, M.D.; Buchbinder, N.; Bui, M.M.; Goodman, M.R.; Hartman, D.J.; Lujan, G.M.; Molani, M.A.; Parwani, A.V.; Lillard, K.; et al. Introduction to Digital Image Analysis in Whole-slide Imaging: A White Paper from the Digital Pathology Association. J. Pathol. Inform. 2019, 10, 9. [Google Scholar] [CrossRef]
  74. Davri, A.; Birbas, E.; Kanavos, T.; Ntritsos, G.; Giannakeas, N.; Tzallas, A.T.; Batistatou, A. Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review. Diagnostics 2022, 12, 837. [Google Scholar] [CrossRef] [PubMed]
  75. Gold, A.; Choueiry, F.; Jin, N.; Mo, X.; Zhu, J. The Application of Metabolomics in Recent Colorectal Cancer Studies: A State-of-the-Art Review. Cancers 2022, 14, 725. [Google Scholar] [CrossRef] [PubMed]
  76. Wang, H.; Jia, H.; Gao, Y.; Zhang, H.; Fan, J.; Zhang, L.; Ren, F.; Yin, Y.; Cai, Y.; Zhu, J.; et al. Serum metabolic traits reveal therapeutic toxicities and responses of neoadjuvant chemoradiotherapy in patients with rectal cancer. Nat. Commun. 2022, 13, 7802. [Google Scholar] [CrossRef]
  77. El Sissy, C.; Kirilovsky, A.; Van den Eynde, M.; Muşină, A.M.; Anitei, M.G.; Romero, A.; Marliot, F.; Junca, A.; Doyen, J.; Mlecnik, B.; et al. A Diagnostic Biopsy-Adapted Immunoscore Predicts Response to Neoadjuvant Treatment and Selects Patients with Rectal Cancer Eligible for a Watch-and-Wait Strategy. Clin. Cancer Res. 2020, 26, 5198–5207. [Google Scholar] [CrossRef]
  78. Momma, T.; Okayama, H.; Kanke, Y.; Fukai, S.; Onozawa, H.; Fujita, S.; Sakamoto, W.; Saito, M.; Ohki, S.; Kono, K. Validation of Gene Expression-Based Predictive Biomarkers for Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Cancers 2021, 13, 4642. [Google Scholar] [CrossRef] [PubMed]
  79. Brettingham-Moore, K.H.; Duong, C.P.; Greenawalt, D.M.; Heriot, A.G.; Ellul, J.; Dow, C.A.; Murray, W.K.; Hicks, R.J.; Tjandra, J.; Chao, M.; et al. Pretreatment transcriptional profiling for predicting response to neoadjuvant chemoradiotherapy in rectal adenocarcinoma. Clin. Cancer Res. 2011, 17, 3039–3047. [Google Scholar] [CrossRef] [Green Version]
  80. Gim, J.; Cho, Y.B.; Hong, H.K.; Kim, H.C.; Yun, S.H.; Wu, H.G.; Jeong, S.Y.; Joung, J.G.; Park, T.; Park, W.Y.; et al. Predicting multi-class responses to preoperative chemoradiotherapy in rectal cancer patients. Radiat. Oncol. 2016, 11, 50. [Google Scholar] [CrossRef] [Green Version]
  81. Izzotti, A.; Ceccaroli, C.; Geretto, M.; Ruggieri, F.G.; Schenone, S.; Di Maria, E. Predicting Response to Neoadjuvant Therapy in Colorectal Cancer Patients the Role of Messenger-and Micro-RNA Profiling. Cancers 2020, 12, 1652. [Google Scholar] [CrossRef]
  82. de Jong, Y.; Ramspek, C.L.; Zoccali, C.; Jager, K.J.; Dekker, F.W.; van Diepen, M. Appraising prediction research: A guide and meta-review on bias and applicability assessment using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Nephrology 2021, 26, 939–947. [Google Scholar] [CrossRef]
  83. Venema, E.; Wessler, B.S.; Paulus, J.K.; Salah, R.; Raman, G.; Leung, L.Y.; Koethe, B.C.; Nelson, J.; Park, J.G.; van Klaveren, D.; et al. Large-scale validation of the prediction model risk of bias assessment Tool (PROBAST) using a short form: High risk of bias models show poorer discrimination. J. Clin. Epidemiol. 2021, 138, 32–39. [Google Scholar] [CrossRef]
  84. Riley, R.D.; Ensor, J.; Snell, K.I.E.; Harrell, F.E., Jr.; Martin, G.P.; Reitsma, J.B.; Moons, K.G.M.; Collins, G.; van Smeden, M. Calculating the sample size required for developing a clinical prediction model. BMJ 2020, 368, m441. [Google Scholar] [CrossRef] [Green Version]
  85. Attia, A.M.; Farrag, A.; Attia, N.M.; Khalaf, L.M.; Hassan, H.M.; Ameen, M.G.; Aboeleuon, E.; El-Raheem, S.S.A.; Mahran, A.; Hefni, A.M. Signet ring cell component predicts the response to neoadjuvant chemoradiotherapy in rectal cancer. Long interim results of a single institution experience. Am. J. Cancer Res. 2022, 12, 1156–1168. [Google Scholar]
  86. McCawley, N.; Clancy, C.; O’Neill, B.D.; Deasy, J.; McNamara, D.A.; Burke, J.P. Mucinous Rectal Adenocarcinoma Is Associated with a Poor Response to Neoadjuvant Chemoradiotherapy: A Systematic Review and Meta-analysis. Dis. Colon. Rectum. 2016, 59, 1200–1208. [Google Scholar] [CrossRef]
  87. Lania, A.; Ferraù, F.; Rubino, M.; Modica, R.; Colao, A.; Faggiano, A. Neoadjuvant Therapy for Neuroendocrine Neoplasms: Recent Progresses and Future Approaches. Front. Endocrinol. 2021, 12, 651438. [Google Scholar] [CrossRef]
  88. Conte, B.; George, B.; Overman, M.; Estrella, J.; Jiang, Z.Q.; Mehrvarz Sarshekeh, A.; Ferrarotto, R.; Hoff, P.M.; Rashid, A.; Yao, J.C.; et al. High-Grade Neuroendocrine Colorectal Carcinomas: A Retrospective Study of 100 Patients. Clin. Colorectal. Cancer 2016, 15, e1–e7. [Google Scholar] [CrossRef] [Green Version]
  89. Kokaine, L.; Gardovskis, A.; Gardovskis, J. Evaluation and Predictive Factors of Complete Response in Rectal Cancer after Neoadjuvant Chemoradiation Therapy. Medicina 2021, 57, 1044. [Google Scholar] [CrossRef]
  90. Glynne-Jones, R.; Wallace, M.; Livingstone, J.I.; Meyrick-Thomas, J. Complete clinical response after preoperative chemoradiation in rectal cancer: Is a “wait and see” policy justified? Dis. Colon. Rectum. 2008, 51, 10–19; discussion 19–20. [Google Scholar] [CrossRef]
Cancers 15 03945 g001
Figure 1. PRISMA flow diagram of systematic review.
Figure 1. PRISMA flow diagram of systematic review.
Cancers 15 03945 g001
Cancers 15 03945 g002
Figure 2. PROBAST risk of bias assessment. ROB = risk of bias.
Figure 2. PROBAST risk of bias assessment. ROB = risk of bias.
Cancers 15 03945 g002
Cancers 15 03945 g003
Figure 3. Discriminative performance with confidence intervals. Y-axis represents reported AUCs; x-axis represents included studies. Apparent performance was reported in six of fourteen model development studies [51,54,56,57,59,60]. The 95% confidence intervals were reported in 10 studies [46,47,49,50,51,55,56,57,58,60] AUC = area under the curve; GR = good response; pCR = pathological complete response.
Figure 3. Discriminative performance with confidence intervals. Y-axis represents reported AUCs; x-axis represents included studies. Apparent performance was reported in six of fourteen model development studies [51,54,56,57,59,60]. The 95% confidence intervals were reported in 10 studies [46,47,49,50,51,55,56,57,58,60] AUC = area under the curve; GR = good response; pCR = pathological complete response.
Cancers 15 03945 g003
Table 1. Types of prediction model studies according to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) statement [39].
Table 1. Types of prediction model studies according to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) statement [39].
Study TypeTRIPOD TypeDevelopmentType of
Validation		Explanation
Development studies
Development only1aYesNoDevelopment and evaluation using the apparent performance.
Development and validation studies
Development and validation using resampling1bYesInternalDevelopment and evaluation using resampling methods (e.g., cross-validation or bootstrapping). The term “internal validation” commonly refers to resampling methods.
Development and validation using random split2aYesInternalRandom split of data into a development and validation cohort.
Development and validation using nonrandom split2bYesIntermediaryNonrandom split (e.g., location or time) of data into a development and validation cohort. Type 2b studies are considered as intermediary between internal and external validation.
Development and validation using separate data3YesExternalDevelopment using one cohort and validation in a separate cohort (e.g., from a different study).
Validation studies
Validation only4NoExternalValidation of an existing (published) prediction model in a separate cohort.
Table 2. Cohort characteristics of the 16 included studies.
Table 2. Cohort characteristics of the 16 included studies.
Study, [Ref.]Model
Development (Yes/No)		Validation
Cohort(s) (No. of Cohorts and TRIPOD Category)		Cohort Designnn with Outcome (%)StageNATSurgery
Clinical
Joye (2016), [47]Yes1. 1bR/S620pCR: 120 (19)
GR: 170 (27)		cT1-4N0-2CRTTME
Kim (2021), [48]Yes1. 1bR/S19083 (43)LARC ¥¥CRTTME
2. 2aR/S4119 (47)LARCCRTTME
3. 2aR/S4119 (47)LARCCRTTME
Ren (2019), [49]Yes1. 1bR/S12645 (36)cT2-4N0-+CRTTME
Clinical, serological and radiological
Buijsen (2014), [50]Yes1. 1bP/S276pCR: 57 (21)
GR: 130 (47)		cT3-4N0-2CRTTME
Genetics
Cho (2019), [51]Yes1. 1bR/S184102 (55)cT3-4N0-+CRTTME (178) + LE (6)
Emons (2022), [52]Yes1. 1bP/M6432 (50)II-IIICRTTME
2. 1b/2b §P/M16132 (20)II-IIICRTTME
3. 2bP/M144 (28)II-IIICRTTME
4. 3R/S388 (21)cT3-4N0-2CRTTME
5. 3P/M244 (16)II-IIICRTTME
Wang, L. (2022), [53]No1. 4R/S8035 (44)cT3-4N0-2CRTTME
2. 4R/S8545 (53)II-IIIUnknownUnknown
Wei (2021), [54]Yes1. 1bR/S5927 (46)cT3-4N0-2CRTTME (52) + LE (7)
Metabolites
Jia (2018), [55]Yes1. 1bP/S10556 (53)cT3-4N0-2CRT + Con.TME
Lv (2022), [46]Yes1. 1bP/S10656 (53)cT2-4N0-2CRT + Con.TME
Pathology
Jiang (2021), [56]Yes1. 1bR/M299163 (55)cT3-4N0-+CRTTME
2. 2aR/M12970 (54)cT3-4N0-+CRTTME
3. Cohort 1 + 2 ‡R/M428233 (54)cT3-4N0-+CRTTME
Jiang (2022), [57]Yes1. 1bR/M35376 (22)cT3-4N0-+CRTTME
2. 3R/S16337 (23)cT3-4N0-+CRTTME
3. Cohort 1 + 2 ‡R/M516113 (22)cT3-4N0-+CRTTME
Lou (2022), [58]Yes1. –R/M666171 (26)cT3-4N0-2CRTTME
2. 2aR/M11730 (26)cT3-4N0-2CRTTME
3. 3R/M10224 (24)cT3-4N0-2CRTTME
Wang, A. (2022), [59]Yes1. -R/S5521 (38)cT2-4N0-2Ind. + CRTTME
2. 2aR/S145 (37)cT2-4N0-2Ind. + CRTTME
Zhang (2020), [60]Yes1. -R/S12059 (49)cT2-4N0-2CRT + Con.Unknown
2. 2aR/S3120 (65)cT2-4N0-2CRT + Con.Unknown
Huang (2020), [61]No1. 4R/S5534 (62)cT3-4N0-+CRTTME
Con. = consolidation chemotherapy; CRT = chemoradiotherapy; GR = good response; Ind. = induction chemotherapy; LARC = locally advanced rectal cancer; LE = local excision; M = multicenter; n = number of patients; NAT = neoadjuvant therapy; P = prospective; pCR = pathological complete response; R = retrospective; Ref. = reference; S = single-center; TME = total mesorectal excision; TRIPOD = transparent reporting of a multivariable prediction model for individual prognosis or diagnosis. ¥¥ = more detailed tumor stage was not reported; § = the 64 patients of cohort 1 were also included in cohort 2; ‡ = cohort that used all patients, from the development as well as the validation cohort.
Table 3. Model characteristics of the 16 included studies.
Table 3. Model characteristics of the 16 included studies.
Study, [Ref.]Model with Final Predictors (No. of Predictors)MethodOutcomeApparent AUCAUC of Validation Cohort(s) (No. of Cohorts and TRIPOD Category)
Clinical
Joye (2016), [47]- Model 1 (pCR): age, ASA score, CEA, cN stage, gender and Hb. (6)
- Model 2 (GR): CEA, cT stage/MRF, cN stage and Hb. (4)		LRpCR
GR		pCR. Unknown
GR. Unknown 		1. 1b. (pCR) 0.60 (range 0.56–0.62)
1. 1b. (GR) 0.63 (range 0.62–0.64)
Kim (2021), [48]- Age, alcohol, ASA, BMI, diabetes, distance from anal verge, gender, hypertension, smoking and tumor grade. (10)RRGRUnknown1. 1b. 0.65 (mean ± std: 0.02)
2. 2a. 0.53 (mean ± std 0.08)
3. 2a. 0.51 (mean ± std 0.08)
Ren (2019), [49]- MRF and tumor length. (2)LRpCRUnknown1. 1b. 0.70 (95% CI 0.61–0.79)
Clinical, serological and imaging
Buijsen (2014), [50]- CEA, cT stage, cN stage and tumor length, IL-6, IL-8, osteopontin, SUVmax. (8)LRpCR
GR		pCR. Unknown
GR. Unknown 		1. 1b. (pCR) 0.81 (95% CI 0.73–0.88)
1. 1b. (GR) 0.78 (95% CI 0.71–0.85)
Genetics
Cho (2019), [51]Radio-response
prediction index
- mRNAs. (8)		LRGR0.85 (95% CI 0.80–0.90)1. 1b. 0.81 (95% CI 0.71–0.91)–0.91 (95% CI: 0.84–0.98) ¥¥
Emons (2022), [52]21-transcript signature
- Transcripts. (21)		SVMpCRUnknown1. 1b. 0.75
2. 1b/2b §. 0.81
3. 2b. 0.70
4. 3. 0.76
5. 3. 0.81
Wang, L. (2022), [53]Transient receptor potential channels (TRPC) score
- TRPC-related genes. (8)		R package “glmnet”GRNA1. 4. 0.66
2. 4. 0.71
Wei (2021), [54]Response-Related
Prediction Nomogram
- IRDEGs, age and gender. (6)		R package “glmnet”GR0.82 1. 1b. 0.75
Metabolites
Jia (2018), [55]Metabolite panel
- Metabolites. (15)		PLSGRUnknown1. 1b. 0.80 (95% CI 0.67– 0.91)
Lv (2022), [46]Metabolite panel
- Metabolites. (8)		RM-ANOVA, ML-PLS-DA and LRGRUnknown1. 1b. 0.54 (95% CI 0.43–0.65)
Pathology
Jiang (2021), [56]Collagen feature nomogram
- CFs-SVM classifier, CEA, cT stage, and tumor differentiation. (6)		SVM and LRGR0.83 (95% CI 0.79–0.88)1. 1b. 0.84
2. 2a. 0.85 (95% CI 0.79–0.92)
3. Cohort 1 + 2 ‡. 0.84
Jiang (2022), [57]Collagen feature nomogram
- Collagen signature, CEA, cT stage, tumor differentiation, and tumor dimension (length). (8)		LRpCR0.89 (95% CI 0.85–0.94)1. 1b. 0.89
2. 3. 0.91 (95% CI 0.86–0.96)
3. Cohort 1 + 2 ‡. 0.90 (95% CI 0.90–0.93)
Lou (2022), [58]Deep pathological complete response model
- WSIs. (NA)		MILpCRUnknown2. 2a. 0.71 (95% CI 0.60–0.81)
3. 3. 0.72 (95% CI 0.59–0.84)
Wang, A. (2022), [59]Digital-pathology-based deep learning model
- WSIs. (NA)		CNN and GNNGR0.782. 2a. 0.73
Zhang (2020), [60]Pathology signature
- WSIs. (17)		SVM and LRGR0.93 (95% CI 0.88–0.97)2. 2a. 0.88 (95% CI 0.72–0.97)
Huang (2020), [61]Immunoscore
- TILs. (4)		LRGRNA1. 4. 0.72 (95% CI 0.57–0.86)
95% CI = 95% confidence interval; ASA = American Society of Anesthesiologists score; AUC = area under the curve; CEA = carcinoembryonic antigen; CFs-SVM classifier = Collagen Features Support Vector Machine classifier; CNN = convolutional neural network; glmnet = lasso and elastic-net regularized generalized linear models; GNN = graph neural network; GR = good response; Hb = hemoglobin; IL = interleukin; IRDEGs = immune-related differentially expressed genes; MIL = multi-instance learning; ML-PLS-DA = multilevel partial least squares discriminant analysis; MRF = mesorectal fascia; mRNA = messenger RNA; n = number of predictors in final model; NA = not applicable; LR = logistic regression; pCR = pathological complete response; PLS = partial least squares; Ref. = reference; RM-ANOVA = repeated measure analysis of variance; RR = ridge regression; SUVmax = maximal standardized uptake value; SVM = support vector machine; TILs = tumor-infiltrating lymphocytes; TRIPOD = transparent reporting of a multivariable prediction model for individual prognosis or diagnosis; TRPC = transient receptor potential channels; U = unknown; WSIs = whole slide images. ¥¥ = range of cross-validated AUCs; § = the 64 patients of cohort 1 were also included in cohort 2; ‡ = cohort that used all patients, from the development as well as the validation cohort.
Table 4. PROBAST risk of bias assessment.
Table 4. PROBAST risk of bias assessment.
StudyTypeROBApplicabilityOverall
ParticipantsPredictorsOutcomeAnalysisParticipantsPredictorsOutcomeROBApplicability
Clinical
Joye (2016), [47]DLowLowLowHighLowLowLowHighLow
Kim (2021), [48]DLowLowLowHighLowUnclearLowHighUnclear
Ren (2019), [49]DLowLowLowHighLowLowLowHighLow
Clinical, serological and radiological
Buijsen (2014), [50]DLowLowLowHighLowLowLowHighLow
Genetics
Cho (2019), [51]DLowLowLowHighLowLowLowHighLow
Emons (2022), [52]D + VLowLowLowHighLowLowLowHighLow
Wang, L. (2022), [53]VLowHighUnclearHighHighLowUnclearHighHigh
Wei (2021), [54]DLowLowLowHighLowLowLowHighLow
Metabolites
Jia (2018), [55]DLowUnclearLowHighLowLowLowHighLow
Lv (2022), [46]DLowUnclearLowHighLowLowLowHighLow
Pathology
Jiang (2021), [56]DLowLowLowHighLowLowLowHighLow
Jiang (2022), [57]D + VLowLowLowHighLowLowLowHighLow
Lou (2022), [58]D + VLowLowLowHighLowLowLowHighLow
Wang, A. (2022), [59]DLowHighUnclearHighLowLowUnclearHighUnclear
Zhang (2020), [60]DLowHighLowHighLowLowLowHighLow
Huang (2020), [61]VLowLowLowHighLowLowLowHighLow
D = development; ROB = risk of bias; V = validation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tanaka, M.D.; Geubels, B.M.; Grotenhuis, B.A.; Marijnen, C.A.M.; Peters, F.P.; van der Mierden, S.; Maas, M.; Couwenberg, A.M. Validated Pretreatment Prediction Models for Response to Neoadjuvant Therapy in Patients with Rectal Cancer: A Systematic Review and Critical Appraisal. Cancers 2023, 15, 3945. https://doi.org/10.3390/cancers15153945

AMA Style

Tanaka MD, Geubels BM, Grotenhuis BA, Marijnen CAM, Peters FP, van der Mierden S, Maas M, Couwenberg AM. Validated Pretreatment Prediction Models for Response to Neoadjuvant Therapy in Patients with Rectal Cancer: A Systematic Review and Critical Appraisal. Cancers. 2023; 15(15):3945. https://doi.org/10.3390/cancers15153945

Chicago/Turabian Style

Tanaka, Max D., Barbara M. Geubels, Brechtje A. Grotenhuis, Corrie A. M. Marijnen, Femke P. Peters, Stevie van der Mierden, Monique Maas, and Alice M. Couwenberg. 2023. "Validated Pretreatment Prediction Models for Response to Neoadjuvant Therapy in Patients with Rectal Cancer: A Systematic Review and Critical Appraisal" Cancers 15, no. 15: 3945. https://doi.org/10.3390/cancers15153945

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop