Machine-intelligence for developing a potent signature to predict ovarian response to tailor assisted reproduction technology

Materials and Methods

Data processing

The clinical data of 1,110 infertile women who had undergone IVF/ICSI treatment for the first time between July 2018 and May 2019 in Renmin Hospital of Wuhan University was retrospectively analyzed. Women with several different infertility factors were incorporated in order to establish a universal approach for POR prediction at our center.

Patients’ characteristics and main outcomes

In the prospective cohort analysis, the main outcome measure was POR, which was defined as the retrieval of four or fewer oocytes or cycle cancellation [24]. Variables with a potential relationship to ovarian response were incorporated into our research, and models were constructed based on the various therapeutic stages of the treatment cycle:

(1) Variables of COS pre-launch model: age, BMI, infertility cause, infertility duration, infertility type, AMH, basic hormone levels (E₂, FSH, and LH), AFC, pelvic surgery, and gravidity history.

(2) Variables of hCG pre-trigger model: all factors of the COS pre-launch model, plus therapeutic regimen, dosage of Gn (recombinant human follicle-stimulating hormone for injection, Gonal-f, German Merck Serono), days of Gn, E₂ level on hCG day, and follicle number on hCG day (follicles with a diameter of ≥ 14 mm in bilateral ovaries).

Feature selection

EpiData 3.1 software was used for establishing a database, and this was double-entered and validated by two qualified personnel. Once it was checked, the data was transferred to R software (version 3.6.4) and parameters proven to have a direct effect on ovarian response were screened using logistic regression; variables with a P < 0.05 were chosen for further analysis. After the effects of features on outcomes were fully assessed, least absolute shrinkage and selection operator (LASSO) regression was used for further minimization of the risk of over-fitting, and variables with high collinearity were eliminated. The LASSO regression was dependent on a cyclical coordinate descent algorithm and was conducted using a glmnet package in R software. The workflow of the study is presented in Figure 1.

Figure 1. Schematic workflow for poor ovarian response prediction. (A) 11 features relating to ovarian response were obtained following logistic regression and LASSO. (B) These 11 candidate features were analyzed using multivariable logistic regression and machine learning, and then validated using ROC, calibration plot, C-index and correlation analysis to conduct CPLM and HPTM. (C) Variable importance of CPLM and HPTM were described to further understand and investigate of the models.

Construction of model

All data was randomly divided into a training dataset (70%) for feature selection and model training, and an independent validation dataset (30%) for repeated optimization and verification of the prediction model. And the models were set to use default parameters in R software.

Multivariable logistic model

Normality was evaluated using a Kolmogorov-Smirnov test and Spearman’s Rho (nonparametric), or Pearson’s (parametric) bivariate correlation analysis was completed as deemed appropriate. For independent variables selected for the generalized multivariable logistic model, stepwise Akaike information criterion (AIC) was applied for eliminating multicollinearity and for selection of the model with the lowest AIC as the final model. A multivariate logistic model was also used to construct the ovarian response predictive model (ORPM). To facilitate this, the risk score was calculated using the following formula:

$$\text{Risk} \text{score}={\displaystyle {\sum }_{i=1}^n{\beta }_i\times E_i}$$

where the risk score defined as ORPM-based risk signature was calculated by the ORPM - n represents the total number included in the ORPM, β _i represents the regression coefficient of feature i, and E _i refers to the coefficient of feature i in the constructed model.

Machine learning

Decision tree

Decision tree algorithms use the Gini index to measure each decision point and create an optimal separation of the independent variables [25]. A dataset which minimizes the Gini index was selected after division as the optimal distribution in the subset of data. This splits the data which exhibited the best optimization criteria (subject to tree depth (11)) on our predictor.

Random forest (RF)

RF combines multiple decision trees and randomizes and summarizes the use of variables and data [26]. This study conducted RF containing 1,000 trees, where the maximum depth of each tree was determined based on the final numbers of the included features.

eXtreme gradient boosting (XGBoost)

XGBoost introduces the gradient descent algorithm and minimizes the loss when a new model is added, which helps it continuously learn a new function matching the residual of the previous prediction [27]. Similarly, XGBoost served as iterative model before reaching 1,000 cycles, and the maximum depth of each tree was determined based on the final numbers of the included features.

Support vector machines (SVM)

The aim of SVM is the establishment of a classification hyperplane that can correctly classify each sample and make the largest possible distance between the sample closest to the hyperplane for each sample type and the hyperplane [28].

Artificial neural network

ANN consists of an input layer, an output layer and one or more hidden layers between the input and the output layers. The most outstanding representative of the algorithm is resilient backpropagation learning [29]. In a typical process, hidden layers are determined to refer to the actual status, and the threshold is set as 0.005, the learning rate is set as 0.1, and parameter optimization is performed using rprop+ method.

Validation of COS pre-launch model (CPLM) and hCG pre-trigger model (HPTM)

Several different approaches were utilized for the assessment of all models’ stratification abilities. Area under curve (AUC) was calculated from the receiver-operating characteristic (ROC) curve and was used to estimate the discrimination of each model. The accuracy of the derived models was evaluated by calibration plot, and models which shared a high goodness of fit with the dotted line were regarded as providing good calibration [30]. Notably, the net-classification index (NRI) was used to quantify the improvement of the predictive abilities of each model. The models with the highest ovarian response prediction accuracy in COS pre-launch and hCG pre-trigger models were defined as CPLM and HPTM. The contribution and importance of each CPLM/HPTM-based signature were quantified using mean concordance-index (C-index). Spearman correlation analysis was then performed to accurately determine the correlation between the CPLM and HPTM scores of each patient and the corresponding retrieved oocytes.

Grouped analysis for potential difference of clinical features

Statistical comparisons of patients’ clinical characters were performed using Wilcoxon’s test, and P-value adjustment using the Benjamini-Hochberg procedure.

Statistics

R software (version 3.6.4) was used for data processing and analysis.

Ethics approval and consent to participate

Written informed consent was obtained from each participant and the study was approved by the ethical committee of the Renmin Hospital of Wuhan University.

Results

Demographic and clinical characteristics of participants

Based on the number of oocytes retrieved, the prevalence of POR was 14.59% in the present cohort. The demographic parameters of participants are displayed in Table 1. Poor ovarian responders were older than the normal to high responders, and exhibited significantly higher E₂, FSH, days of Gn, dosages of Gn, E₂ level and follicle number on hCG day. Significantly differences were also presented regarding infertility cause and therapeutic regimen.

Table 1. Baseline participant characteristics.

Parameters		Normal to high responders	Poor responders	P	Effect size
E2 level on hCG day		3090.00(2295.00-4539.00)	1422.20(799.80-2267.80)	<0.001	r = 0.396
follicle number on hCG day		12.00(8.00-16.25)	5.00(3.00-8.00)	<0.001	r = 0.402
AFC		16.00(11.00-21.00)	9.00(5.00-13.00)	<0.001	r = 0.365
AMH		3.09(1.99-5.05)	1.03(0.56-1.75)	<0.001	r = 0.420
infertility duration		3.00(2.00-6.00)	4.00(2.00-6.00)	0.297	r = -0.031
infertility cause	pelvic and fallopian tube factors	347(36.60)	51(31.48)	<0.001	V = 0.211
	ovulatory obstacle	74(7.81)	1(0.62)
	endometriosis and uterine factors	42(4.43)	19(11.73)
	decreased ovarian reserve	147(15.50)	32(19.75)
	unexplained infertility	64(6.75)	22(19.75)
	male factor	175(18.46)	20(12.35)
	multiple confounding factors	99(10.44)	17(10.49)
therapeutic regimen	long protocol	430(45.36)	51(31.48)	<0.001	V = 0.268
	super-long protocol	263(27.74)	21(12.96)
	antagonist regimen	162(17.09)	34(20.99)
	PPOS	87(9.18)	54(33.33)
	others	6(0.63)	2(1.23)
age		30.0(28.0-33.0)	32.00(29.00-36.00)	<0.001	r = -0.146
E2		35.60(31.45-41.99)	49.31(37.03-55.77)	<0.001	r = -0.303
FSH		6.870(5.707-8.210)	8.315(6.798-10.515)	<0.001	r = -0.217
LH		3.645(2.730-4.832)	3.515(2.525-4.562)	0.125	r = 0.046
days of Gn		10.00(9.00-11.00)	10.000(8.000-11.000)	<0.001	r = 0.116
dosages of Gn		2250(1725-2900)	2700(2025-3200)	<0.001	r = -0.117
pelvic surgery	No	718(75.74)	120(74.07)	0.649	r = 0
	Yes	230(24.26)	42(25.93)
gravidity history	0	484(51.05)	77(47.53)	0.522	r = 0
	1	236(24.89)	41(25.31)
	2	116(12.24)	21(12.96)
	≥3	112(11.82)	23(14.19)
height		160.0(158.0-163.0)	160.0(158.0-163.0)	0.904	r = 0.004
weight		56.00(51.00-62.00)	56.00(51.00-64.00)	0.525	r = -0.019
BMI		21.64(19.98-23.93)	21.95(20.20-24.23)	0.357	r = -0.028
infertility type	primary infertility	452(47.68)	72(44.44)	0.446	r = 0
	secondary infertility	496(52.32)	90(55.56)
Normally distributed data, skewed distribution data and nominal data are described by mean ± SD, median ± interquartile range and frequency (relative frequency) respectively. Wilxon signed-rank test and chi-square test were applied in skew distribution data and nominal data respectively, and properly used r and V as the effect size to quantify the significance.

Feature engineering

In order to prevent the risk of over-fitting and to screen the important features which impact outcomes for the optimization of the constructed models, feature engineering was conducted. LASSO regression combined with univariable logistic regression was performed to narrow the candidate features, the results of which were displayed in Table 2 and Supplementary Figure 1A, 1B. A total of 11 features remained of the original 19 features, and those selected were confirmed to be important regarding outcome. The significant variables identified following the selection procedure were recorded as follows: AFC, AMH, Age, E₂, FSH, and infertility factors were incorporated in the COS pre-launch model. Variables in the hCG pre-trigger model included all factors from the COS pre-launch model, in addition to E₂ level and follicle number on hCG day, therapeutic regimen, days of Gn, and dosages of Gn.

Table 2. Odds ratio and p-values calculated from the univariable logistic regression for quantifying the impacts of parameters included in the present study on ovarian response.

Parameters	Odds ratio (95% CI)	p
E2 level on the hCG day	1.00 (1.00-1.00)	5.97E-27**
follicle number on hCG day	0.75 (0.71-0.79)	1.34E-29**
AFC	0.81 (0.78-0.84)	4.91E-27**
AMH	0.37 (0.30-0.44)	1.92E-25**
infertility years	1.03 (0.98-1.08)	0.274
infertility cause	1.09 (1.01-1.17)	0.025*
therapeutic regimen	1.71 (1.47-1.98)	1.44E-12**
age	1.10 (1.06-1.14)	4.79E-07**
E2	1.07 (1.06-1.09)	3.82E-22**
FSH	1.25 (1.18-1.33)	5.43E-13**
LH	0.94 (0.85-1.03)	0.222
days of Gn	0.88 (0.82-0.95)	0.001026*
dosages of Gn	1.00 (1.00-1.00)	7.20E-05**
pelvic surgery	1.09 (0.74-1.59)	0.649
gravidity history	1.01 (0.89-1.14)	0.815
height	0.99 (0.96-1.03)	0.684
weight	1.00 (0.99-1.02)	0.614
BMI	1.02 (0.97-1.07)	0.440
infertility type	1.14 (0.82-1.60)	0.446
*, P<0.05; **, P<0.01.

Construction and comparison of method performance

After the process of feature selection completed, statistic models and machine-learning models were trained and validated according to the aforementioned methods. For COS pre-launch models, parameters of logistic model and decision tree were represented in Table 3 and Supplementary Figure 3, respectively; and framework of other machine-learning models including RF, XGBoost, SVM, and ANN please visit our data online at https://data.mendeley.com/datasets/tpj39wptts/1. For hCG pre-trigger models, components of logistic model and decision tree were exhibited in Table 4 and Supplementary Figure 4, respectively; and framework of machine-learning models please visit our data online at https://data.mendeley.com/datasets/tpj39wptts/1.

Table 3. Coefficients of trained logistic regression for COS pre-launch models.

Parameters	coefficients	Odds ratio (95% CI)	p
AFC	-0.115	0.891(0.843-0.94)	<0.001**
AMH	-0.729	0.482(0.368-0.616)	<0.001**
*, P<0.05; **, P<0.01.

Table 4. Coefficients of trained logistic regression for hCG pre-trigger models.

Parameters	coefficients	Odds ratio (95%CI)	p
E2 level on the hCG day	-0.001	1.000(0.9999-1.000)	0.004*
follicle number on hCG day	-0.071	0.932(0.850-1.014)	0.115
AFC	-0.063	0.939(0.880-1.000)	0.054
AMH	-0.618	0.539(0.405-0.697)	< 0.001**
FSH	-0.077	0.926(0.848-1.009)	0.081
dosages of Gn	-0.001	1.000(1.000-1.000)	0.082
*, P<0.05; **, P<0.01.

It has been demonstrated that the area under the ROC curve (AUC) is a puissant indicator for the prediction of dichotomous outcomes, and then the AUC was examined to assess the accuracy of the constructed models. As can be seen in Figure 2A–2L, ANN yielded optimum predictive ability and accuracy in all algorithms with an AUC of 0.859 in COS pre-launch models, and the RF had the highest AUC (0.903) in hCG pre-trigger models. The predictive ability and accuracy of logistic regression (AUC = 0.848 and 0.883 corresponded to COS pre-launch and hCG pre-trigger models) and decision tree (AUC= 0.701 and 0.800) were slightly worse in comparison to ANN or RF. XGBoost produced relatively poor results with AUC of 0.724 and 0.693, and SVM exhibited the minimum prediction efficiency, with AUC of 0.556 and 0.519. Similar trends were also observed in the training cohort.

Figure 2. Construction and comparison of method performance. (A–F) ROC curve of ANN, RF, LR, XGBoost, decision tree, and SVM for target cohort in COS pre-launch models, respectively. (G–L) ROC curve of ANN, RF, LR, XGBoost, decision tree, and SVM for target cohort in hCG pre-trigger models.

Validation of CPLM and HPTM

As they have been proven to be the best models for the estimation of ovarian response, derived ANN and RF models were considered as CPLM and HPTM and further investigations were conducted. C-index was determined for reaffirming CPLM and HPTM prediction accuracy. After 1,000 estimations were made using the bootstrap method, the mean C-index of the validation cohort’s CPLM and HPTM were 0.87 and 0.90, respectively. This demonstrated that the predicted results for CPLM and HPTM were highly consistent with the actual value, and represented high accuracy among the constructed models [31]. The training cohort also demonstrated similar results regarding C-index.

In addition, a calibration plot measuring calibration ability also showed that the predicted value of the CPLM and HPTM-based signature was in accordance with the observed proportion (Figure 3A, 3B).

Figure 3. Validation of CPLM and HPTM. (A) The calibration plot for training and validation data was used to analyze the consistency of poor ovarian response between the predicted value and the observed proportion in CPLM. (B) The calibration plot for training and validation data was used to analyze the consistency of poor ovarian response between the predicted value and the observed proportion in HPTM. (C) Correlation analysis of the CPLM score and relevant retrieved oocytes in the cohort. (D) Correlation analysis of the HPTM score and relevant retrieved oocytes in the cohort.

For further evaluation of the model’s credibility, correlation analysis between the CPLM and HPTM scores and the corresponding number of retrieved oocytes for each patient were determined. The analysis results demonstrated that each patient’s CPLM and HPTM scores were correlated negatively with retrieved oocytes, thereby suggesting that the retrieved oocytes gradually decreased as the score increased (Figure 3C, 3D), and the relevant correlation coefficient was 0.59 and 0.69 in CPLM and HPTM, respectively.

All aforementioned evidence was presented following a series of investigations, which strongly indicated that the constructed models reached an optimum contribution and employed a small enough number of clinical characters without losing their predictive value.

Comparison between CPLM/HPTM and common clinical characteristics

Numerous studies have proven AMH and AFC to be the most effective parameters for the prediction of poor ovarian response in ART [4, 32]. An evaluation of the effectiveness of obtained CPLM and HPTM was performed through a comparison of the above characteristics to establish both their superiority and applicability in clinical practice. The results were encouraging and revealed that the AUC of CPLM and HPTM (0.903 and 0.859) (Figure 2A, 2H) were superior to those of the most common clinical characteristics - AMH and AFC (0.824 and 0.796) (Figure 4A, 4B), indicating that the constructed models had more valuable prediction signatures than common clinical characteristics.

Figure 4. Comparison between HPTM and common clinical characteristics. (A) ROC curve and the corresponding AUC of AMH for training and validation cohort. (B) ROC curve and the corresponding AUC of AFC for training and validation cohort. (C) Variable importance ranking in HPTM.

NRI is a method for measuring a model’s accuracy based on changes made to the number of correct classifications. Results showed that CPLM had better accuracy compared to AMH and AFC (NRI =13.4% and 18.8%, respectively). In addition, HPTM’s accuracy was considerably higher than that of AMH and AFC (compared to AMH, NRI = 74.7%; compared to AFC, NRI = 82.6%), and CPLM and HPTM’s prediction efficiency was preferable. Similar trends were observed in the training cohort (Table 5).

Table 5. NRI results of CPLM and HPTM when compared to AMH and AFC.

Models	Cohort	AMH	AFC
		(95% CI)	(95% CI)
CPLM vs.	Train	13.4%**	18.8%**
		(9.3%-14.2%)	(5.5%-33.1%)
	Validation	18.4%**	21.1%**
		(5.3%-19.1%)	(29.3%-49.6%)
HPLM vs.	Train	74.7%**	82.6%**
		(66.4%-83.0%)	(75.4%-89.8%)
	Validation	27.50%	37.7%**
		(11.4%-43.6%)	(21.8%-53.7%)
*, P<0.05; **, P<0.01.

Variable importance ranking in CPLM and HPTM

For facilitation of the clinical decision process, variable importance figures of CPLM and HPTM were used to investigate the models. As can be seen in Supplementary Figure 2 and Figure 4C, AMH was the most important predictor for POR, conforming to findings of the latest study which emphasized the significance of AMH [33]. Indictors including AFC and FSH that are commonly used for the assessment of ovarian response also displayed significant contribution to the objective function. In addition, HPTM highlighted the illustrious positions of E₂ level and follicle number on hCG day in the prediction of hCG pre-trigger model. However, age, dosages of Gn, E₂, therapeutic regimen, and days of Gn were proven to be slightly less significant in the models.

Potential differences between high- and low-risk group identified by CPLM or HPTM

In order to detect potential differences in clinical characteristics between the high-risk group (with a higher risk of predicting to be POR) and the low-risk group defined by CPLM and HPTM, grouped analyses were performed. As is shown in Figure 5A–5J, significant differences were discovered between both groups, with the exception of age and days of Gn. The validation results were as follows: AMH (r=-0.424, P<0.001), oocytes retrieved (r=-0.407, P=0.001), E₂ level on hCG day (r=-0.420, P<0.001), follicle number on hCG day (r=-0.405, P<0.001), AFC (r=-0.366, P<0.001), E₂ (r=0.276, P<0.001), FSH (r=0.253, P<0.001), dosages of Gn (r=0.164, P=0.003). However, training results indicated that AMH (r=-0.430, P<0.001), oocytes retrieved (r=-0.609, P=0.001), E₂ level on hCG day (r=-0.383, P<0.001), follicle number on hCG day (r=-0.405, P<0.001), AFC (r=-0.373, P<0.001), E₂ (r=0.304, P<0.001), FSH (r=0.219, P<0.001), age (r=0.146, P<0.001), dosages of Gn (r=-0.096, P=0.008), and days of Gn (r=-0.111, P=0.002), all exhibited significant differences between both groups. The results presented above further prove the efficacy of CPLM and HPTM, and emphasize the value of AMH, E₂ level on hCG day, follicle number on hCG day, AFC, E₂, and FSH.

Figure 5. Potential differences between high- and low-risk groups identified by CPLM or HPTM. (A–J) Distribution of AMH, oocytes retrieved, follicle number on hCG day, E₂ level on hCG day, AFC, E₂, FSH, dosages of Gn, age, and days of Gn in the high-risk group and poor ovarian response in the high-risk group.

Abbreviations

POR: Poor ovarian response; ART: assisted reproduction technology; COS: Controlled ovarian stimulation; RF: Random forest; XGBoost: eXtreme Gradient Boosting; SVM: Support vector machine; ANN: Artificial neural networks; CPLM: COS pre-launch model; HPTL: hCG pre-trigger model; AMH: Anti-Müllerian hormone; AFC: Antral follicle count; FSH: follicle stimulating hormone; LASSO: least absolute shrinkage and selection operator; AUC: Area under curve; NRI: net-classification index; C-index: concordance-index.

Author Contributions

S. Y.: conceptualization; S. Y. and W. J.: methodology; S. Y. and J. D: data curation; S. Y. and W. J: formal analysis; S. Y. and J. D.: writing original draft preparation; J. Y.: writing-review and editing; S. Y.: visualization; J. Y, T. Y and Y. Z: project administration; J. Y: funding acquisition.

Conflicts of Interest

The authors declared that no conflicts of interest exists.

Funding

This work was supported by the National Key Research and Development Program of China (No. 2018YFC1002804, 2016YFC1000600), and the National Natural Science Foundation of China (No. 81771618, 81971356 81771662 and 81801540).

References

• 1. La Marca A, Sunkara SK. Individualization of controlled ovarian stimulation in IVF using ovarian reserve markers: from theory to practice. Hum Reprod Update. 2014; 20:124–40. https://doi.org/10.1093/humupd/dmt037 [PubMed]
• 2. Polyzos NP, Devroey P. A systematic review of randomized trials for the treatment of poor ovarian responders: is there any light at the end of the tunnel? Fertil Steril. 2011; 96:1058–61.e7. https://doi.org/10.1016/j.fertnstert.2011.09.048 [PubMed]
• 3. Bozdag G, Polat M, Yarali I, Yarali H. Live birth rates in various subgroups of poor ovarian responders fulfilling the bologna criteria. Reprod Biomed Online. 2017; 34:639–44. https://doi.org/10.1016/j.rbmo.2017.03.009 [PubMed]
• 4. Broer SL, van Disseldorp J, Broeze KA, Dolleman M, Opmeer BC, Bossuyt P, Eijkemans MJ, Mol BW, Broekmans FJ, and IMPORT study group. Added value of ovarian reserve testing on patient characteristics in the prediction of ovarian response and ongoing pregnancy: an individual patient data approach. Hum Reprod Update. 2013; 19:26–36. https://doi.org/10.1093/humupd/dms041 [PubMed]
• 5. Kailasam C, Keay SD, Wilson P, Ford WC, Jenkins JM. Defining poor ovarian response during IVF cycles, in women aged <40 years, and its relationship with treatment outcome. Hum Reprod. 2004; 19:1544–47. https://doi.org/10.1093/humrep/deh273 [PubMed]
• 6. Surrey ES, Schoolcraft WB. Evaluating strategies for improving ovarian response of the poor responder undergoing assisted reproductive techniques. Fertil Steril. 2000; 73:667–76. https://doi.org/10.1016/s0015-0282(99)00630-5 [PubMed]
• 7. Fauser BC, Diedrich K, Devroey P, and Evian Annual Reproduction Workshop Group 2007. Predictors of ovarian response: progress towards individualized treatment in ovulation induction and ovarian stimulation. Hum Reprod Update. 2008; 14:1–14. https://doi.org/10.1093/humupd/dmm034 [PubMed]
• 8. Lin MH, Wu FS, Hwu YM, Lee RK, Li RS, Li SH. Dual trigger with gonadotropin releasing hormone agonist and human chorionic gonadotropin significantly improves live birth rate for women with diminished ovarian reserve. Reprod Biol Endocrinol. 2019; 17:7. https://doi.org/10.1186/s12958-018-0451-x [PubMed]
• 9. Zhang J, Wang Y, Mao X, Chen Q, Hong Q, Cai R, Zhang S, Kuang Y. Dual trigger of final oocyte maturation in poor ovarian responders undergoing IVF/ICSI cycles. Reprod Biomed Online. 2017; 35:701–07. https://doi.org/10.1016/j.rbmo.2017.09.002 [PubMed]
• 10. Grisendi V, Mastellari E, La Marca A. Ovarian reserve markers to identify poor responders in the context of poseidon classification. Front Endocrinol (Lausanne). 2019; 10:281. https://doi.org/10.3389/fendo.2019.00281 [PubMed]
• 11. Fleming R, Seifer DB, Frattarelli JL, Ruman J. Assessing ovarian response: antral follicle count versus anti-müllerian hormone. Reprod Biomed Online. 2015; 31:486–96. https://doi.org/10.1016/j.rbmo.2015.06.015 [PubMed]
• 12. Mutlu MF, Erdem M, Erdem A, Yildiz S, Mutlu I, Arisoy O, Oktem M. Antral follicle count determines poor ovarian response better than anti-müllerian hormone but age is the only predictor for live birth in in vitro fertilization cycles. J Assist Reprod Genet. 2013; 30:657–65. https://doi.org/10.1007/s10815-013-9975-3 [PubMed]
• 13. Zheng H, Chen S, Du H, Ling J, Wu Y, Liu H, Liu J. Ovarian response prediction in controlled ovarian stimulation for IVF using anti-müllerian hormone in Chinese women: a retrospective cohort study. Medicine (Baltimore). 2017; 96:e6495. https://doi.org/10.1097/MD.0000000000006495 [PubMed]
• 14. Scheinhardt MO, Lerman T, König IR, Griesinger G. Performance of prognostic modelling of high and low ovarian response to ovarian stimulation for IVF. Hum Reprod. 2018; 33:1499–505. https://doi.org/10.1093/humrep/dey236 [PubMed]
• 15. Tsakos E, Tolikas A, Daniilidis A, Asimakopoulos B. Predictive value of anti-müllerian hormone, follicle-stimulating hormone and antral follicle count on the outcome of ovarian stimulation in women following GnRH-antagonist protocol for IVF/ET. Arch Gynecol Obstet. 2014; 290:1249–53. https://doi.org/10.1007/s00404-014-3332-3 [PubMed]
• 16. Iliodromiti S, Nelson SM. Ovarian response biomarkers: physiology and performance. Curr Opin Obstet Gynecol. 2015; 27:182–86. https://doi.org/10.1097/GCO.0000000000000175 [PubMed]
• 17. Brodin T, Hadziosmanovic N, Berglund L, Olovsson M, Holte J. Comparing four ovarian reserve markers—associations with ovarian response and live births after assisted reproduction. Acta Obstet Gynecol Scand. 2015; 94:1056–63. https://doi.org/10.1111/aogs.12710 [PubMed]
• 18. Caraviello DZ, Weigel KA, Craven M, Gianola D, Cook NB, Nordlund KV, Fricke PM, Wiltbank MC. Analysis of reproductive performance of lactating cows on large dairy farms using machine learning algorithms. J Dairy Sci. 2006; 89:4703–22. https://doi.org/10.3168/jds.S0022-0302(06)72521-8 [PubMed]
• 19. Katz MH. Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003; 138:644–50. https://doi.org/10.7326/0003-4819-138-8-200304150-00012 [PubMed]
• 20. Kotanidis L, Nikolettos K, Petousis S, Asimakopoulos B, Chatzimitrou E, Kolios G, Nikolettos N. The use of serum anti-mullerian hormone (AMH) levels and antral follicle count (AFC) to predict the number of oocytes collected and availability of embryos for cryopreservation in IVF. J Endocrinol Invest. 2016; 39:1459–64. https://doi.org/10.1007/s40618-016-0521-x [PubMed]
• 21. Devjak R, Burnik Papler T, Verdenik I, Fon Tacer K, Vrtačnik Bokal E. Embryo quality predictive models based on cumulus cells gene expression. Balkan J Med Genet. 2016; 19:5–12. https://doi.org/10.1515/bjmg-2016-0001 [PubMed]
• 22. Mirsky SK, Barnea I, Levi M, Greenspan H, Shaked NT. Automated analysis of individual sperm cells using stain-free interferometric phase microscopy and machine learning. Cytometry A. 2017; 91:893–900. https://doi.org/10.1002/cyto.a.23189 [PubMed]
• 23. Manna C, Nanni L, Lumini A, Pappalardo S. Artificial intelligence techniques for embryo and oocyte classification. Reprod Biomed Online. 2013; 26:42–49. https://doi.org/10.1016/j.rbmo.2012.09.015 [PubMed]
• 24. Al-Azemi M, Killick SR, Duffy S, Pye C, Refaat B, Hill N, Ledger W. Multi-marker assessment of ovarian reserve predicts oocyte yield after ovulation induction. Hum Reprod. 2011; 26:414–22. https://doi.org/10.1093/humrep/deq339 [PubMed]
• 25. Speybroeck N. Classification and regression trees. Int J Public Health. 2012; 57:243–46. https://doi.org/10.1007/s00038-011-0315-z [PubMed]
• 26. Menze BH, Kelm BM, Masuch R, Himmelreich U, Bachert P, Petrich W, Hamprecht FA. A comparison of random forest and its gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinformatics. 2009; 10:213. https://doi.org/10.1186/1471-2105-10-213 [PubMed]
• 27. Mills S, Lee JK, Rassekh BM. An introduction to the civil registration and vital statistics systems with applications in low- and middle-income countries. J Health Popul Nutr. 2019 (Suppl 1); 38:23. https://doi.org/10.1186/s41043-019-0177-1 [PubMed]
• 28. Xiong J, Cai L, Wang F, He X. SVM-based spectral analysis for heart rate from multi-channel WPPG sensor signals. Sensors (Basel). 2017; 17:506. https://doi.org/10.3390/s17030506 [PubMed]
• 29. Serpen G, Corra J. Training simultaneous recurrent neural network with resilient propagation for static optimization. Int J Neural Syst. 2002; 12:203–18. https://doi.org/10.1142/S0129065702001199 [PubMed]
• 30. Held U, Bové DS, Steurer J, Held L. Validating and updating a risk model for pneumonia - a case study. BMC Med Res Methodol. 2012; 12:99. https://doi.org/10.1186/1471-2288-12-99 [PubMed]
• 31. Longato E, Vettoretti M, Di Camillo B. A practical perspective on the concordance index for the evaluation and selection of prognostic time-to-event models. J Biomed Inform. 2020; 108:103496. https://doi.org/10.1016/j.jbi.2020.103496 [PubMed]
• 32. Broer SL, Dólleman M, van Disseldorp J, Broeze KA, Opmeer BC, Bossuyt PM, Eijkemans MJ, Mol BW, Broekmans FJ, and IPD-EXPORT Study Group. Prediction of an excessive response in in vitro fertilization from patient characteristics and ovarian reserve tests and comparison in subgroups: an individual patient data meta-analysis. Fertil Steril. 2013; 100:420–9.e7. https://doi.org/10.1016/j.fertnstert.2013.04.024 [PubMed]
• 33. Moolhuijsen LM, Visser JA. Anti-müllerian hormone and ovarian reserve: update on assessing ovarian function. J Clin Endocrinol Metab. 2020; 105:3361–73. https://doi.org/10.1210/clinem/dgaa513 [PubMed]
• 34. Uyar A, Bener A, Ciray HN. Predictive modeling of implantation outcome in an in vitro fertilization setting: an application of machine learning methods. Med Decis Making. 2015; 35:714–25. https://doi.org/10.1177/0272989X14535984 [PubMed]
• 35. Chen F, De Neubourg D, Debrock S, Peeraer K, D’Hooghe T, Spiessens C. Selecting the embryo with the highest implantation potential using a data mining based prediction model. Reprod Biol Endocrinol. 2016; 14:10. https://doi.org/10.1186/s12958-016-0145-1 [PubMed]
• 36. Verhagen TE, Hendriks DJ, Bancsi LF, Mol BW, Broekmans FJ. The accuracy of multivariate models predicting ovarian reserve and pregnancy after in vitro fertilization: a meta-analysis. Hum Reprod Update. 2008; 14:95–100. https://doi.org/10.1093/humupd/dmn001 [PubMed]
• 37. Silberstein T, MacLaughlin DT, Shai I, Trimarchi JR, Lambert-Messerlian G, Seifer DB, Keefe DL, Blazar AS. Mullerian inhibiting substance levels at the time of HCG administration in IVF cycles predict both ovarian reserve and embryo morphology. Hum Reprod. 2006; 21:159–63. https://doi.org/10.1093/humrep/dei270 [PubMed]
• 38. Riggs RM, Duran EH, Baker MW, Kimble TD, Hobeika E, Yin L, Matos-Bodden L, Leader B, Stadtmauer L. Assessment of ovarian reserve with anti-Müllerian hormone: a comparison of the predictive value of anti-Müllerian hormone, follicle-stimulating hormone, inhibin B, and age. Am J Obstet Gynecol. 2008; 199:202.e1–8. https://doi.org/10.1016/j.ajog.2008.05.004 [PubMed]
• 39. Xu H, Zeng L, Yang R, Feng Y, Li R, Qiao J. Retrospective cohort study: AMH is the best ovarian reserve markers in predicting ovarian response but has unfavorable value in predicting clinical pregnancy in GnRH antagonist protocol. Arch Gynecol Obstet. 2017; 295:763–70. https://doi.org/10.1007/s00404-016-4274-8 [PubMed]
• 40. Lan VT, Linh NK, Tuong HM, Wong PC, Howles CM. Anti-müllerian hormone versus antral follicle count for defining the starting dose of FSH. Reprod Biomed Online. 2013; 27:390–99. https://doi.org/10.1016/j.rbmo.2013.07.008 [PubMed]
• 41. Chalumeau C, Moreau J, Gatimel N, Cohade C, Lesourd F, Parinaud J, Leandri R. Establishment and validation of a score to predict ovarian response to stimulation in IVF. Reprod Biomed Online. 2018; 36:26–31. https://doi.org/10.1016/j.rbmo.2017.09.011 [PubMed]
• 42. Melo MA, Garrido N, Alvarez C, Bellver J, Meseguer M, Pellicer A, Remohí J. Antral follicle count (AFC) can be used in the prediction of ovarian response but cannot predict the oocyte/embryo quality or the in vitro fertilization outcome in an egg donation program. Fertil Steril. 2009; 91:148–56. https://doi.org/10.1016/j.fertnstert.2007.11.042 [PubMed]
• 43. Jamil Z, Fatima SS, Ahmed K, Malik R. Anti-mullerian hormone: above and beyond conventional ovarian reserve markers. Dis Markers. 2016; 2016:5246217. https://doi.org/10.1155/2016/5246217 [PubMed]
• 44. Li R, Gong F, Zhu Y, Fang W, Yang J, Liu J, Hu L, Yang D, Liang X, Qiao J. Anti-müllerian hormone for prediction of ovarian response in Chinese infertile women undergoing IVF/ICSI cycles: a prospective, multi-centre, observational study. Reprod Biomed Online. 2016; 33:506–12. https://doi.org/10.1016/j.rbmo.2016.07.003 [PubMed]
• 45. Christensen MW, Kesmodel US, Christensen K, Kirkegaard K, Ingerslev HJ. Early ovarian ageing: is a low number of oocytes harvested in young women associated with an earlier and increased risk of age-related diseases? Hum Reprod. 2020; 35:2375–90. https://doi.org/10.1093/humrep/deaa188 [PubMed]
• 46. Peluso C, Oliveira R, Laporta GZ, Christofolini DM, Fonseca FL, Laganà AS, Barbosa CP, Bianco B. Are ovarian reserve tests reliable in predicting ovarian response? results from a prospective, cross-sectional, single-center analysis. Gynecol Endocrinol. 2021; 37:358–66. https://doi.org/10.1080/09513590.2020.1786509 [PubMed]
• 47. Lee RW, Khin LW, Hendricks MS, Tan HH, Nadarajah S, Tee NW, Loh SF, Tai BC, Chan JK. Ovarian biomarkers predict controlled ovarian stimulation for in vitro fertilisation treatment in Singapore. Singapore Med J. 2020; 61:463–68. https://doi.org/10.11622/smedj.2020130 [PubMed]
• 48. Reljic M, Vlaisavljevic V, Gavric V, Kovacic B, Cizek-Sajko M. Value of the serum estradiol level on the day of human chorionic gonadotropin injection and on the day after in predicting the outcome in natural in vitro fertilization/intracytoplasmic sperm injection cycles. Fertil Steril. 2001; 75:539–43. https://doi.org/10.1016/s0015-0282(00)01735-0 [PubMed]
• 49. Vural B, Cakiroglu Y, Vural F. The predictor markers of ovarian response in poor responders under 40 years of age. Clin Exp Obstet Gynecol. 2016; 43:650–53. [PubMed]
• 50. Sahmay S, Cetin M, Ocal P, Kaleli S, Senol H, Birol F, Irez T. Serum anti-müllerian hormone level as a predictor of poor ovarian response in in vitro fertilization patients. Reprod Med Biol. 2010; 10:9–14. https://doi.org/10.1007/s12522-010-0066-1 [PubMed]
• 51. Toth TL, Awwad JT, Veeck LL, Jones HW Jr, Muasher SJ. Suppression and flare regimens of gonadotropin-releasing hormone agonist. Use in women with different basal gonadotropin values in an in vitro fertilization program. J Reprod Med. 1996; 41:321–26. [PubMed]
• 52. Abbara A, Vuong LN, Ho VN, Clarke SA, Jeffers L, Comninos AN, Salim R, Ho TM, Kelsey TW, Trew GH, Humaidan P, Dhillo WS. Follicle size on day of trigger most likely to yield a mature oocyte. Front Endocrinol (Lausanne). 2018; 9:193. https://doi.org/10.3389/fendo.2018.00193 [PubMed]
• 53. Siddhartha N, Reddy NS, Pandurangi M, Tamizharasi M, Radha V, Kanimozhi K. Correlation of serum estradiol level on the day of ovulation trigger with the reproductive outcome of intracytoplasmic sperm injection. J Hum Reprod Sci. 2016; 9:23–27. https://doi.org/10.4103/0974-1208.178631 [PubMed]