Comparing an artificial neural network to logistic regression for predicting ED visit risk among patients with cancer: a population-based cohort study

Background — Prior work using symptom burden to predict ED visits among cancer patients has utilized traditional statistical methods such as logistic regression. Machine learning approaches for prediction, such as artificial neural networks, are gaining attention but are yet to be commonly applied in practice.

Methods — This was a population-based study of patients diagnosed with cancer between 2007 and 2015 in Ontario, Canada. After splitting the cohort into training and test sets, an artificial neural network (ANN) model and a logistic regression (LR) model were developed on the training cohort to predict the risk of an ED visit within 7 days following an assessment of symptom burden. The predictive performance of each risk model was assessed on the test cohort and compared with respect to area under the curve and calibration.

Results — The training cohort consisted of 170,092 patients undergoing 1,015,125 symptom assessments and the remaining 42,523 patients undergoing 252,169 symptom assessments were set aside as the test cohort. Both models performed similarly with respect to specificity (ANN 67.0%, LR 67.3%) and accuracy (ANN 67.1%, LR 67.2%), and only minor improvement was found with respect to sensitivity (ANN 68.9%, LR 67.1%), discrimination (ANN 74.3%, LR 73.7%), and calibration under the ANN model compared to the logistic regression model. The most notable improvement in calibration was found among patients in the highest ED visit risk percentile.

Conclusion — Although both models were similar in predictive performance using our data, ANNs have an important role in prediction due to their flexible structure and data-driven distribution-free benefits, and should thus be considered as a potential modeling approach when developing a prediction tool.