This multicenter, prospective study evaluated the performance of machine learning algorithms in predicting mental health conditions using brain MRI data. The sample was split into two groups of individuals with and without diagnoses of psychiatric or neurological illness. The control group consisted of 27117 individuals in a training set, 26985 individuals in a test set, and 1757 individuals in a clinical set. The clinical set in particular measured the algorithms' capability to generalize to real patient cases, serving as external validation. MRI data for individuals with diagnoses were extracted from several established neuroimaging research datasets. Nearest-neighbor matching was used based on factors such as sex and age to ensure that identifications made by algorithms would only be based on biomarkers in MRI data rather than demographics. Model classes included gated recurrent units (GRU), multilayer perceptrons (MLPs), XGBoost, support vector machines (SVMs), and Random Forest, with predictions compared to clinical diagnoses. No temporal validation was performed, as algorithm performance was not tested across different time periods. Algorithms achieved stronger predicting performance for autism spectrum disorder and schizophrenia, with AUROC values ranging from 0.56 to 0.65. Prediction performance for mild cognitive impairment, bipolar disorder, and Alzheimer’s disease did not rise above chance level.