This article investigates the use of a video-based deep neural network for predicting left ventricular ejection fraction (LVEF) from angiograms. Data from 3,404 patients with both angiograms and transthoracic echocardiograms (TTE) were used as the training and evaluation data sets for the video-based deep neural network, called CathEF, and then externally validated with a testing data set made from randomly selected coronary angiogram studies from another patient cohort with TTEs. The LVEF of the neural network derived from the angiograms was compared to the TTE LVEF certified by cardiologists. In the evaluation set, the neural network could sort between LVEF greater than 40% and LVEF lower than 40% with an AUROC value of 0.911. Reduced LVEF values were 22.7 times more likely to be predicted by the neural network as LVEF less than 40%, also showing high discriminatory value. The neural network’s predictions of LVEF were correlated with the TTE LVEF, with an ICC value of 0.77 and a Pearson r  value of  0.71, indicating high agreement between the values. The algorithm remained consistent across biological factors. For the external validation testing set, the neural network had an AUROC of 0.906 for predicting LVEF greater than or below 40%, as well as an ICC value of 0.62 and Pearson’s r value of  0.72 with TTE LVEF, indicating strong correlation. The performance of the neural network remained consistent even when only provided with the three most common angiographic angles. This article is the first time that angiogram data has been used by a machine learning algorithm to provide LVEF. This could give physicians a novel and more accessible way of determining LVEF compared to the standard TTE approach.