This study explored whether the large language model, GPT-4o, could correctly identify dietary errors in hospital menus for patients recovering from stroke. Researchers conducted a retrospective validation study at a neurorehabilitation hospital, analyzing 264 real inpatient menus across 22 diet types, including diabetic diets, allergen-free diets, and dysphagia diets graded by the Penetration-Aspiration Scale. GPT-4o’s classifications (0 errors, 1 error, or 2+ errors) were compared with expert dietitians’ assessments. The model showed high agreement, with an overall accuracy of 92.8% and strong inter-rater reliability (Cohen’s Kappa = 0.892, 95% CI 0.845–0.939). The sensitivity ranged from 86.2% for single-error menus to 97.8% for multiple-error menus, and specificity ranged from 92.6% to 100%. Most disagreements (24 of the 26 cases) were false positives, especially in dysphagia and diabetic diets, where GPT-4o applied dietary rules more rigidly than the experts. Thematic analysis revealed that AI often overlooked real-world food preparation, patient tolerance, and contextual clinical judgment.