The main objective of this study was to evaluate how well large language models, ChatGPT and Gemini, know information related to the field of child and adolescent psychiatry. To test the knowledge of these language models, 150 multiple choice questions were selected from a specialty board review study guide with five answer choices. Each question was asked 10 times with randomized answer orders to reduce bias. All the models had scored between 68.3% and 78.9% on the questions. Gemini 2.0 Flash (76.3%) outperformed its predecessor Gemini 1.5 Flash (68.3%). ChatGPT 4o (78.9%) outperformed ChatGPT o1-mini (76.7%). Each model did better in topics with well-defined diagnostic criteria such as schizophrenia and eating disorders but struggled in certain topics such as psychopharmacology or anxiety disorders. In the discussion, the authors talked about how the language models were about on par with how well humans do on the same questions. Another point raised by the authors is how language models should also be tested with open-ended and case-based questions to test potential of language models in the field.