Comprehensive Summary
Demi̇r et al. conducted a comparative evaluation of ChatGPT-4o and Gemini Pro in generating structured abstracts for orthodontic systematic reviews and meta-analyses following the PRISMA-A (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstracts) framework. The researchers compiled 162 full-text articles from Q1 orthodontic journals published since 2019 and applied a custom-engineered prompt designed to align with PRISMA-A checklist criteria. Each model’s outputs were scored using an Overall Quality Score (OQS) derived from 11 checklist items. Statistical analyses, including Mann-Whitney U tests and Intraclass Correlation Coefficients, were used to assess inter-rater reliability and model consistency. Both LLMs demonstrated high reliability and produced abstracts that adhered well to PRISMA-A standards, but ChatGPT-4o achieved significantly higher OQS values (mean = 21.67 ± 0.53) than Gemini Pro (mean = 21.00 ± 1.00; p < 0.001). The most notable performance differences were seen in the “Included Studies,” “Synthesis of Results,” and “Funding” sections, where ChatGPT-4o produced more complete and accurate content. The findings suggest that prompt engineering and multimodal input preparation (including preserved tables and figures) substantially improve LLM performance in generating structured, guideline-compliant scientific summaries.
Outcomes and Implications
This study highlights the growing potential of AI tools like ChatGPT-4o and Gemini Pro to assist researchers in producing standardized, evidence-based scientific writing. By demonstrating that LLMs can reliably generate PRISMA-A-compliant abstracts, the research points toward a future where AI may streamline academic reporting, reduce the manual workload of writing structured abstracts, and accelerate the dissemination of systematic-review evidence in dentistry and other medical fields. ChatGPT-4o’s superior handling of complex data elements, particularly table-based quantitative information and synthesis statements, suggests that it may be a more effective assistant for producing publication-ready abstracts. However, the authors emphasize that these models should complement rather than replace expert oversight, since accuracy, interpretation, and ethical integrity still rely on human review. As leading journals increasingly require compliance with frameworks such as PRISMA and CONSORT, this structured AI-prompting approach could support more efficient and transparent scientific communication across disciplines.