DermaGPT: A federated multimodal framework with a meta-learned trust function for interpretable dermatology diagnostics

Comprehensive Summary

The following study introduces DermaGPT, a privacy-aware, federated multimodal AI framework designed to deliver interpretable dermatologic diagnosis and structured patient explanation. The system combines a PaLI-Gemma 2 vision-language backbone (fine-tuned via LoRA with ~30M trainable parameters) for image-based diagnosis with a separate explanatory large language model (LLM) module. Training was performed across four institutional datasets using federated learning (FL), preventing centralized image pooling. A novel Meta-Learned Trust Function (MLTF) dynamically reweights client updates based on uncertainty, calibration error, validation loss, and domain-shift indicators, improving cross-site robustness. On an external biopsy-comfirmed Stanford cohort of 4,452 images, DermaGPT achieved 90.2% multi-class diagnostic accuracy across 11 lesion types and 93.3% accuracy for benign vs. malignant classification, with improved calibration (3.5% ECE vs 4.2% in non-federated baselines). The architecture explicitly decouples diagnostic prediction from language generation to reduce the risk of hallucination and enhance interpretability.

Outcomes and Implications

DermaGPT represents a significant advancement in trust-aware, federated dermatology AI, addressing three persistent barriers to clinical deployment: computational inefficiency, privacy risks, and limited interpretability. The integration of MLTF improves fairness and robustness across heterogeneous hospital datasets without centralized validation data. The Advanced Retrieval-Augmented Generation (A-RAG) module significantly improved explanation quality and reduced hallucination rate (e.g., DeepSeek-V3 factual accuracy improved from 86.1% to 94.3%). Importantly, dermatologist evaluation confirmed that AI-generated explanations were clinically useful, patient-friendly, and grounded in dermatologic knowledge. However, the authors highlight safety-critical failure modes–such as confidently incorrect explanation when upstream predictions are wrong–reinforcing the system’s role as a clinical decision-support adjunct, not a replacement for physician judgment. By combining federated privacy preservation, efficient edge deployment (0.1-0.5 s diagnostic latency), and interpretable multimodal reasoning, DermaGPT advances scalable, ethically aligned dermatologic AI suitable for real-world healthcare environments.