Explainable AI for sign language recognition models: Integrating Grad-Cam LIME and Integrated Gradients

Comprehensive Summary

El-Qoraychy et. al examines how explainable artificial intelligence (XAI) can improve the reliability and transparency of sign language recognition systems using deep learning. While general sign language recognition models have high accuracy, their operations are similar to a black box, making it difficult to create assistive technology that creates supportive communication. For this, researchers developed a framework that incorporates explainability methods into a recognition pipeline with a convolutional neural network. The VGG19 architecture was used to build a core classifier and static images of standard RGB and binary hand mask images were used for training. With U-net segmentation and data augmentation to isolate hand shape and reduce overfitting respectively, XAI techniques Grad-CAM, LIME, and Integrated Gradients were used for model decision making. Visual classification was found to be based on distinct hand regions like fingers and palm contours, but segmentation in data processing often removes subtle cues that are needed for better discernment. Overall, integrating XAI into sign language recognition is essential for building trustworthy, fair, and deployable assistive AI systems.

Outcomes and Implications

This study improves the use of artificial intelligence for assistive communication, clinical accessibility, and equitable healthcare delivery. Patients who are deaf or hard-of-hearing can now have improved communication with the people around them as well as healthcare providers, preventing miscommunication and ensuring a high standard of care. This technology could also be improved with a larger dataset to ensure that minority patient populations are also represented with aspects like skin tone, hand shape, and motor differences. Transparent gesture recognition models could help restore function and document after stroke, traumatic brain injury, and neurodegenerative diseases by tracking specific hand movements. Explainable AI can aid regulatory approval and ethical deployment of medical AI tools, especially with the pressing need for clinical decision support systems.