Hybrid BCI-based instruction set for dual robotic arm control using EEG and eye movement signals

Comprehensive Summary

This paper presents a hybrid brain–computer interface (hBCI) framework designed to enable high-degree-of-freedom control of dual collaborative robotic arms using a combination of motor imagery (MI) EEG signals and eye-movement signals. The central challenge addressed is the limited number of reliably distinguishable commands obtainable from non-invasive EEG alone, which constrains the control of complex robotic systems. To overcome this, the authors introduce an extended instruction set that hierarchically combines EEG-based MI classification with gaze-based symbolic selection, significantly expanding the effective command space without increasing cognitive burden or the number of MI classes. At the signal-processing level, the study proposes a lightweight spatiotemporal convolution–attention (ST2A) neural network for MI decoding. This model integrates temporal and spatial convolutions with a Transformer-based attention mechanism and a Squeeze-and-Excitation (SE) block to improve robustness against EEG non-stationarity and inter-subject variability. The decoder achieves a mean classification accuracy of 83.8% on the BCI Competition IV-2a dataset while using only 8.8k parameters, demonstrating strong computational efficiency relative to existing CNN- or Transformer-based approaches. Importantly, eye-movement signals are not directly fused at the feature level but instead serve as a symbolic control layer for arm selection, reducing interference between modalities. Building on this decoding framework, the authors design a hierarchical command-expansion strategy that maps four MI classes (left hand, right hand, feet, tongue) and gaze direction into 12 executable robotic commands, enabling three-axis Cartesian motion control for each robotic arm. The system is validated through kinematic and three-dimensional simulations of dual 6-DOF UR3e collaborative robotic arms in the Webots environment. The results demonstrate that complex grasping and positioning tasks can be achieved using discrete MI-based commands augmented by gaze, confirming the feasibility and scalability of the proposed approach for multi-DoF robotic control.

Outcomes and Implications

The proposed hybrid BCI framework has significant implications for assistive and rehabilitative medicine, particularly for individuals with severe motor impairments such as spinal cord injury, stroke, amyotrophic lateral sclerosis (ALS), or advanced neuromuscular disorders. By relying entirely on non-invasive EEG and eye-tracking, the system avoids the surgical risks associated with invasive BCIs while still enabling control over complex assistive devices. The ability to operate dual robotic arms expands functional independence beyond simple single-limb assistance, supporting activities of daily living that require bilateral coordination, such as object manipulation, feeding, or dressing. From a rehabilitation perspective, the use of motor imagery as the primary control paradigm aligns well with neuroplasticity-driven recovery strategies. Repeated MI-based interaction with robotic devices may reinforce sensorimotor cortical activation, potentially contributing to motor recovery when integrated into neurorehabilitation programs. Furthermore, the lightweight and efficient decoding architecture enhances the feasibility of real-time, portable BCI systems suitable for clinical environments. Overall, this work supports the translation of hybrid BCI-controlled robotics from laboratory demonstrations toward clinically viable assistive technologies, provided that future studies address online implementation, long-term usability, and patient safety.