Experimental design of BMI training

BMI training was applied to 10 patients with phantom limb due to brachial plexus root avulsion (n=9) and amputation of the right forearm (n=1; Table 1 and Supplementary Table 1)20. All patients had phantom limb symptoms, and the associated pain is consequently thought to share a common mechanism with phantom limb pain due to amputation3. All patients participated in BMI training to control the robotic hand (Fig. 1a). Cortical plasticity was evaluated by comparing the same offline task to move the phantom hand before and after each training session (Fig. 1b). At the same time, pain was evaluated with a visual analogue scale (VAS) and the Japanese version of the short-form McGill Pain Questionnaire 2 (SF-MPQ2)27.

Table 1 Clinical profiles of patients. Full size table

Figure 1: BMI training and experimental design. (a) BMI training. Patients were instructed to control the prosthetic hand by moving their phantom hands in each experiment. (b) A diagram of the tasks in each experiment. First, the patients performed the offline phantom hand task to move their phantom hand according to the instructions. Then, after evaluation of their pain, BMI training was performed for 10 min. Here three types of decoders were used to control the prosthetic hand, each for three experiments. After evaluation of their pain, the same offline phantom hand task was performed. *1For the experiment with the real hand decoder, the patients also performed the offline task with their intact hand after the task with their phantom hand. Full size image

Decoding of the phantom hand movement using MEG signals

First, we performed an ‘offline’ task (pre-BMI) in which patients were instructed to try to move their phantom hands to be in the posture of grasping or opening according to given instructions (Fig. 2a,b)16. The patients could not see their actual arms during the experiment. The MEG signals from 84 selected sensors were recorded during the task (Fig. 2c) and then time-averaged using windows of 500 ms slid by 100 ms, in the range from −500 to 1,000 ms with respect to the timings of the execution cues. Then, the averaged signals were converted into z-scores using the mean and s.d. estimated from the initial 50 s of the offline task.

Figure 2: Offline evaluation of movements of real hands and phantom hands. (a) Schematic figure representing the offline task. The patients intended to move their phantom hand according to the instruction presented on the screen in front of them. (b) Experimental paradigm. An epoch began with a 3-s visual presentation of a black cross. A Japanese word was shown for 1 s to instruct the subjects which movement to perform. After two 1-s timing cues, the execution cue of the cross sign was presented for 0.5 s with a sound. The patients performed the instructed movements once when the execution cue was presented. These cues with sounds were repeated four times for each instruction. Each of the movement types was assigned in random order 10 times each. (c) We recorded the MEG signals of the 84 selected sensors, which are shown as red points on the picture of the sensor location. (d) The average classification accuracy of movement types using the z-scored MEG sensor signals (mean and 95% confidence interval). Real hand (black), phantom hand (red) and the randomly relabelled data of the real hand (grey) and phantom hand (orange; n=10). **P<0.01, Bonferroni-corrected, two-tailed Student’s t-test. Full size image

A nested 10-fold cross-validation28 was performed with a support vector machine (SVM) to evaluate the accuracy of classifying the performed movement types using the z-scores of the selected MEG sensors20. The accuracy was evaluated for each 500-ms window, and the maximum value of the accuracies was estimated as the classification accuracy. The classification accuracy of the phantom movement during the pre-BMI task was 65.6±9.4% (mean±95% confidence interval, n=10), which was significantly greater than the accuracies for classifying the same signals with randomly relabelled movement types (50.9±4.3%, P=0.0012, Bonferroni-corrected, t(18)=2.79, two-tailed Student’s t-test, n=10; Fig. 2d).

For comparison, the patients performed actual movements of their intact hands according to the same instructions during another experiment. The classification accuracy was evaluated by the same method. For the intact hand movements before the BMI training, the classification accuracy was 75.3±7.5% (mean±95% confidence interval, n=10), which exceeded the accuracies of randomized movement types (46.0±6.3%, P=0.000016, Bonferroni-corrected, t(18)=5.83, two-tailed Student’s t-test, n=10; Fig. 2d). Notably, the classification accuracies were comparable between the phantom movements and the intact hand movements (n=10, P=0.13, Bonferroni-corrected, t(18)=1.57, two-tailed Student’s t-test), although the electromyographic signals of the affected hands were not comparable to those of the intact hands (Supplementary Fig. 1).

Estimated cortical currents

We evaluated the cortical representation during the offline task using cortical current source estimation and decoding accuracy. First, the cortical currents were estimated from the obtained MEG signals using variational Bayesian multimodal encephalography (VBMEG)29. Next, the estimated cortical currents were averaged using a 500-ms time window from the execution cue and z-scored by the estimated cortical currents of the initial 50-s period of the offline session. The z-scored cortical currents were averaged for the 10 patients and colour-coded on the normalized brain surface. The right hand was designated as the affected hand; for those two patients with affected left hands, the cortical activation was switched. Figure 3a shows the cortical currents activated on the sensorimotor cortex contralateral to the phantom hand when the patients attempted to move their phantom hands.

Figure 3: Estimated cortical currents and the decoding of movements. (a) The mean z-scored cortical currents during grasping and opening of the phantom hand were colour-coded on a normalized brain surface at the time of movement onset (the side of the phantom hand is shown on the right, n=10). (b) The mean F-values of ANOVA of the z-scored cortical currents between the two movements were colour-coded on the normalized brain surface (the side of the phantom hand is shown on the right, n=10). (c) The mean z-scored cortical currents during grasping and opening of the real hand (n=10). (d) The mean F-values of ANOVA of the z-scored cortical currents between the two movements of the real hand (n=10). (e) The 126 vertices were selected in the sensorimotor cortex contralateral (red) and ipsilateral (blue) to the phantom hand. (f) The classification accuracy of movement types using the z-scored cortical currents on the sensorimotor cortex is shown with the 95% confidence interval for each hemisphere of each hand (n=10; black, real hand; red, phantom hand). The asterisks denote significant differences (*P<0.05, **P<0.01, Bonferroni-corrected, two-tailed Student’s t-test). The dashed, grey line indicates the accuracy by chance (50%). Full size image

The z-scored cortical currents were compared between two types of movements with a one-way analysis of variance (ANOVA) for each vertex. The F-value of the ANOVA was averaged for the 10 patients and colour-coded on the normalized brain surface. Figure 3b shows that cortical currents on the contralateral sensorimotor cortex varied between movement types with high F-values (see also Supplementary Fig. 2).

Similarly, for the intact hand movements, the cortical currents were activated on the sensorimotor cortex contralateral to the moved hand, depending on the movement types, with high F-values (Fig. 3c,d). The cortical currents on the contralateral sensorimotor cortex varied according to the movement types for both the real and the phantom hand.

Next, the decoding method was applied for the z-scored cortical currents to quantify how accurately the cortical currents represent the two types of phantom movements. A total of 126 vertices were selected on the sensorimotor cortex of each hemisphere (Fig. 3e). The z-scored cortical currents were estimated at the selected vertices from −500 to 1,000 ms for each 100 ms. Using the same method for the z-scored MEG sensor signals, the z-scored cortical currents were evaluated to determine the accuracy for classifying the movement types.

In all patients, the classification accuracies varied significantly between each side of the sensorimotor cortex for the movements of the real and phantom hands (n=10 each, P=0.0032, F(3, 36)=5.52, one-way ANOVA; Fig. 3f). The accuracy for classifying the phantom movement was significantly higher using the currents on the sensorimotor cortex contralateral to the phantom hand compared with the currents on the ipsilateral cortex (n=10, P=0.007, Bonferroni-corrected, t(18)=3.06, two-tailed Student’s t-test; Fig. 3f). Thus, the classifier clearly distinguished the information on the ipsilateral and contralateral hemispheres. Moreover, the accuracy of the real hand movement with the contralateral sensorimotor cortex was significantly higher than the accuracy of the phantom movement with the same hemisphere, ipsilateral to the phantom hand (n=10, P=0.00017, Bonferroni-corrected, t(18)=4.72, two-tailed Student’s t-test; Fig. 3f). This indicates the specificity of motor information represented in the sensorimotor cortex.

The classification accuracy using the cortical currents contralateral to the moved hand was not significantly different between the phantom hand and real hand (n=10, P=0.37, Bonferroni-corrected, t(18)=0.93, two-tailed Student’s t-test). Moreover, the classification accuracies of the phantom movements before BMI training were not significantly different in any experiment (n=10 each, P=0.75, F(2, 27)=0.29, one-way ANOVA; Supplementary Fig. 3). Thus, in summary, the representation of the hand’s movement was preserved in the contralateral sensorimotor cortex, even for phantom movements.

BMI training with a neuroprosthetic hand

The BMI training to control the robotic hand was performed as a randomized crossover trial consisting of two training sessions on two different days (Fig. 1b). Each training session was performed with two different decoders to control the robotic hand: a phantom decoder and a random decoder. Using the z-scored MEG sensor signals of the offline tasks to move the phantom hand, we constructed the phantom decoder to infer phantom hand movements at an arbitrary time in order to control the robotic hand in real time20. In contrast, the random decoder was constructed from the MEG signals of the same task with randomly relabelled movement types.

For training with either decoder, the patients were instructed to freely control the neuroprosthetic hand for grasping and releasing a ball by trying to move their phantom hands while watching the movement of the prosthetic hand in closed-loop conditions (Supplementary Movie 1 and Fig. 1a). The neuroprosthetic hand was controlled according to the movements inferred by a selected decoder with the z-scored MEG sensor signals obtained online. The patients performed the experiments twice with each decoder selected randomly for the crossover portion of the trial without knowing the type of the decoder.

After each experiment we asked the patients about their feelings during controlling the neuroprosthesis. Nine patients reported improvement of their control after training with the phantom decoder (Supplementary Table 2).

Effects of BMI training on cortical currents and pain

For each offline task, the F-values of the z-scored cortical currents at the execution cue were averaged across the 10 patients (Fig. 4a–c). After BMI training with the phantom decoder, the F-values increased in the contralateral sensorimotor cortex (Fig. 4a). Surprisingly, the pain scores also significantly increased from 38.2±18.5 (mean±95% confidence interval) to 45.8±18.4 in VAS (1/100; n=10, P=0.0066, uncorrected, t(9)=3.51, paired Student’s t-test), whereas the total scores of the SF-MPQ2 did not significantly increase, changing from 20.4±15.2 to 23.8±17.1 (n=10, P=0.086, uncorrected, Wilcoxon signed-rank test).

Figure 4: Sensorimotor plasticity induced by BMI training. (a–c) Among 10 patients, the F-value for the two phantom movements was averaged at each vertex and colour-coded on the normalized brain surfaces for the pre-BMI (upper panel), post-BMI (middle panel) and the difference between the pre-BMI and post-BMI (lower panel). Each raw image corresponds to each experiment: (a) phantom hand decoder; (b) random decoder; and (c) real hand decoder. (d,e) The difference in the F-value between pre-BMI and post-BMI compared with that of the random decoder. (d) ΔF of the phantom hand decoder−ΔF of the random decoder; (e) ΔF of the real hand decoder−ΔF of the random decoder (uncorrected paired Student’s t-test). Full size image

On a different day, the same patients were trained with a random decoder. Using this decoder, the classification accuracies of the phantom movements were similar to chance (47.5±4.46%, n=10). After BMI training, the F-value of the contralateral sensorimotor cortex did not increase, although the patient was instructed in the same way as during the first experiment (Fig. 4b). Moreover, the pain scores were not significantly altered, changing from 31.9±15.9 (mean±95% confidence interval) to 32.9±15.9 in VAS (1/100; n=10, P=0.46, uncorrected, t(9)=0.77, paired Student’s t-test) and from 21.2±19.9 to 23.1±20.8 in total scores of the SF-MPQ2 (n=10, P=0.063, uncorrected, Wilcoxon signed-rank test). Notably, the order of the two experiments was random (Supplementary Table 3).

As expected, BMI training with the phantom decoder increased the discriminability of the cortical activity representing the phantom hand movements. However, contrary to the naive hypothesis, pain significantly increased after training with the phantom decoder. Therefore, we added a subsequent training experiment with a real hand decoder to reduce the discriminability. The real hand decoder was constructed from MEG signals obtained while moving the intact hand. During training with the real hand decoder, the patients were similarly instructed to freely control the neuroprosthetic hand by trying to move their phantom hand, not the intact hand. Therefore, the patients intended to associate their phantom hand movements with the movements of the prosthetic hand, which was actually controlled by a decoder to classify the MEG signals based on the intact hand’s movement. As a result, the patients were expected to unknowingly associate the phantom movements with the cortical representation of the intact hand’s movements, which were different from the cortical representation of the phantom movements in pre-BMI training. We expected that BMI training with the real hand decoder would accelerate the dissociation of the link between the phantom hand and the original cortical representation by creating a new link to the real hand. The association of the different neural representation might dissociate the prosthetic hand and the original neural representation of phantom movements even more so than the association of the randomly moved prosthetic hand and the neural representation.

Consistent with this prediction, after BMI training with the real hand decoder, the F-values of the phantom hand movements decreased for the sensorimotor cortex contralateral to the phantom hand (Fig. 4c). Moreover, the pain scores decreased significantly from 38.3±15.5 (mean±95% confidence interval) to 34.6±14.8 in VAS (1/100; n=10, P=0.029, uncorrected, t(9)=2.60, paired Student’s t-test). Similarly, the total scores of the SF-MPQ2 significantly decreased from 26.0±21.0 to 20.7±16.3 (n=10, P=0.016, uncorrected, Wilcoxon signed-rank test). The F-values of the intact hand movements decreased for the sensorimotor cortex contralateral to the intact hand (Supplementary Fig. 4).

To statistically evaluate plastic changes of the discriminability in cortical currents, the differences in the F-values before and after training (post–pre) were compared among the training sessions (also see Supplementary Fig. 5 for the alteration in cortical currents). The increases in the F-values on the contralateral sensorimotor cortex were significantly larger for the phantom hand decoder than the random decoder (Fig. 4d). In contrast, the decreases in the F-values were significantly larger for the real hand decoder than the random decoder (Fig. 4e). Thus, the F-values on the contralateral sensorimotor cortex varied significantly after BMI training depending on the decoders.

According to their reports, patients noticed a difference in their ability to control the robotic hand. They attributed the ability to use the robotic hand to themselves; however, they were not aware of the experimental manipulations (Supplementary Table 2). Moreover, no patients reported any subjective feelings of being in control of the prosthetic hand as a part of their body.

Pain and discriminability of cortical currents

The above analyses showed that the increases in the pain VAS scores were significantly changed depending on the decoder type (n=10 each, P=0.0002, F(2, 27)=11.5, one-way ANOVA; Fig. 5a). After training with the real hand decoder, the VAS scores decreased significantly compared with those of the random decoder and the phantom decoder (n=10, P=0.025 and 0.0003, uncorrected, t(18)=2.45 and 4.36, respectively, two-tailed Student’s t-test). In contrast, the VAS scores increased significantly after training with the phantom decoder compared with the random decoder (n=10, P=0.017, uncorrected, t(18)=2.62, two-tailed Student’s t-test). Notably, these increased scores from the phantom decoder spontaneously returned to the previous state after more than 2 weeks and were not significantly different from the scores before the training (n=10, P=0.55, uncorrected, t(9)=0.63, paired Student’s t-test).

Figure 5: Alteration in pain correlates with alteration in F-values for the sensorimotor cortex. (a) The averaged differences in VAS scores (post–pre) are shown with the 95% confidence interval for three experiments (n=10). The asterisks denote significant differences (*P<0.05, **P<0.01, uncorrected, two-tailed Student’s t-test). (b) The Pearson correlation coefficient between the ΔF-value and the ΔVAS at each vertex was colour-coded on the normalized brain (n=30). (c) The Pearson correlation coefficient between Δcurrent and ΔVAS at each vertex was colour-coded on the normalized brain (n=30). (d) The averaged differences in total scores of SF-MPQ2 (post–pre) are shown with the 95% confidence interval for three experiments (n=10). The asterisks denote significant differences (n=10, **P<0.01, uncorrected, Mann–Whitney U-test). (e) The averaged differences in subscores of the SF-MPQ2 (post–pre) are shown with the 95% confidence interval for three experiments (n=10). The asterisks denote significant differences (n=10, *P<0.05, **P<0.01, uncorrected, Mann–Whitney U-test). Full size image

The alteration in the F-value was evaluated with respect to pain. For all three experiments, the increase in the F-value after BMI training (ΔF) was compared with the increase in the VAS scores (ΔVAS). The correlation coefficients between ΔF and ΔVAS at each of the vertices were colour-coded on the normalized brain (n=30, Pearson correlation coefficient; Fig. 5b). Interestingly, ΔVAS was positively correlated with ΔF on the sensorimotor cortex contralateral to the phantom hand. That is, pain increased as the discriminability in cortical currents representing phantom movements increased.

In contrast, the alteration in the mean cortical currents was not significantly correlated with the alteration in pain. The z-scored cortical currents were averaged among all trials of grasping and opening for each patient. The increase in the mean cortical currents after BMI training (Δcurrent) was compared with the increase in VAS scores (ΔVAS). No significant correlation was found between ΔVAS and Δcurrent for the sensorimotor cortex (n=30, Pearson’s correlation coefficient; Fig. 5c, also see Supplementary Fig. 6).

Moreover, the total scores of the SF-MPQ2 varied significantly depending on the decoders (n=10, P=0.0077, F(2, 27)=5.86, one-way ANOVA; Fig. 5d). These scores decreased significantly after training with the real hand decoder compared with the random decoder and phantom decoder (n=10, P=0.001 and 0.0034, uncorrected, respectively, Mann–Whitney U-test). However, the alterations of scores were not significantly different between the phantom decoder and random decoder (n=10, P=0.59, uncorrected, Mann–Whitney U-test). Among the four types of subscores in the SF-MPQ2, only the continuous pain scores varied significantly among the decoders (n=10, continuous, P=0.0033, F(2, 27)=7.12; intermittent, P=0.15, F(2, 27)=2.07; neuropathic, P=0.090, F(2, 27)=2.63; affective, P=0.064, F(2, 27)=3.04, one-way ANOVA; Fig. 5e). This score decreased significantly after training with the real hand decoder compared with the random decoder and phantom decoder (n=10, P=0.001 and 0.045, uncorrected, respectively, Mann–Whitney U-test). The alterations of the continuous scores were not significantly different between the phantom decoder and random decoder (n=10, P=0.12, uncorrected, Mann–Whitney U-test).

BMI training altered decoding accuracy of phantom movements

In addition to the univariate analyses, we evaluated the alteration in the cortical currents using the decoding method (multivariate analysis). For all patients, we compared the accuracies for classifying the phantom movements using the estimated currents in the sensorimotor cortex contralateral and ipsilateral to the phantom hand. The differences in accuracies varied significantly among the three conditions using the currents contralateral to the phantom hand (n=10 each, P=0.00001, F(2, 27)=17.52, one-way ANOVA; Fig. 6a). The accuracy decreased significantly after training with the real hand decoder compared with the phantom decoder and random decoder (n=10, P=0.00004 and 0.006, Bonferroni-corrected, t(18)=7.44 and 3.59, respectively, two-tailed Student’s t-test). Moreover, the classification accuracy increased significantly after training with the phantom decoder compared with the random decoder (n=10, P=0.006, Bonferroni-corrected, t(18)=3.60, two-tailed Student’s t-test). In contrast, the accuracies using the currents for the sensorimotor cortex ipsilateral to the phantom hand did not significantly change among the three types of decoders (n=10 each, P=0.25, F(2, 27)=1.45, one-way ANOVA; Fig. 6b). In addition, the accuracy in classifying the intact hand movements was not significantly changed after the BMI training with the real hand decoder for the cortical currents of the contralateral or ipsilateral sensorimotor cortex (n=10, P>0.05 for each, Bonferroni-corrected, two-tailed Student’s t-test; Supplementary Fig. 4).

Figure 6: Alterations in classification accuracies among the three experiments were correlated with alterations in pain. (a,b) The accuracies for classifying two types of phantom movements were evaluated using the currents on the motor cortex contralateral (a) and ipsilateral (b) to the phantom hand. Each bar shows the average difference in the accuracy with 95% confidence intervals for each experiment. The asterisks denote significant differences (n=10, **P<0.01, Bonferroni-corrected, two-tailed Student’s t-test). (c,d) The increase in VAS scores was significantly correlated with the increase in the accuracy for classifying the phantom movement types using the z-scored cortical currents on the sensorimotor cortex contralateral to the phantom hand (n=30; c), although the increase in VAS scores was not correlated with that for the cortex ipsilateral to the phantom hand (d). Full size image

Pain changes with accuracy of classifying phantom movements

The changes in the VAS scores were significantly correlated with the changes in classification accuracy using the currents for the sensorimotor cortex contralateral to the phantom hand (n=30, R=0.66, P=0.0001, Pearson’s correlation coefficient; Fig. 6c), but not with the changes in accuracy using the currents ipsilateral to the phantom hand (n=30, R=0.037, P=0.76, Pearson’s correlation coefficient; Fig. 6d). In addition, the changes in the total SF-MPQ2 scores were significantly correlated with the changes in accuracy using the contralateral currents (n=30, R=0.51, P=0.0044, Spearman’s rank correlation coefficient), but not with accuracy using ipsilateral currents (n=30, R=−0.14, P=0.47, Spearman’s rank correlation coefficient).