Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans.
Free full text in Europe PMC
Abstract
Free full text
Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans
Abstract
Restoration of touch after hand amputation is a desirable feature of ideal prostheses. Here, we show that texture discrimination can be artificially provided in human subjects by implementing a neuromorphic real-time mechano-neuro-transduction (MNT), which emulates to some extent the firing dynamics of SA1 cutaneous afferents. The MNT process was used to modulate the temporal pattern of electrical spikes delivered to the human median nerve via percutaneous microstimulation in four intact subjects and via implanted intrafascicular stimulation in one transradial amputee. Both approaches allowed the subjects to reliably discriminate spatial coarseness of surfaces as confirmed also by a hybrid neural model of the median nerve. Moreover, MNT-evoked EEG activity showed physiologically plausible responses that were superimposable in time and topography to the ones elicited by a natural mechanical tactile stimulation. These findings can open up novel opportunities for sensory restoration in the next generation of neuro-prosthetic hands.
eLife digest
Our hands provide us with a wide variety of information about our surroundings, enabling us to detect pain, temperature and pressure. Our sense of touch also allows us to interact with objects by feeling their texture and solidity. However, completely reproducing a sense of touch in artificial or prosthetic hands has proven challenging. While commercial prostheses can mimic the range of movements of natural limbs, even the latest experimental prostheses have only a limited ability to ‘feel’ the objects being manipulated. Oddo, Raspopovic et al. have now brought this ability a step closer by exploiting an artificial fingertip and appropriate neural interfaces through which different textures can be identified.
The initial experiments were performed in four healthy volunteers with intact limbs. Oddo, Raspopovic et al. connected the artificial fingertip to the volunteers via an electrode inserted into a nerve in the arm. When moved over a rough surface, sensors in the fingertip produced patterns of electrical pulses that stimulated the nerve, causing the volunteers to feel like they were touching the surface. The volunteers were even able to tell the difference between the different surface textures the artificial fingertip moved across.
The temporary electrodes used in this group of volunteers are unsuitable for use with prosthetic limbs because they can easily be knocked out of position. Therefore, in a further experiment involving a volunteer who had undergone an arm amputation a number of years previously, Oddo, Raspopovic et al. tested an implanted electrode array that could, in principle, remain in place long-term. This volunteer could also identify the different textures the artificial fingertip touched, with a slightly higher degree of accuracy than the previous group of intact volunteers. Further studies are now required to explore the potential of this approach in larger groups of volunteers.
Introduction
One of the most remarkable characteristics of a human hand is its ability to gather a rich variety of sensory information about the external world. In particular, tactile information conveyed by the four classes of low-threshold mechanoreceptor afferent units in the fingertips (Johnson et al., 2000; Johnson, 2001; Vallbo and Johansson, 1984a; Abraira and Ginty, 2013) is fundamental for manipulation activities (Edin et al., 1992; Edin and Johansson, 1995). In addition, the biological sensors innervating the glabrous skin can provide complex types of information such as the onset of the contact with an object, the level of performed grasping force, and discrimination of textural features (Weber et al., 2013; Darian-Smith et al., 1980a).
The possibility of providing natural and rich sensory information to hand prosthesis users represents a major achievement. In fact, even though a significant research in biology, medicine and engineering has produced artificial hands that progressively approached the performance of a “natural hand”, it is a shared belief that providing a sense of touch is still the missing milestone (Kwok, 2013). This achievement will allow the “symbiosis” between the user and interface to become more adaptive (to changing tasks and situations), more robust (beyond interfering stimuli), more effective (learn from the past to anticipate or predict the future), and more natural (rapidly becoming part of the body scheme). Therefore, the restoration of sensory perception is the crucial step to achieve in the development of next generation of artificial limbs and hand prostheses, in particular.
Thus far, promising results have been recently achieved to restore information about the touch of objects (Tan et al., 2014) and also the level of produced grasping force (Raspopovic, 2014). The restoration of ability to judge textural features represents the next significant step towards the re-establishment of close-to-natural sensory skills of a natural hand.
Here, we sought to achieve this goal via an integrated approach to mimic natural coding using a neuromorphic, real-time, mechano-neuro-transduction (Spigler et al., 2012) (MNT) process through a sensorized artificial finger (Oddo et al., 2011a) that integrates a Micro Electro-Mechanical System (MEMS) tactile sensor (Beccai et al., 2005). In this framework, the temporal coding of tactile information was based on the use of a biologically plausible neural model (Izhikevich, 2003) , which has shown promise as a versatile and computationally efficient framework for reproducing a wide range of phenomenological neural responses to stimuli.
In this study, the MNT process was first tested in intact subjects by delivering electrical stimulation to their sensory peripheral nerve fibers during microstimulation via tungsten needle microelectrodes (Vallbo et al., 1984b; Torebjörk et al., 1987). Furthermore, the effects of natural mechanical stimulation at the fingertip and of the MNT-based electrical stimulation were qualitatively compared in terms of electrophysiological signals elicited in the contralateral sensory cortex of the subjects.
Moreover, we wanted to evaluate whether the results achieved during needle microstimulation could be translated into experiments, which would be related to the real-time and longer-term use of hand prostheses. In fact, percutaneous needle microstimulation of peripheral nerves cannot be used in amputees as part of an effective long-term assistive device because a stable needle-to-fibers spatial relationship cannot be maintained in a moving limb. Therefore, its use can only be limited to a single experimental session lasting a few hours at most (Vallbo et al., 2004). Instead, implantable neural interfaces, such as Transverse Intrafascicular Multichannel Electrodes (TIMEs) (Boretius et al., 2010) , are surgically positioned and firmly stabilized to adhere to nerve fascicles and do not require an arm rest compared with needle interfaces. Therefore, implantable neural interfaces are suitable for long-term implants. In this study, we developed a novel computational model to investigate the similarities between neural effects, such as in afferent nerve recruitment curves, achieved using microstimulation with percutaneous needles and with TIME neural interfaces. This comparison allowed verification of the possibility of extending the results achieved using percutaneous needle stimulation to TIMEs.
The MNT process was tested in acute conditions in four intact subjects and was validated in a TIMEs implanted subject with a transradial amputation. For the first time in human hand neuroprosthetics, these integrated approaches allowed to show that discrimination of textural features can be reliably provided to users in different experimental conditions using peripheral intraneural electrical stimulation. The range of tested tactile stimuli in the current work is on the order of millimeters (0.5–3.0 mm). Thus, these stimuli pertain to the lower boundary (towards the fine region) of stimuli that are typically classified as coarse (Weber et al., 2013).
Results
Experiments with intact subjects using needle microstimulation of the median nerve
The MNT process translates surface coarseness into the injection of current pulses into the nerve. It qualitatively mimics the neuronal activity recorded during human microneurographic experiments (Oddo et al., 2011b). The MNT approach was initially tested in four intact volunteers using percutaneous electrical microstimulation of the median nerve (Vallbo et al., 1984b; Torebjörk and Ochoa, 1980) (Figure 1a, Figure 2). The participants - without visual or acoustic cues about the stimuli - were asked to discriminate surface pairs (Figure 1b) that differed in the Spatial Period (SP) of alternating ridges and grooves (gratings), i.e., in the distance between consecutive ridges separated by grooves (defined in Figure 2a), which was a constant quantity in each half grating (as shown in Figure 1b).
(a) Sensorized artificial finger and tactile stimulation platform. (b) Tactile stimuli that were used in the three-alternative forced-choice (3AFC) psychophysical protocol and the raster plot of spike trains that were generated in all sessions with one subject by the artificial finger while the gratings were slid. (c) Setup of percutaneous electrical microstimulation (left) and implanted intrafascicular stimulation (right) of the median nerve, and discrimination performance during all experimental sessions involving four intact subjects and one transradial upper limb amputee.Source data of the spike trains that were transduced by the artificial finger while the gratings were indented and slid over have been deposited in Dryad (Oddo et al., 2016). Such spikes were used to trigger the neural stimulator in all the experimental sessions with DAS amputee (raster plot depicted in Figure 1b).
(a) MEMS sensor with 4 transducing piezoresistors implanted at the base of a cross-shaped structure (sensor piezoresistive outputs Sx+ and Sx- are represented in blue and in green, respectively), grating with ridges and grooves that alternate with spatial period SP, and sensorized fingertip, which is in contact with tactile stimulus. (b) The sequence of presentation of surface pair to the artificial finger. (c) Example of implementation of the Izhikevich model for real-time conversion of MEMS sensor data into a sequence of artificial neural spikes. The blue and green traces show raw sensor outputs from the pair of opposing channels depicted in panel a. The red trace shows the input to the Izhikevich artificial neuron, which results from the application of Equation 1 and 2 (described in the Materials and methods section). The black lines depict the spikes that are generated when the membrane potential of the Izhikevich artificial neuron reaches the threshold (Equation 5 in the Materials and methods section) and, thus, the neural stimulator is triggered. (d–e) Implementation of the Izhikevich model with a close-up view during the sliding motion over the first and second halves of the grating. The traces and color-coding are shown in panel c.
Via percutaneous electrical neural microstimulation, they reported mechanical sensation pertaining to the palmar side of the first four fingers of the hand. Microstimulation allowed users to reach discrimination ability above 77% (107/138, Figure 1c, Figure 3a) during a three-alternative forced-choice (3AFC) psychophysical procedure (Perez et al., 2010; Gibson and Craig, 2005) mediated by the artificial touch system, which is based on the use of a MEMS sensor embedded into a human-sized robotic fingertip (Video 1). Confidence analyses indicated that percutaneous electrical microstimulation successfully induced percepts that were used to assess the coarseness of textured surfaces (Figure 3b). The capability to discriminate between the two sides of the surface pairs was correlated with the difference between their spatial periods (Figure 3c).
Video 1.
The video includes an interview with DAS amputee subject reporting the percepts immediately after one experimental session.
Each column reports results of the analyses on individual subject basis. (a) Each panel displays the confusion matrix of behavioral responses relative to the four intact subjects with microstimulation. The titles indicate correct/total responses (percentage). (b) Vertical bars display correct responses that are associated with each stimulus. The vertical solid lines over each bar indicate the 95% confidence intervals (Clopper Pearson exact interval) per stimulus. The dashed horizontal line indicates chance level (1/3). (c) The fraction of trials a pair of stimuli is perceived as different as a function of difference in spatial period (ΔSP) between the two stimulus halves, and logistic fit (dashed line). The title reports the R2 associated with the fit, i.e., the fraction of data variance explained by the logistic function, and the significance of the Pearson correlation between data points and the fit.
As described hereafter, the comparison between the EEG activity that was evoked by the natural mechanical tactile stimulation of the real fingertip in the right hand and the one evoked by the substitutive electrical stimulation showed no significant differences in source topography, response timing, and clustering of cortical connections between the two stimulation modalities. Event-related potentials (Figure 4a) after substitutive electrical (n = 4, estimated power 0.75, Figure 4—figure supplement 2) and natural mechanical stimulation (n = 4, estimated power 0.79, Figure 4—figure supplement 3) conditions did not reveal any statistical difference (Montecarlo statistics with cluster correction for multiple comparisons). Furthermore, a network graph analysis approach (Vecchio et al., 2015a) revealed a lateralized EEG frequency modulation that was evoked both by electrical and mechanical stimuli (Figure 4b). Indeed, the primary sensorimotor areas in the hemisphere contralateral to the stimulus presented a significant reduction (3-way ANOVA followed by Duncan’s multiple range test, F(1,6) = 6.48, p<0.05, comparison to the ipsilateral hemisphere) in the clustering coefficient following the incoming sensory stimulus, regardless of its tactile or substitutive nature (Figure 4c). Moreover, the generator sources of short-latency components of Somatosensory Evoked Potentials (SEPs) that were elicited by the substitutive electrical stimulation were localized at the Postcentral Gyrus (Brodmann Areas 2 and 3), which was consistent with a physiological tactile activation of the primary somatosensory cortex (Figure 5), as previously described using a hand area functional source separation method (Di Pino et al., 2012).
(a) Grand average event related potentials (ERPs) of all subjects (n = 4) for both substitutive neuromorphic electrical (red) and natural mechanical tactile (blue) stimulation, ranging from -1500 to 3000 ms with respect to the stimulus onset. Each channel was normalized for the standard deviation of the prestimulus. (b) eLORETA connectivity maps for delta, theta, alpha, low beta and high beta bands. Each tract (red for electrical and blue for mechanical stimulations) among the 7 sensorimotor regions of interest (Brodmann Areas BA 1–7) reports the connectivity value higher than the cut-off threshold (functional coupling >0.3). (c) Clustering modulation (percentage of variation during stimulation with respect to baseline) in the left and right hemispheres with electrical and mechanical stimulations. A significant reduction in clustering modulation across all frequencies occurred in the hemisphere contralateral to the stimulation (p<0.05, comparison to the right hemisphere, Duncan test after ANOVA).
Figure 4—figure supplement 1.
The horizontal black bar indicates the time window (210–240 ms) when the evoked potential reached significance compared with the prestimulus voltage (2 Standard Deviations from the mean prestimulus voltage), both after the electrical microstimulation and after the mechanical stimulation.
Figure 4—figure supplement 2.
Figure 4—figure supplement 3.
Figure 4—figure supplement 4.
Figure 4—figure supplement 5.
(a) Butterfly plot of SEPs for all 64 channels of one subject (M4). All traces are aligned to the electrical stimulus delivery. On top, the topographic representation of amplitude distribution at different time lags (ms). (b) Position of the associated equivalent dipole, which corresponds to P27 peak, superimposed on the individual horizontal, coronal and sagittal structural MRI planes of the subject.
Translatability from needle microstimulation to TIME-based stimulation
The possibility to translate results from needle microstimulation to TIMEs was investigated by developing a novel hybrid model (Raspopovic et al., 2011) of the median nerve (Figure 6) with both a microneedle and TIME interfaces inserted inside the nerve trunk (Figure 7, Figure 8). The model, which takes into account realistic anatomical (Jabaley et al., 1980) and neurophysiological (Vallbo et al., 1984) data, indicated that the stimulated portion of axonal population (Figure 7), and, therefore, the type of sensation, together with the stimulation threshold (statistically non-different: p>0.05, Kruskal-Wallis test) were similar for the two interfaces. The range of electric charge necessary for recruitment in both cases was comparable (Figure 7, Figure 8). These findings provided evidence that implanted intraneural electrodes could achieve results that were comparable to needle percutaneous microstimulation and justified the transition from the acute to implantable interface to pursue a long-term use.
(a) Implementation for the TIME electrode. (b) Implementation for the microneedle. (c) Different locations for stimulating active site and tip, that were used in the model to compare the TIME interface versus the microstimulation needle. Red dot represents the evaluated intrafascicular positioning of the TIME active site and the end of microstimulation needle tip. Yellow dot represents close-to-fascicle location, where the end of microstimulation needle tip and the active site of TIME interface were placed. Blue dot represents an example position with a shielding fascicle. The X marker represents the targeted fascicle, where the fiber activation was calculated for different locations of microneedle and TIME for 9 different populations that emulated biologically inherent uncertainty in the placement and extension of fibers innervating a specific hand district. This procedure was performed analogously for medium and small fascicles and confirmed the results.
Figure 6—figure supplement 1.
(a,b), Recruitment curves of sensory axon populations resulting from different positions of the active sites for the needle microstimulation and TIME stimulation (mean +/- S.E.M. of percentage of recruited fibers). R4/L4 is the pair of active sites within the target fascicle; R5/L5 is close to it, and R6/L6 is shielded with respect to it. For each position of the stimulating tip/active site, nine different axonal populations were computed. The fiber was considered active when the spike travelled until the last node of Ranvier that was implemented. Figure insets represent voltage distributions for different positions of active sites, as calculated using the FEM solver. These results are representative for several electrode insertion configurations. The same computations were performed for medium and small fascicles and confirmed the results.
(a) Recruitment results obtained for placement of the microneedle tip (insulated and non-insulated tip) and the TIME active site (L4 and R4) within the fascicle of interest. The recruitment curves are the mean +/- S.E.M. for the 9 implemented populations in 4 different FEM configurations. The results for the microstimulation needle and TIME interface are similar, which supports the translation from percutaneous towards intraneural stimulation. The 2 cases of microneedle tip exposure yielded comparable outcomes. (b) Voltage distribution induced by the close active site and positioning of 9 different fiber populations that were implemented to emulate the inherent anatomical uncertainty of sensory axon locations (see Figure 8—figure supplement 1 for the correspondence between each color of the fibers illustrated in the panel and the specific implemented population). (c) The colored dots indicate another randomized positioning of 9 different fiber populations implemented in the simulations of the hybrid electrical-biophysical model of the median nerve, with active site inside of fascicle.
Figure 8—figure supplement 1.
Experiments with a transradial amputee
The 3AFC psychophysical protocol used during microstimulation experiments was also evaluated during sessions with subject DAS, a transradial amputee that was implanted with TIMEs (Raspopovic, 2014). The sensations elicited by temporal modulation of spiking electrical stimulation of the median nerve through a single implanted electrode were reported by DAS as pertaining to the palmar side of the index finger of the missing hand and as providing a realistic representation of the mechanical alternation between ridges and gaps of the experimented gratings (Video 1). DAS showed to be able to comply with the entire 3AFC psychophysical protocol via neural stimulation, without any visual input or guidance by the experimenter, and without interruption in each session, and in 77/80 trials (Figure 1c, Figure 9a) of trials he was able to correctly identify whether the two sides of the surface had the same SP or which one had a larger SP (Figure 10). This performance was significant (p<0.05 confidence, Clopper Pearson exact interval) for each presented single surface (Figure 9b). These results indicate that the discrimination ability obtained using TIME stimulation was higher than the one with microstimulation (Figure 1c, Figure 3). This was attributed to the prolonged use/training of the implanted neural interface before the present study (Raspopovic, 2014).
(a) Confusion matrix of the responses given by DAS subject. (b) Vertical bars display the correct responses, which are associated with each stimulus. The vertical solid lines over each bar indicate the 95% confidence intervals (Clopper Pearson exact interval) per stimulus. The dashed horizontal line indicates chance level (1/3). (c) Fraction of trials for which a pair of stimuli is perceived as different as a function of the difference in spatial period (ΔSP) between the two stimulus halves, and logistic fit (dashed line). The title reports the R2 associated with the fit, i.e., the fraction of data variance explained by the logistic function, and the significance of the Pearson correlation between data points and the fit. (d,e) Comparison between IBI-based and AFR-based discrimination. Difference between IBIs (ΔIBI, panel d) and difference between AFRs (ΔAFR, panel e) measured in the spike patterns elicited by the two sides of each stimulus and plotted as a function of the difference between grating SPs (ΔSP). Error bars indicate the interquartile range. The title reports the fraction of explained variance.
In all panels the spatial structure (scale: 2 mm) of the grating is superimposed over a sample temporal pattern (scale: 200 ms) of the spike train that was obtained as a result of transduction with the artificial finger. The ratio between the spatial and temporal scales turns into the stimulus sliding velocity (10 mm/s). The instantaneous firing rate (scale: 50 spikes/s) is shown together with the average firing rate during the stimulus sliding motion. (a) (above) Stimulus ![[increment]](/corehtml/pmc/pmcents/x2206.gif)
0.0+ is characterized by thw presentation of two half surfaces in the SP1 - SP2 order (first: 1.5 mm, second: 1.5 mm); (below) stimulus 0.0-, is characterized by the presentation of two half surfaces in the SP2 – SP1 order (first: 1.5 mm, second: 1.5 mm). For stimulus 0.0, the two half surfaces have the same coarseness. Therefore, 0.0+ and 0.0- result in spiking patterns with a common temporal structure. (b), (above) Stimulus 1.0+ is characterized by the presentation of two half surfaces in the SP1 - SP2 order (first: 2.0 mm, second: 1.0 mm); (below) stimulus 1.0- is characterized by the presentation of two half surfaces in the SP2 – SP1 order (first: 1.0 mm, second: 2.0 mm). (c) (above) Stimulus 2.0+ is characterized by the presentation of two half surfaces in the SP1 - SP2 order (first: 3.0 mm, second: 1.0 mm); (below) stimulus 2.0- is characterized by the presentation of two half surfaces in the SP2 – SP1 order (first: 1.0 mm, second: 3.0 mm). (d) (above) Stimulus 2.5+ is characterized by the presentation of two half surfaces in the SP1 - SP2 order (first: 3.0 mm, second: 0.5 mm); (below) stimulus 2.5- is characterized by the presentation of two half surfaces in the SP2 – SP1 order (first: 0.5 mm, second: 3.0 mm).
Figure 10—figure supplement 1.
The depicted spatial modulations have the same monotonic trend versus the period of grating. The spatial modulation of our spike trains is globally higher in comparison with the neurophysiological data that was previously shown (Phillips and Johnson, 1981) for surfaces with the same spatial period (highlighted by a red box in the figure). We hypothesize that this difference exists because our surfaces had a constant ridge width of 0.25 mm, whereas, the surfaces in the study performed by Phillips and Johnson were square-wave gratings (i.e., ridge width was equal to groove width and was equal to half the spatial period).
Analysis of neural coding strategies
The behavioral performance as a function of stimulus in subject DAS can be fit exactly (R = 1) using a logistic function of the difference of SPs between the two halves of each surface (Figure 9c). To understand how the very good surface discrimination, which was shown both by intact subjects with microstimulation and by DAS with the TIME implant, was achieved (Figure 1c), we tested which neural features correlated with the difference of SPs. During the conditions of constant sliding velocity (10 mm/s) and regulated load force (400 mN), the periodic contact between the sensor and the texture ridges generated regular patterns in the sensor output. The spiking response was given by a sequence of bursts of reliable duration, time-locked to the alternation of ridges and grooves during the sliding motion of gratings (Figure 10). This suggested that the Inter Burst Interval (IBI, illustrated in Figure 10a), given by the temporal distance between the onset of consecutives bursts, might characterize the response to each texture. Indeed, the MNT process led to linear correlation between the stimulus SP of each texture and a highly specific IBI (ANOVA test, Tukey Kramer correction for multiple comparisons, p<0.00001) reproducing neurophysiological findings in non-human (Darian-Smith et al., 1980a; Phillips and Johnson, 1981) and human (Oddo et al., 2011b) primate studies with similar gratings. Consequently, the difference in IBI between the MNT output that was elicited by the two sides of each stimulus correlated almost perfectly (R2 = 0.997) with the difference in SP (Figure 9d). Additionally, the behavioral responses of DAS were perfectly (R = 1) fit by a logistic function of the difference in IBI.
The abovementioned results show that DAS behavioral responses may be based on the temporal structure of stimulation. However, they do not rule out the possibility that the responses were based simply on the count of the total number of spikes injected during each stimulus. Then, we tested whether a rate code not taking into account the temporal structure of the response was able to lead to comparable discrimination of stimuli as the one observed behaviorally. A commonly accepted definition of a rate code is the count of spikes within the time frame of encoding window (Theunissen and Miller, 1995; Panzeri et al., 2010) which in our case is assumed to be the 2 s presentation of each stimulus. Then, the sensor response to each stimulus, based on which we performed the IBI-based analysis, was also quantified by measuring the overall average firing rate (AFR), i.e., the ratio between the total number of spikes emitted during the sliding and the (fixed) 2
s sliding duration. We found that the AFRs generated in response to different SPs were often similar because a larger IBI was associated with a more intense bursting activity (Figure 10). In particular, the AFRs of responses to 0.5, 1.0 and 1.5 mm were not significantly different, and the AFRs of responses to 2.0 mm and 3.0 mm were not significantly different (p<0.05 ANOVA test with multiple comparisons). Consequently, the difference ΔAFR in AFR between the two sides of a stimulus correlated very weakly with the difference between their SP (R2 = 0.066 for ΔAFR vs R2 = 0.997 for ΔIBI, compare Figure 9e and Figure 9d). Note that no logistic function of the difference in AFR was able to fit DAS behavioral responses (R2 = 0.04, p = 0.8), which was a significant contrast with the perfect fit achieved with ΔIBI. Therefore, because the total spike count was not able to encode differences between stimuli to a degree compatible with behavioral responses, we concluded that the rate code was not responsible for the subject perception of the presented textures. This further strengthens the hypothesis that the subject exploited the temporal structure of the response to discriminate the stimuli.
Figure 10 shows the instantaneous firing rate (Lánský et al., 2004) of different responses computed as the inverse of the inter-spike interval. It is apparent that the temporal evolution of the instantaneous firing rate matches the alternation of ridges and grooves. Indeed, it is already known that, when sliding occurs at fixed velocity, the temporal structure of the sensor output is a linear transformation of the spatial structure of the texture (Oddo et al., 2011b; 2011c) , at least for regular textures such as those considered here. Therefore, this is a case of temporal coding of the stimulus (Theunissen and Miller, 1995; Borst and Theunissen, 1999) : the temporal evolution of the spiking response reflects the time evolution of the stimulus rather than internal dynamics of stimulus encoding. Because the spatial structure is highly regular and the textures primarily differ for a single spatial variable (the SP), it is possible to discriminate between them using the corresponding single temporal variable (the IBI). Then, this may be the feature of the injected spike trains that allows subjects to precisely decode the presented stimuli.
Discussion
Recent neuroprosthetic studies showed that pressure sensation can be elicited by injecting a train of pulses with a fixed shape and different frequencies via intrafascicular (Dhillon and Horch, 2005) or cuff (Tan et al., 2014; Ortiz-Catalan et al., 2014) interfaces. Additionally, reconstruction of tactile sensation of force levels and object shape was shown using multi-electrode stimulation via pulses with current amplitude linearly increasing with sensor outputs (Raspopovic, 2014). In contrast, this work focused on eliciting textural features by injecting a biomimetic (Saal and Bensmaia, 2015) temporal structure of pulse trains with a fixed current amplitude in each experimental session. Discrimination of textural features is a remarkable skill of our somatosensory system, which is used in everyday activities to interact with a peri-personal space. The subjects that use hand prostheses will significantly benefit from the restoration of this sensory function, which represents the next desirable feature after recent achievements (Tan et al., 2014; Raspopovic, 2014; Ortiz-Catalan et al., 2014). The results of this study show that the texture discrimination skills can be artificially provided to users using both needle microstimulation in intact subjects and implantable intraneural interfaces in a transradial amputee. Both in needle and TIME-based experiments, the subjects were able to use our ‘artificial’ feedback to perform a 3AFC psychophysical test with very good and comparable performance.
The similarity between the needle and TIME-based stimulations was predicted by our hybrid FEM-biophysical model, which supported the transition from acute preparation towards chronically implanted electrodes. In addition, this result opens up interesting scientific and clinical opportunities. In the future, needle microstimulation can be used during translational experiments to test different types of peripheral neural coding strategies, which - if successful in this prototypal situation - would be implemented using implantable neural interfaces in disabled patients. Furthermore, model predictions were confirmed by the quasi-perfect discrimination (96% overall stimuli, see Figure 1c, and see Figure 9a,b for the analysis on a per-stimulus basis) that was achieved by DAS amputee with TIME stimulation.
The results of the present study showed that a temporal neuronal coding of spatial structure (Weber et al., 2013) can successfully elicit tactile cues in case of coarse surfaces that were explored under a passive- dynamic-touch protocol with a constant sliding velocity and a regulated load force, thus, involving controlled motion of the tactile stimulus while the fingertip was not moving (Yoshioka et al., 2001). In such passive-touch framework there is a lack of voluntary movement. This allows decoupling of cutaneous information (which was our interest in this study) from the kinaesthetic afferent sensory feedback and efference copy associated with movement dynamics. Thus, it was possible to compare the EEG activities evoked by the substitutive neuromorphic electrical stimulation versus the natural mechanical tactile stimulation. No significant differences in topography or frequency modulation clustering were shown by the EEG signals in the two cases.
The behavioral results showed that injecting the sensor output via a single stimulation channel was sufficient to induce accurate responses (particularly from DAS) in the designed experimental protocol with locked indentation force and tangential velocity of stimulus pairs. Additionally, these experiments allowed us to investigate how the subjects were able to achieve these results in this specific case. In fact, the relationship between IBI and behavioral performance seems to indicate that responses of subject DAS were based on temporal structures of the injected stimuli (which, due to the linearity of our MNT process in mapping the geometry of stimuli, correspond to temporal modulation in the spiking activity captured by the IBI for regular textures), rather than on their average firing rate alone. IBI was previously shown to be given by the ratio between the surface spatial period SP and the sliding velocity V in human (Oddo et al., 2011b) and animal (Weber et al., 2013; Darian-Smith et al., 1980a) neurophysiological investigations and in artificial touch (Oddo et al., 2011c) studies. Because in our protocol the sliding velocity V was fixed (10 mm/s), the IBI was proportional to the SP, and so were their differences (IBI and SP) between the two sides of the stimuli (Figure 9d).
Our results indicate that a temporal single-channel MNT-based intraneural stimulation allows gathering of textural features of medium-coarse surfaces with constant and slow sliding velocity. However, for more complex texture discriminating tasks it is plausible that integration of spatiotemporal information from neighboring receptors distributed along the fingertip (Johansson and Flanagan, 2009; Jörntell et al., 2014) and, thus, multi-channel stimulation, combined with amplitude modulation (Raspopovic, 2014), would be needed in less restrictive conditions that involve velocity or force variations and everyday life stimuli (Weber et al., 2013; Dépeault et al., 2008; Rongala et al., 2015).
The promising results obtained with microstimulation in four intact subjects, combined with robust translational indications from the hybrid model and an excellent outcome from one amputee, prompt the idea that neuromorphic stimulation could be a natural and effective tool for eliciting texture discrimination abilities via hand prostheses. Neuroprosthetic research (Tan et al., 2014; Raspopovic, 2014; Dhillon and Horch, 2005; Rossini et al., 2010) can in parallel contribute to the evaluation of open neuroscientific hypotheses about tactile perception (Weber et al., 2013; Yoshioka et al., 2001; Johansson and Flanagan, 2009; Hollins and Risner, 2000) for the advancement of understanding of human somatosensory physiology.
Materials and methods
Sensorized finger
The core element of the artificial fingertip was a Micro Electro Mechanical System (MEMS) sensor with 4 transducing piezoresistors implanted at the base of a cross-shaped structure (Beccai et al., 2005) (Figure 3). The MEMS was packaged with polymeric compliant material (Dragon Skin, Smooth-On, USA). Sensor data were sampled at 380Hz per channel by a 24-bit Analog to Digital Converter (ADS1258, Texas Instruments, USA) integrated in the fingertip, and acquired via SPI by a Field Programmable Gate Array (Cyclone II FPGA, Altera, USA).
Mechano-neuro-transduction (MNT) process
The FPGA streamed the acquired information via Ethernet to a PC for real-time implementation of an artificial mechanoreceptor model emulating the tactile coding recorded during human microneurography sessions carried out with surfaces and experimental apparatus as those of current study (Oddo et al., 2011b; 2011d). To this aim, MEMS sensor signals were converted into sequences of neural spikes via a real-time C++ implementation of the Izhikevich spiking neuron (Izhikevich, 2003). Such model was originally conceived to emulate neuron-to-neuron signaling, whereas in this modified implementation the input is haptic rather than synaptic. As detailed in the following sections, these transduced spike trains were injected in the median nerve as train of pulses of fixed width and amplitude through microstimulation electrode in the case of intact subjects and through TIME interface in the case of DAS amputee.
We did not model the fine details of the complex spatio-temporal mechanical interaction between physical stimuli, soft tissues and receptors (Hayward et al., 2014), but we injected as input current in the neuron model a signal generated from the outputs of sensors integrated in the artificial fingertip, as follows. Sensor piezoresistive outputs (Sx+ and Sx-, represented in blue and in green in Figure 2c) belonging to opposite tethers of the cross-shaped structure were subtracted (Equation 1) to obtain a component, proximal to distal on the biomimetic fingertip, highly correlated to the frictional shear stress (Oddo et al., 2007) arising along the direction of the stimulus sliding motion. This component was half-rectified and amplified (Equation 2, as represented in red in Figure 2c) in order to inject it as input current in the spiking neuron model (Ix in Equation 3). Equations 3 and 4 describe the subthreshold evolution of the membrane potential v and the recovery variable u in the implemented artificial neuron model (Spigler et al., 2012; Izhikevich, 2003). Whenever the membrane potential v reached the threshold level, a spike was triggered, v was set to a reset value c and u was increased of a fixed value d (Equation 5). The spike was broadcasted by TCP communication to successive services and graphical user interface (Labview, NI, USA).
The computed binary response constituted the output of the neuromorphic artificial touch system, which triggered via USB the current stimulator (STG4008, Multichannel System, Germany) each time that a spike template had to be delivered.
The coefficients of the model (Spigler et al., 2012; Izhikevich, 2003) were tuned so to achieve a phasic firing with respect to the moving ridges of surfaces (Gardner and Palmer, 1989). The MNT process emulated to some extent the neural representation of stimulus spatial patterning observed in previous non-human (Weber et al., 2013; Darian-Smith et al., 1980a) and human (Oddo et al., 2011b) primate recordings of SA1 afferents associated to Merkel mechanoreceptors, a functionally relevant class of tactile units able to encode medium-coarse textures (Weber et al., 2013; Phillips and Johnson, 1981; Yoshioka et al., 2001; Darian-Smith et al., 1980b) such as the surfaces evaluated in the present study. The MNT process coded the temporal period of the surface, a feature scaling linearly with spatial period of the grating and inversely with the stimulus tangential velocity. The resulting firing (Figure 2) was characterized by biomimetic (Sathian et al., 1989) bursts of multiple spikes per ridge with coarser surfaces (see 3.0 mm and 2.0 mm SP in Figure 10), by triplets or duplets for surfaces with medium coarseness (see 1.5 mm and 1.0 mm SP in Figure 10), down to a single spike per ridge with finer surfaces (see 0.5 mm SP in Figure 10), thus implementing a neural code of stimulus geometry based on the modulation of discharge rather than on the mean discharge frequency, resembling SA1 electrophysiological recordings under similar tactile stimulation conditions (Darian-Smith et al., 1980a; Phillips and Johnson, 1981) (Figure 10—figure supplement 1). Furthermore, the instantaneous firing rate of the implemented artificial mechanoreceptor model achieves a modulation up to tens of spikes/s (see red traces in Figure 10), coherently with typical values reported in the literature for SA1 units (Johnson, 2001; Phillips and Johnson, 1981).
The following parameters were used: K = 15,000; A = 0.04/sV; B = 5/s; C = 140 V/s; Cm =1F; R = 1C; a = 0.02; b = 0.2/s; c = -65 mV; d = 8 mV; vth = 30 mV.
Percutaneous electrical microstimulation of the median nerve with intact subjects
Four intact subjects (2 males and 2 females, 25–26 years old) underwent the experiments of texture discrimination with tactile feedback elicited by a stimulation injected through two microneurographic electrodes (FHC UNP40GAS, shaped as needles). The enrolment was subjected to signing a written Informed Consent, approved by the Campus Bio-Medico University Ethics Committee, where this set of experiments took place.
A neurologist trained in microneurography performed the procedure (Hagbarth and Vallbo, 1967) , consisting of two phases of electrical stimulation, superficial and percutaneous, necessary to identify the correct site for electrodes insertion. The reference electrode was positioned just through the skin and the active one into the nerve.
Initially, we stimulated the skin of the right arm of the subject in an area centred 2–3 cm proximal and medial to the elbow. A 1Hz cathodic biphasic balanced square current was delivered to the subject. Pulse duration was 0.2 ms, while amplitude was changed in the range 1–10 mA. The nerve area was identified whenever a hand muscle twitch occurred in correspondence of 1–2 mA amplitude. Reference and active electrodes were then inserted through subject’s skin, after cleaning.
A cathodic biphasic balanced square wave was injected through the needles with a frequency of 1Hz. Pulse duration was 0.2 ms, while the current changed within the 1–100 μA range. The neurologist moved slightly the active electrode towards the inner of the arm seeking for a reported sensation or a muscular twitch over/of the hand. If one of those conditions occurred with amplitude of the injected current between 1–10 μA, the electrode impaled the nerve fiber. The neurologist then moved the needle in the area of insertion until the subject reported a distinct tactile sensation over one of the first four fingers.
As final evaluation of proper placement of the electrodes, the electroneurogram (ENG) was visualized and acoustically identified: afferent ENG from the median nerve had to be discriminated over the background nerve activity as some mechanical stimuli were exerted over the fingers. Acquisition was performed by means of a system comprising a single channel amplifier (15LT Bipolar, Grass Instrument, USA) and a 16-bit data acquisition board (PCI-6251, NI, USA) installed on a personal computer running a custom interface (Labview, NI, USA).
The subject was instructed to avoid any voluntary movement that could alter the needle placement once the nerve search procedure was completed. An experimenter-guided session of microstimulation was performed in order to identify appropriate parameters to use in the upcoming 3AFC psychophysical protocol: 500 ms trains of cathodic biphasic balanced squares were delivered while changing frequency, pulse duration (Ts) and amplitude (As) starting from 10Hz, 5 μs and 2 μA, respectively. The tuning of stimulation parameters was operated heuristically among the subjects while performing pilot trials, such as: a) subject connected to the stimulator while the experimented touched manually the artificial fingertip, b) subject exploring the gratings with the non-stimulated hand and then c) perceiving the same surfaces via the artificial fingertip. An additional experimental session of recording somatosensory evoked potentials (SEP) (Kunesch et al., 1995) at 1Hz stimulation rate was performed with subjects M3 and M4 before starting the 3AFC psychophysical experimental protocol: the spike template was injected in the nerve through microstimulation. Stimulation was delivered as square pulses lasting 200 μs at 1Hz with a number of repetitions (180 for M3 and 700 for M4) allowing a robust Independent Component Analysis (Artoni et al., 2014) on EEG data (as detailed in the specific session). In order to allow a better comparison among subjects, stimulation amplitude was individually tailored and chosen as an amplitude that was clearly perceived by the subject, without producing any discomfort (between 5 and 10 μA). Pilot trials of the 3AFC psychophysical test with neuromorphic sensory feedback were then repeated for variable duration among subjects just before starting the actual experimental protocol.
Intraneural stimulation of the median nerve with implanted interface in transradial amputee
Participant DAS had suffered a traumatic transradial left arm amputation 10 years before the experiments. He was selected from a group of 31 persons with hand amputation because of the stump characteristics and his psychophysical abilities. All procedures were approved by the Institutional Ethics Committees of Policlinic A. Gemelli at Catholic University, where the surgery was performed, IRCCS S. Raffaele Pisana (Rome), where the experiments were performed, the Ethics Committee of Campus Bio-Medico University and the Italian Ministry of Health.
TIME neural interfaces were implanted under general anaesthesia. After superficial disinfection of the medial aspect of the left upper limb, placed extrarotated, the skin was cut along the medial edge of the biceps muscle for about 15 centimetres, from few centimetres below the axilla to about 6 cm above the elbow. The ulnar and the medial nerves were exposed along their course, through careful smooth dissection of the dermal and hypodermic structures, fascial bands, and muscles. Following an epineurial microdissection, performed under a surgical microscope (Zeiss, Pentero) to visualize the fascicles, two TIME electrodes were inserted into the nerve trunk of each nerve (median and ulnar, only the former of which was used for electrical stimulation in the present work). Electrodes were pulled inside the nerve, until the embedded active contacts reached the targeted location, close in contact with the nerve fascicles. Cable segments were placed in subcutaneous pockets, while four holes were made in close proximity of the skin incision, two medially and two laterally for the electrode emission (Raspopovic, 2014; Di Pino et al., 2014).
The electrodes were removed after 30 days, in accordance with EU guidelines. At the time of removal however, the TIME electrodes were still performing extremely well, and did not cause any discomfort to the subject. The follow-up of the clinical condition of the participant almost 2 years after the end of the protocol did not reveal any subjective or objective side effects.
Upon connection of the neural interface implanted in the median nerve with the current stimulator, an experimenter-guided session of intraneural stimulation was performed in order to identify appropriate parameters to use in the upcoming 3AFC psychophysical protocol, also capitalizing on the information gathered during the previous days of experimental activities with DAS. Pulse duration (Ts) and amplitude (As) were initially set to 100 μs and 100 μA, respectively. The tuning of stimulation parameters was operated heuristically in 33 min while performing pilot trials, such as: a) DAS connected to the stimulator while he (with the intact hand, to have a sort of self-touch experience) or the experimenter touched manually the artificial fingertip, b) DAS exploring the gratings with the intact hand and then c) perceiving the same surfaces via the artificial fingertip. Pilot trials of the 3AFC psychophysical test with neuromorphic sensory feedback were then repeated for 8 min with the selected parameters (As = 160 μA, Ts = 100 μs) just before starting the experimental protocol.
Three-alternative forced-choice (3AFC) psychophysical protocol
The protocol structure was essentially the same during microstimulation of intact subjects, and stimulation through implanted TIMEs in the amputee.
The artificial fingertip underwent mechanical stimulation with surface pairs that were presented using a mechatronic platform (Oddo et al., 2011d) designed to implement standardized human and artificial passive-touch (Jones and Smith, 2014) experimental protocols. The tactile stimuli were gratings, fabricated with 3D printing of plastic material (Project HD 3000, 3D Systems), consisting of a sequence of alternating ridges and grooves with spatial period (SP) from a minimum of 0.5 mm to a maximum of 3.0 mm depending on the stimulus and on the half portion of the surface under test (Figure 1b, Figure 2, Figure 10). The sequence of presented stimuli was randomized within sessions of 16 trials composed of 4 presentations of the 4 surfaces 0.0, 1.0, 2.0 and 2.5. In each session, the 4 trials per each stimulus were composed of 2 trials with presentation of the two surface halves in the SP1-SP2 order (labeled +: 0.0+, 1.0+, 2.0+ and 2.5+), and 2 trials with reversed SP2-SP1 order (labeled 2: 0.0-, 1.0-, 2.0- and 2.5-). The first half-surface was indented at 400 mN on the artificial fingertip and, after 4s of indentation without tangential movement, it was slid at 10 mm/s for 2s under regulated load force. The sliding motion was followed by 2s of indentation at 400 mN without stimulus movement, and then the surface was detached. The same sequence was applied to the second half-surface, starting 3 s after the end of presentation of the first half (see Figure 2 and Video 1 for the representation of the whole stimulation sequence). At the end of the presentation of the surface pair, the subject stated whether the first half-surface was perceived as having coarser, finer or same spatial coarseness in comparison to the second half. During the experiment the subject received no feedback about the correctness of the responses.
For the intact subjects who participated in the microstimulation sessions, the gratings were also presented mechanically, via the tactile stimulation platform (Oddo et al., 2011d) , directly on the finger (fixed to the platform via a finger holder glued to the nail) that was identified as source of the sensory perception elicited with microstimulation. During the experiment the subjects received no feedback about the correctness of the responses.
The 95% confidence intervals for performance estimate from individual subjects responses were computed with exact Clopper Pearson method (binofit function in Matlab) and compared against chance level (1/3) to assess performance significance. Parameters for logistic fit of performance as a function of stimulus features were estimated with generalized linear regression (glmfit function in Matlab), then the optimal logistic fit function was generated (glmval function in Matlab) and its accuracy evaluated as the squared correlation coefficient between data and fit.
EEG signals recording
In order to investigate the neural correlates of natural and substitutive texture discrimination, EEG signals were recorded in all microstimulation sessions using a 64 channel EEG device (SD LTM Express, Micromed S.p.A., Italy) with a 2 kHz sampling rate. The montage was in accordance with the 5% 10/20 system (Oostenveld and Praamstra, 2001). Careful scalp preparation granted electrodes impedance below 5 kΩ in at least 90% of derivations, as measured at the experiment onset.
EEG signal processing
Data were analyzed by Matlab scripts based on the EEGLAB toolbox (Delorme and Makeig, 2004). EEG signals were processed via independent component analysis (ICA) filtering to remove non-neural sources and artifacts. In order to optimize the dipolarity of the independent components (IC) extracted (Artoni et al., 2014) and to maximize the amount of data fed to the algorithm (while maintaining a sufficient density of information) the EEG continuous data were epoched.
Data were high-pass filtered using a zero-phase 1Hz, 24th order, Chebyshev type II filter and low-pass filtered using a zero-phase, 45Hz, 71th order, Chebyshev type II filter to remove slow drifts and high-frequency noise respectively, then resampled at 256Hz. Channels with prolonged prominent artifacts (by visual inspection) or with probability more than five times the standard deviation from the mean across all channels were removed (in the end the remaining numbers of channels were 21, 40, 59, 61 respectively for the four subjects), then a common average reference was used for the remaining channels. Epochs containing high-amplitude artefactual potentials, high-frequency muscle noise and other irregular artifacts, as identified by visual inspection, were removed and remaining data were submitted to AMICA (Palmer et al., 2007), a generalization of the Infomax algorithm (Makeig et al., 1996) to multiple mixture approaches (Lee et al., 1999) under the hypothesis that the ICs are spatially static (general stationarity, e.g. recording environment). ICA decompositions were performed separately on each subject over all conditions. Stereotyped artifacts such as eye movements, eye blinks and muscle tension were removed by ICA. The ICA decomposition was then saved and reapplied to data, pre-processed using the same described procedure, but high-passed using a 0.5Hz, 94th order, Chebyshev type II filter but using the pre-computed weights. This procedure allowed to efficiently remove artifacts while retaining the low-frequency EEG information.
Regarding source localization, for subject M4 a realistic head model was obtained by means of the NFT toolbox (Acar and Makeig, 2010) using a T1-weighted magnetic resonance (MR) image of the subject and a 4-layer model accounting for scalp (σ = 0.33), skull (σ = 0.0132), brain tissue (σ = 0.33), and cerebrospinal fluid (σ = 1.79) respectively. Electrodes positions on the head were co-registered with an optoelectronic neuronavigation system SoftAxic (E.M.S. srl, Italy) and aligned with the Montreal Neurological Institute MNI, Canada. The Finite Element Method (FEM) was used for the numerical solution of the forward problem with Boundary Element Method (BEM) meshes as boundaries. Given the potentials distribution across the scalp, the position of a best-fitting single equivalent current dipole (inverse problem) was determined using the (Dipfit toolbox of EEGLAB) (Oostenveld and Oostendorp, 2002; Delorme et al., 2012).
EEG recording time-locked with 1Hz intraneural median nerve microstimulation was exploited to evidence the early somatosensory potentials, the components more strictly due to the stimulation. SEP data epochs were selected from 20 ms before to 100 ms after each stimulation onset. Noisy epochs were rejected by careful visual inspection. Similarly to the continuous data, the criteria for epoch removal were the presence of high amplitude artifacts (e.g., Jaw clenching). Source localization was then performed on the most reliable short-latency potential of cortical origin, namely the P27 peak (Figure 5).
Event-related potentials (ERPs) were time-locked to the onset of either the electrical microstimulation or of the sliding phase of the mechanical stimulation (Figure 4, Figure 4—figure supplement 1). ERPs were normalized for the standard deviation of the prestimulus (1000 ms). ERP’s statistical significance between conditions (electrical microstimulation vs. mechanical stimulation) was assessed using a Montecarlo statistics with cluster correction for multiple comparisons (triangulation and maxsum as clustered statistics) (Maris and Oostenveld, 2007) , adapted from the FieldTrip toolbox (Oostenveld et al., 2011). Statistical power of ERPs comparisons between stimulation conditions was computed for dependent groups (GPower, Duesseldorf, Germany) (Figure 4—figure supplements 2–5).
EEG functional connectivity analysis and EEG graph analysis
EEG functional connectivity analysis was performed using the eLORETA exact low-resolution electromagnetic tomography (Vecchio et al., 2015; Pascual-Marqui, 2011; Vecchio et al., 2014a; 2014b; 2015b) software.
To obtain a topographic view of the sensorimotor network, brain connectivity was computed with sLORETA/eLORETA software in 7 regions, positioning the center in Brodmann Areas (BAs: 1–7) separately on the left and right hemispheres.
For each subject and for each hemisphere, among the eLORETA current density time series of the 7 Regions of Interest (ROIs), intracortical Lagged Linear Coherence, extracted using a sphere with 19 mm of radius (Pascual-Marqui, 2011; 2007) , was computed between all possible pairs of the ROIs for each of the five independent EEG frequency bands: delta (2–4Hz), theta (4–8Hz), alpha (8–13Hz), low beta (13–20Hz) and high beta (20–30Hz).
A weighted network was built based on the connectivity between ROIs. The nodes of the network were ROIs, and the edges of the network were weighed by the lagged linear coherence values.
The vertices of the network were the estimated cortical sources in the BAs, and the edges were weighted by the Lagged Linear value within each pair of vertices.
The measure considered here was the clustering (C) that characterizes the tendency of the nearest neighbors of a node to be interconnected. The mean clustering coefficient was computed for all nodes of the graph and was then averaged to estimate the tendency of network elements to form local segregated clusters. Finally, to obtain individual normalized relative measures, the values of each mean clustering coefficient were divided by the mean values obtained by their average in all bands of each subject.
Statistical analysis of percent clustering modulation with respect to baseline period was performed with Statistica v.7 software (StatSoft Inc., USA). Greenhouse and Geisser correction was used for the protection against the violation of the sphericity assumption in the repeated measure ANOVA. Besides, statistical significance was determined by 3-way ANOVA followed by Duncan’s multiple range test. ANOVA was performed between three factors: stimulation mode (mechanical and electrical; independent variable), hemisphere (left and right), and band (delta, theta, alpha, low beta, high beta).
Hybrid electrical-biophysical model of the median nerve for the comparison between microstimulation needle and implanted TIME
Hybrid models (Raspopovic et al., 2011; McIntyre et al., 2002) account for the anisotropy of extracellular conductivity during the calculation of the electrical field induced by the injection of the electrical current into the tissue, present in real nerves, and for the nonlinear response of cells to the extracellular stimulation. Those two aspects were addressed by means of a finite element method (FEM) to solve the voltage distribution generated by injected currents, and by calculating the neuronal dynamics to estimate the axonal response to the electrical stimulation. The volume conductor model implemented via multiphysics FEM (COMSOL Multiphysics, Sweden) can solve Poisson’s equation provided by proper boundary conditions. To do so, an anatomically shaped geometrical model of the median nerve (Jabaley et al., 1980) was generated by image segmentation (ImageJ, USA). Coordinates of the image segmentation were exported (livelink COMSOL-Matlab) to edit a 3D nerve model (procedure illustrated in Figure 6—figure supplement 1). Perineurium had a thickness of 3% of the diameter of the fascicles (Grinberg et al., 2008) , and coordinates were interpolated. The nerve diameter had a maximum of 4.6 mm and minimum of 2.3 mm, as obtained from unpublished histological data provided by Dr. Xavier Navarro and from post-mortem human cadavers dissections at Universitat Autonoma de Barcelona, Spain. A cylinder, representing the nerve’s outer space, filled with saline, enclosed the nerve. Boundary optimal dimensions of cylinder were found to be 69 mm for the diameter and 140 mm for the height, using convergence calculations (Raspopovic et al., 2011). Electrical ground was therefore fixed in this cylindrical boundary of the structure.
On the biophysical side, MRG model (McIntyre et al., 2002) was used to model the nerve tactile sensory fibers. This model represents the nonlinear modified Hodgkin-Huxley Equations for the active compartment of the axons (the nodes of Ranvier) and a detailed realistic representation of the myelinated tracts. The model (available in NEURON model repository) is capable of reproducing several experimental aspects of cells dynamics (McIntyre et al., 2002). Then, a fiber with 21 segments (Raspopovic et al., 2011; McIntyre et al., 2002) of nodes of Ranvier was built and extracellular stimulation procedure used to excite the cell. For a fiber of diameter D, a model had internodal spacing L = 100D. A fiber was considered recruited when a generated action potential travelled along its whole length (i.e., reached the last node of Ranvier). The total recruitment was calculated as the portion of the fibers activated for the specific charge injected with respect to the total number of fibers implemented. Furthermore, nodal length was fixed at 1 μm and nodal diameter scaled from a previous study (McIntyre et al., 2002). A list of plausible assumptions had to be taken, during the model construction. The sensory axon populations were constructed (NEURON, USA) by using a probabilistic distribution of fibers diameters (Vallbo and Johansson, 1984) for different tactile units of the human hand, resulting in two Gaussian distributions which differentiate nociceptive fibers from fibers responsible for pressure/touch sensation, and the latter was used. One of the assumptions constraining the model was that the stimulation of different fiber types, such as nociceptive fibers, was not induced, at used current range. A total amount of 100 modeled fibers for each fascicle were placed randomly in the specific target fascicle, for several placements (Figure 8), as explained in continuation. As the fiber organization within different fascicles in the nerve is unknown, we assumed that fibers within one fascicle innervate the same hand area (Jabaley et al., 1980) (i.e., the fibers in one fascicle are innervating a single finger tip and not many of them). Moreover, since there is an inherent anatomical uncertainty in placement and extension of fibers innervating a specific hand district, for every position of stimulating electrode/needle we implemented 9 populations having different extensions and centroids for the same large fascicle (i.e., spanning from having the whole fascicle uniformly populated, to the case of very concentrated population where the fibers are almost touching each-other, Figure 8 and Figure 8—figure supplement 1). Then the analysis has been performed for the range of significant fascicle sizes, defined as median size representatives of three groups: small (1 population implemented), medium (5 populations implemented) and large (9 populations). Different device placements were also studied: within, close, or far from fascicle, finally resulting in n = 90 simulations for TIME and n = 45 simulations cases for microneedle (this is because for every position of needle tip there are 2 corresponding positions of TIME: left and right).
The microstimulation needle (FHC UNP40GAS) was replicated as a cylinder with a cone-like ending (Figure 6): the cylinder had 3 mm length, and cone (tip) with semi-angle 12.78° and shank diameter 250 μm. The whole structure was insulated and had electrical conductivity of 6.67 10–14 S/m and the tip was non-insulated with a conductivity of 1.89 107 S/m. Since in the case of microstimulation needle the tip was un-insulated by the neurologist, imminently before the insertion, the precise un-insulated cone dimension is not known. In order to account for this uncertainty, two models were implemented and the results analyzed for both (Figure 8). The TIME electrode was built as a rectangular structure where seven circled active sites were placed in each side of the structure (Figure 6). Conductivity of the polyimide substrate was set to 6.67 10–14 S/m. The radius for the active sites was 40 μm and the thickness 300 nm. The overall structure had 4 mm length, 0.35 mm width and 20 μm thickness. The electrical conductivity values used for the FEM were [0.0826i 0.0826j 0.571k] S/m for the endoneurium (Raspopovic et al., 2011; McIntyre et al., 2002) , 880 μS/m for perineurium, 0.0826 S/m for the epineurium (Raspopovic et al., 2011; McIntyre et al., 2002) , 2 S/m for saline.
Conductivity of perineurium was recalculated from a previous study (Weerasuriya et al., 1984) taking into account the thickness of the perineurium as 3% of diameter of the fascicle and the difference of temperature between frogs and humans. Although TIME electrode was implanted in vivo and no saline placed outside the nerve, this value was used as in previous studies (Raspopovic et al., 2011; McIntyre et al., 2002).
The similarity between needle and TIME is estimated by comparison of charges necessary for 10% of recruitment of fibers, for respectively 45 and 90 populations computed, by means of Kruskal-Wallis test, with significance level fixed at 0.05.
Acknowledgements
The Authors are deeply grateful to all the Participants, and in particular to DAS who freely donated six weeks of his life for the advancement of knowledge and for a better future of People with hand amputation. The Authors are also grateful to Prof. Thomas Stieglitz and his team for the development of the TIMEs, to Prof. Eduardo Fernandez for their surgical implantation, and to Prof. Xavier Navarro for providing histological data of the median nerve. The authors also thank Mr. Matteo Moisé for the contributions in the preparation of the artwork.
The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreements n° 224012 (TIME, Transverse Intrafascicular Multichannel Electrode system), 228844 (NANOBIOTOUCH, Nanoresolved multi-scan investigations of human tactile sensations and tissue engineered nanobiosensor) and 611687 (NEBIAS, NEuro-controlled BIdirectional Artificial upper limb and hand prosthesiS), from the Italian Ministry of Health (NEMESIS project), from the Italian Ministry of Research (PRIN/HandBot, Biomechatronic hand prostheses endowed with bio-inspired tactile perception, bi-directional neural interfaces and distributed sensori-motor control, CUP: B81J12002680008), from the National Institute for Insurance against Industrial Injuries (PPR2, Control of hand prosthesis by invasive neural interfaces), from the Wyss Center for Bio and Neuroengineering, and from the Swiss National Competence Center in Research in Robotics.
Funding Statement
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Funding Information
This paper was supported by the following grants:
Directorate-General for Communications Networks, Content and Technology EU Grant CP-FP-INFSO 224012 TIME project to Stanisa Raspopovic, Francesco Petrini, Federica Giambattistelli, Fabrizio Vecchio, Francesca Miraglia, Giovanni Di Pino, Paolo Maria Rossini, Silvestro Micera.
Directorate-General for Communications Networks, Content and Technology EU Grant FET 611687 NEBIAS Project to Calogero Maria Oddo, Stanisa Raspopovic, Alberto Mazzoni, Domenico Camboni, Paolo Maria Rossini, Silvestro Micera.
Directorate-General for Communications Networks, Content and Technology EU Grant FP7-NMP 228844 NANOBIOTOUCH project to Calogero Maria Oddo, Giacomo Spigler, Domenico Camboni, Maria Chiara Carrozza.
Ministero della Salute Italian NEMESIS (Neurocontrolled mechatronic hand prosthesis) to Calogero M Oddo, Stanisa Raspopovic, Francesco Petrini, Paolo M Rossini, Silvestro Micera.
Ministero dell’Istruzione, dell’Università e della Ricerca Italian project PRIN/HandBot to Calogero M Oddo, Fiorenzo Artoni, Alberto Mazzoni, Giacomo Spigler, Francesco Petrini, Federica Giambattistelli, Loredana Zollo, Giovanni Di Pino, Domenico Camboni, Eugenio Guglielmelli, Silvestro Micera.
Italian National Institute for Insturance against Industrial Injuries National project PPR2 (Control of hand prosthesis by invasive neural interfaces) to Loredana Zollo, Eugenio Guglielmelli.
Swiss National Competence Center in Research in Robotics NCCR Robotics to Stanisa Raspopovic, Francesco Petrini, Silvestro Micera.
Wyss Center for Bio and Neuroengineering ENABLE Project to Silvestro Micera.
Additional information
Competing interests
CMO: Has a patent entitled "Method and apparatus for transmitting tactile sensations to an user" (Italian patent number 0001417070).
SR: Has a patent entitled "Bidirectional limb neuroprosthesis" pending (PCT/IB2014/067143).
The other authors declare that no competing interests exist.
GS: Has a patent entitled "Method and apparatus for transmitting tactile sensations to an user" (Italian patent number 0001417070).
FP: Has a patent entitled "Bidirectional limb neuroprosthesis" pending (PCT/IB2014/067143).
DC: Has a patent entitled "Method and apparatus for transmitting tactile sensations to an user" (Italian patent number 0001417070).
MCC: Has a patent entitled "Method and apparatus for transmitting tactile sensations to an user" (Italian patent number 0001417070).
SM: Has a patent entitled "Bidirectional limb neuroprosthesis" pending (PCT/IB2014/067143).
Author contributions
CMO, Designed the study, supervised and conducted the experiments, developed the sensorized finger and the MNT process, supervised system integration, analyzed data, discussed the results and wrote the paper.
SR, Co-designed the study, Supervised and conducted experiments, Developed the hybrid electrical-biophysical model, Analyzed data, and wrote the paper.
FA, Performed EEG signal processing and wrote the related parts of the paper.
AM, Analyzed sensor output data and behavioral data and wrote the paper.
GS, Developed the MNT process and collaborated during system integration and during the experiments.
FP, Collaborated during system integration and during all the experiments, Conducted intact human experiments and wrote the related parts of the paper, Contributed to the design of the hybrid electrical-biophysical model.
FG, Conducted percutaneous microstimulation sessions with intact subjects.
FV, Performed EEG functional connectivity analysis and EEG graph analysis and wrote the related parts of the paper.
FM, Performed EEG functional connectivity analysis and EEG graph analysis and wrote the related parts of the paper.
LZ, Collaborated during the preparation and execution of the experiments.
GDP, Selected the amputee subject and collaborated during the experiments and to the discussion of EEG data.
DC, Collaborated during system integration, Collaborated during the experiments and contributed to the preparation of artwork of the paper.
MCC, Collaborated during the preparation of the experiments and developed the sensorized finger.
EG, Collaborated during the preparation of the experiments.
PMR, Selected the amputee subject and supervised EEG functional connectivity analysis and EEG graph analysis and wrote the related parts of the paper.
UF, Was responsible of EEG signals recording, supervised EEG signal processing and wrote the related parts of the paper.
SM, Designed the study, Supervised the experiments, Discussed the results, and wrote the paper.
Ethics
Human subjects: Participant DAS had suffered a traumatic transradial left arm amputation 10 years before the experiments. He was selected from a group of 31 persons with hand amputation because of the stump characteristics and his psychophysical abilities. All procedures were approved by the Institutional Ethics Committees of Policlinic A. Gemelli at Catholic University, where the surgery was performed, IRCCS S. Raffaele Pisana (Rome), where the experiments were performed, the Ethics Committee of Campus Bio-Medico University and the Italian Ministry of Health. Four intact subjects (2 males and 2 females, 25-26 years old) underwent the experiments of texture discrimination with tactile feedback elicited by a stimulation injected through two microneurographic electrodes (FHC UNP40GAS, shaped as needles). The enrolment was subjected to signing a written Informed Consent, approved by the Campus Bio-Medico University Ethics Committee, where this set of experiments took place.
Additional files
Major datasets
The following dataset was generated:
Oddo CM, Raspopovic S, Artoni F, Mazzoni A, Spigler G, Petrini F, Federica G, Fabrizio V, Francesca M, Loredana Z, Di Pino G, Camboni D, Carrozza MC, Guglielmelli E, Rossini PM, Faraguna U, Micera S,2015,Data from Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans,http://dx.doi.org/10.5061/dryad.630hf,Available at Dryad Digital Repository under a CC0 Public Domain Dedication
References
- Abraira VE, Ginty DD. The sensory neurons of touch. Neuron. 2013;79:618–639. 10.1016/j.neuron.2013.07.051. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Acar ZA, Makeig S. Neuroelectromagnetic forward head modeling toolbox. Journal of Neuroscience Methods. 2010;190:258–270. 10.1016/j.jneumeth.2010.04.031. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Artoni F, Menicucci D, Delorme A, Makeig S, Micera S. RELICA: a method for estimating the reliability of independent components. NeuroImage. 2014;103:391–400. 10.1016/j.neuroimage.2014.09.010. [Abstract] [CrossRef] [Google Scholar]
- Beccai L, Roccella S, Arena A, Valvo F, Valdastri P, Menciassi A, Carrozza MC, Dario P. Design and fabrication of a hybrid silicon three-axial force sensor for biomechanical applications. Sensors and Actuators A: Physical. 2005;120:370–382. 10.1016/j.sna.2005.01.007. [CrossRef] [Google Scholar]
- Boretius T, Badia J, Pascual-Font A, Schuettler M, Navarro X, Yoshida K, Stieglitz T. A transverse intrafascicular multichannel electrode (tIME) to interface with the peripheral nerve. Biosensors & Bioelectronics. 2010;26:62–69. 10.1016/j.bios.2010.05.010. [Abstract] [CrossRef] [Google Scholar]
- Borst A, Theunissen FE. Information theory and neural coding. Nature Neuroscience. 1999;2:947–957. 10.1038/14731. [Abstract] [CrossRef] [Google Scholar]
- Darian-Smith I, Davidson I, Johnson KO. Peripheral neural representation of spatial dimensions of a textured surface moving across the monkey's finger pad. The Journal of Physiology. 1980a;309:135–146. 10.1113/jphysiol.1980.sp013499. [Abstract] [CrossRef] [Google Scholar]
- Darian-Smith I, Oke LE. Peripheral neural representation of the spatial frequency of a grating moving across the monkey's finger pad. The Journal of Physiology. 1980b;309:117–133. 10.1113/jphysiol.1980.sp013498. [Abstract] [CrossRef] [Google Scholar]
- Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods. 2004;134:9–21. 10.1016/j.jneumeth.2003.10.009. [Abstract] [CrossRef] [Google Scholar]
- Delorme A, Palmer J, Onton J, Oostenveld R, Makeig S. Independent EEG sources are dipolar. PLoS ONE. 2012;7:e09148. 10.1371/journal.pone.0030135. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Dhillon GS, Horch KW. Direct neural sensory feedback and control of a prosthetic arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2005;13:468–472. 10.1109/TNSRE.2005.856072. [Abstract] [CrossRef] [Google Scholar]
- Di Pino G, Porcaro C, Tombini M, Assenza G, Pellegrino G, Tecchio F, Rossini PM. A neurally-interfaced hand prosthesis tuned inter-hemispheric communication. Restorative Neurology and Neuroscience. 2012;30 10.3233/RNN-2012-120224. [Abstract] [CrossRef] [Google Scholar]
- Di Pino G, Denaro L, Vadalà G, Marinozzi A, Tombini M, Ferreri F, Papalia R, Accoto D, Guglielmelli E, Di Lazzaro V, Denaro V. Invasive neural interfaces: the perspective of the surgeon. Journal of Surgical Research. 2014;188:77–87. 10.1016/j.jss.2013.12.014. [Abstract] [CrossRef] [Google Scholar]
- Dépeault A, Meftah el-M, Chapman CE, Chapman CE. Tactile speed scaling: contributions of time and space. Journal of Neurophysiology. 2008;99 10.1152/jn.01209.2007. [Abstract] [CrossRef] [Google Scholar]
- Edin BB, Westling G, Johansson RS. Independent control of human finger-tip forces at individual digits during precision lifting. The Journal of Physiology. 1992;450:547–564. 10.1113/jphysiol.1992.sp019142. [Abstract] [CrossRef] [Google Scholar]
- Edin BB, Johansson N. Skin strain patterns provide kinaesthetic information to the human central nervous system. The Journal of Physiology. 1995;487:243–251. 10.1113/jphysiol.1995.sp020875. [Abstract] [CrossRef] [Google Scholar]
- Gardner EP, Palmer CI. Simulation of motion on the skin. I. receptive fields and temporal frequency coding by cutaneous mechanoreceptors of OPTACON pulses delivered to the hand. Journal of Neurophysiology. 1989;62:1410–1436. [Abstract] [Google Scholar]
- Gibson GO, Craig JC. Tactile spatial sensitivity and anisotropy. Perception & Psychophysics. 2005;67:1061–1079. 10.3758/BF03193632. [Abstract] [CrossRef] [Google Scholar]
- Grinberg Y, Schiefer MA, Tyler DJ, Gustafson KJ. Fascicular perineurium thickness, size, and position affect model predictions of neural excitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2008;16:572–581. 10.1109/TNSRE.2008.2010348. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Hagbarth KE, Vallbo AB. Mechanoreceptor activity recorded percutaneously with semi-microelectrodes in human peripheral nerves. Acta Physiologica Scandinavica. 1967;69:121–122. 10.1111/j.1748-1716.1967.tb03498.x. [Abstract] [CrossRef] [Google Scholar]
- Hayward V, Terekhov AV, Wong S-C, Geborek P, Bengtsson F, Jorntell H. Spatio-temporal skin strain distributions evoke low variability spike responses in cuneate neurons. Journal of the Royal Society Interface. 2014;11:20131015. 10.1098/rsif.2013.1015. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Hollins M, Risner SR. Evidence for the duplex theory of tactile texture perception. Perception & Psychophysics. 2000;62:695–705. 10.3758/BF03206916. [Abstract] [CrossRef] [Google Scholar]
- Izhikevich EM. Simple model of spiking neurons. IEEE Transactions on Neural Networks. 2003;14:1569–1572. 10.1109/TNN.2003.820440. [Abstract] [CrossRef] [Google Scholar]
- Jabaley ME, Wallace WH, Heckler FR. Internal topography of major nerves of the forearm and hand: a current view. The Journal of Hand Surgery. 1980;5:1–18. 10.1016/S0363-5023(80)80035-9. [Abstract] [CrossRef] [Google Scholar]
- Johansson RS, Flanagan JR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature Reviews. Neuroscience. 2009;10:345–359. 10.1038/nrn2621. [Abstract] [CrossRef] [Google Scholar]
- Johnson KO, Yoshioka T, Vega-Bermudez F. Tactile functions of mechanoreceptive afferents innervating the hand. Journal of Clinical Neurophysiology. 2000;17:539–558. 10.1097/00004691-200011000-00002. [Abstract] [CrossRef] [Google Scholar]
- Johnson K. The roles and functions of cutaneous mechanoreceptors. Current Opinion in Neurobiology. 2001;11:455–461. 10.1016/S0959-4388(00)00234-8. [Abstract] [CrossRef] [Google Scholar]
- Jones LA, Smith AM. Tactile sensory system: encoding from the periphery to the cortex. Wiley Interdisciplinary Reviews. Systems Biology and Medicine. 2014;6:279–287. 10.1002/wsbm.1267. [Abstract] [CrossRef] [Google Scholar]
- Jörntell H, Bengtsson F, Geborek P, Spanne A, Terekhov AV, Hayward V. Segregation of tactile input features in neurons of the cuneate nucleus. Neuron. 2014;83:1444–1452. 10.1016/j.neuron.2014.07.038. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Kunesch E, Knecht S, Schnitzler A, Tyercha C, Schmitz F, Freund H-J. Somatosensory evoked potentials elicited by intraneural microstimulation of afferent nerve fibers. Journal of Clinical Neurophysiology. 1995;12:476–487. 10.1097/00004691-199509010-00007. [Abstract] [CrossRef] [Google Scholar]
- Kwok R. Neuroprosthetics: once more, with feeling. Nature. 2013;497:176–178. 10.1038/497176a. [Abstract] [CrossRef] [Google Scholar]
- Lee T-W, Girolami M, Sejnowski TJ. Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation. 1999;11:417–441. 10.1162/089976699300016719. [Abstract] [CrossRef] [Google Scholar]
- Lánský P, Rodriguez R, Sacerdote L. Mean instantaneous firing frequency is always higher than the firing rate. Neural Computation. 2004;16:477–489. 10.1162/089976604772744875. [Abstract] [CrossRef] [Google Scholar]
- Makeig S, Bell AJ, Jung T-P, Sejnowski TJ. Independent component analysis of electroencephalographic data. Advances in Neural Information Processing Systems. 1996:145–151. [Google Scholar]
- Maris E, Oostenveld R. Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods. 2007;164:177–190. 10.1016/j.jneumeth.2007.03.024. [Abstract] [CrossRef] [Google Scholar]
- McIntyre CC, Richardson AG, Grill WM. Modeling the excitability of mammalian nerve fibers: influence of afterpotentials on the recovery cycle. Journal of Neurophysiology. 2002;87:995–1006. [Abstract] [Google Scholar]
- Oddo CM, Valdastri P, Beccai L, Roccella S, Carrozza MC, Dario P. Investigation on calibration methods for multi-axis, linear and redundant force sensors. Measurement Science and Technology. 2007;18:623–631. 10.1088/0957-0233/18/3/011. [CrossRef] [Google Scholar]
- Oddo CM, Controzzi M, Beccai L, Cipriani C, Carrozza MC. Roughness encoding for discrimination of surfaces in artificial active-touch. IEEE Transactions on Robotics. 2011a;27:522–533. 10.1109/TRO.2011.2116930. [CrossRef] [Google Scholar]
- Oddo CM, Beccai L, Wessberg J, Wasling HB, Mattioli F, Carrozza MC. Roughness encoding in human and biomimetic artificial touch: spatiotemporal frequency modulation and structural anisotropy of fingerprints. Sensors. 2011b;11:5596–5615. 10.3390/s110605596. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Oddo CM, Controzzi M, Beccai L, Cipriani C, Carrozza MC. Roughness encoding for discrimination of surfaces in artificial active-touch. IEEE Transactions on Robotics. 2011c;27:522–533. 10.1109/TRO.2011.2116930. [CrossRef] [Google Scholar]
- Oddo CM, Beccai L, Vitiello N, Wasling HB, Wessberg J, Carrozza MC. A mechatronic platform for human touch studies. Mechatronics. 2011d;21:604–613. 10.1016/j.mechatronics.2011.02.012. [CrossRef] [Google Scholar]
- Oddo CM, Raspopovic S, Artoni F, Mazzoni A, Spigler G, Petrini F, Federica G, Fabrizio V, Francesca M, Loredana Z, Di Pino G, Camboni D, Carrozza MC, Guglielmelli E, Rossini PM, Faraguna U, Micera S. 2016. Data from: Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. Dryad Digital Repository. [CrossRef]
- Oostenveld R, Praamstra P. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology. 2001;112:713–719. 10.1016/S1388-2457(00)00527-7. [Abstract] [CrossRef] [Google Scholar]
- Oostenveld R, Oostendorp TF. Validating the boundary element method for forward and inverse EEG computations in the presence of a hole in the skull. Human Brain Mapping. 2002;17:179–192. 10.1002/hbm.10061. [Abstract] [CrossRef] [Google Scholar]
- Oostenveld R, Fries P, Maris E, Schoffelen JM. FieldTrip open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience. 2011;156869 [Europe PMC free article] [Abstract] [Google Scholar]
- Ortiz-Catalan M, Hakansson B, Branemark R. An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs. Science Translational Medicine. 2014;6:257re6. 10.1126/scitranslmed.3008933. [Abstract] [CrossRef] [Google Scholar]
- Palmer JA, Kreutz-Delgado K, Rao BD, Makeig S. Independent component analysis and signal separation; 2007. pp. 97–104. [Google Scholar]
- Panzeri S, Brunel N, Logothetis NK, Kayser C. Sensory neural codes using multiplexed temporal scales. Trends in Neurosciences. 2010;33:111–120. 10.1016/j.tins.2009.12.001. [Abstract] [CrossRef] [Google Scholar]
- Pascual-Marqui RD. Instantaneous and lagged measurements of linear and nonlinear dependence between groups of multivariate time series: frequency decomposition. arXiv 2007 [Google Scholar]
- Pascual-Marqui RD, Lehmann D, Koukkou M, Kochi K, Anderer P, Saletu B, Tanaka H, Hirata K, John ER, Prichep L, Biscay-Lirio R, Kinoshita T. Assessing interactions in the brain with exact low-resolution electromagnetic tomography. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2011;369:3768–3784. 10.1098/rsta.2011.0081. [Abstract] [CrossRef] [Google Scholar]
- Perez CA, Donoso JR, Medina LE. A critical experimental study of the classical tactile threshold theory. BMC Neuroscience. 2010;11:76. 10.1186/1471-2202-11-76. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Phillips JR, Johnson KO. Tactile spatial resolution. II. neural representation of bars, edges, and gratings in monkey primary afferents. Journal of Neurophysiology. 1981;46:1192–1203. [Abstract] [Google Scholar]
- Raspopovic S, Capogrosso M, Micera S. A computational model for the stimulation of rat sciatic nerve using a transverse intrafascicular multichannel electrode. Ieee T Neur Sys Reh. 2011;19:333–344. 10.1109/Tnsre.2011.2151878. [Abstract] [CrossRef] [Google Scholar]
- Raspopovic S, Capogrosso M, Petrini FM, Bonizzato M, Rigosa J, Di Pino G, Carpaneto J, Controzzi M, Boretius T, Fernandez E, Granata G, Oddo CM, Citi L, Ciancio AL, Cipriani C, Carrozza MC, Jensen W, Guglielmelli E, Stieglitz T, Rossini PM, Micera S. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Science Translational Medicine. 2014;6:222ra19. 10.1126/scitranslmed.3006820. [Abstract] [CrossRef] [Google Scholar]
- Rongala UB, Mazzoni A, Oddo CM. Neuromorphic artificial touch for categorization of naturalistic textures. IEEE Transactions on Neural Networks and Learning Systems. 2015:1–11. 10.1109/TNNLS.2015.2472477. [Abstract] [CrossRef] [Google Scholar]
- Rossini PM, Micera S, Benvenuto A, Carpaneto J, Cavallo G, Citi L, Cipriani C, Denaro L, Denaro V, Di Pino G, Ferreri F, Guglielmelli E, Hoffmann KP, Raspopovic S, Rigosa J, Rossini L, Tombini M, Dario P. Double nerve intraneural interface implant on a human amputee for robotic hand control. Clinical Neurophysiology. 2010;121:777–783. 10.1016/j.clinph.2010.01.001. [Abstract] [CrossRef] [Google Scholar]
- Saal HP, Bensmaia SJ. Biomimetic approaches to bionic touch through a peripheral nerve interface. Neuropsychologia. 2015;79 10.1016/j.neuropsychologia.2015.06.010. [Abstract] [CrossRef] [Google Scholar]
- Sathian K, Goodwin AW, John KT, Darian-Smith I. Perceived roughness of a grating: correlation with responses of mechanoreceptive afferents innervating the monkey's fingerpad. The Journal of Neuroscience. 1989;9:1273–1279. [Abstract] [Google Scholar]
- Spigler G, Oddo CM, Carrozza MC. 4th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics (BioRob); 2012. pp. 1913–1918. [Google Scholar]
- Tan DW, Schiefer MA, Keith MW, Anderson JR, Tyler J, Tyler DJ. A neural interface provides long-term stable natural touch perception. Science Translational Medicine. 2014;6:257ra138. 10.1126/scitranslmed.3008669. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Theunissen F, Miller JP. Temporal encoding in nervous systems: a rigorous definition. Journal of Computational Neuroscience. 1995;2:149–162. [Abstract] [Google Scholar]
- Torebjörk HE, Ochoa JL. Specific sensations evoked by activity in single identified sensory units in man. Acta Physiologica Scandinavica. 1980;110:445–447. 10.1111/j.1748-1716.1980.tb06695.x. [Abstract] [CrossRef] [Google Scholar]
- Torebjörk HE, Vallbo AB, Ochoa JL. Intraneural microstimulation in man. its relation to specificity of tactile sensations. Brain. 1987;110:1509–1529. [Abstract] [Google Scholar]
- Vallbo AB, Johansson RS. Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Human Neurobiology. 1984a;3:3–14. [Abstract] [Google Scholar]
- Vallbo AB, Olsson KA, Westberg KG, Clark FJ. Microstimulation of single tactile afferents from the human hand. sensory attributes related to unit type and properties of receptive fields. Brain. 1984b;107:727–749. [Abstract] [Google Scholar]
- Vallbo AB, Hagbarth KE, Wallin BG. Microneurography: how the technique developed and its role in the investigation of the sympathetic nervous system. Journal of Applied Physiology. 2004;96:1262–1269. 10.1152/japplphysiol.00470.2003. [Abstract] [CrossRef] [Google Scholar]
- Vecchio F, Miraglia F, Bramanti P, Rossini PM. Human brain networks in physiological aging: a graph theoretical analysis of cortical connectivity from EEG data. Journal of Alzheimer's Disease. 2014a;41:1239–1249. 10.3233/JAD-140090. [Abstract] [CrossRef] [Google Scholar]
- Vecchio F, Miraglia F, Marra C, Quaranta D, Vita MG, Bramanti P, Rossini PM. Human brain networks in cognitive decline: a graph theoretical analysis of cortical connectivity from EEG data. Journal of Alzheimer's Disease. 2014b;41:113–127. 10.3233/JAD-132087. [Abstract] [CrossRef] [Google Scholar]
- Vecchio F, Miraglia F, Curcio G, Della Marca G, Vollono C, Mazzucchi E, Bramanti P, Rossini PM. Cortical connectivity in fronto-temporal focal epilepsy from EEG analysis: a study via graph theory. Clinical Neurophysiology. 2015a;126:1108–1116. 10.1016/j.clinph.2014.09.019. [Abstract] [CrossRef] [Google Scholar]
- Vecchio F, Miraglia F, Valeriani L, Scarpellini MG, Bramanti P, Mecarelli O, Rossini PM. Cortical brain connectivity and b-type natriuretic peptide in patients with congestive heart failure. Clinical EEG and Neuroscience. 2015b;46:224–229. 10.1177/1550059414529765. [Abstract] [CrossRef] [Google Scholar]
- Weber AI, Saal HP, Lieber JD, Cheng J-W, Manfredi LR, Dammann JF, Bensmaia SJ. Spatial and temporal codes mediate the tactile perception of natural textures. Proceedings of the National Academy of Sciences of the United States of America. 2013;110:17107–17112. 10.1073/pnas.1305509110. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Weerasuriya A, Spangler RA, Rapoport SI, Taylor RE. AC impedance of the perineurium of the frog sciatic nerve. Biophysical Journal. 1984;46:167–174. 10.1016/S0006-3495(84)84009-6. [Europe PMC free article] [Abstract] [CrossRef] [Google Scholar]
- Yoshioka T, Gibb B, Dorsch AK, Hsiao SS, Johnson KO. Neural coding mechanisms underlying perceived roughness of finely textured surfaces. The Journal of Neuroscience. 2001;21:6905–6916. [Abstract] [Google Scholar]
Decision letter
Thank you for submitting your work entitled "Intraneural stimulation can restore discrimination of textural features by artificial fingertips in humans" for peer review at eLife. Your submission has been favorably evaluated by Timothy Behrens (Senior editor), a Reviewing editor, and two reviewers, one of whom is a member of our Board of Reviewing Editors.
The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.
Summary:
This paper reports further on the very good pioneering work by these authors (previously reported) regarding the sensory/tactile input into the peripheral nerve. This paper extends the previous work, with some different aspects, although there is also a considerable overlap (which must be explained). Specifically, the authors study the use of NMT and TIME, in volunteer studies, in the study of a transradial amputee and in a simulation investigation as a way to restore tactile sensation.
The main result of neuromorphic stimulation, and neural response are interesting first presentations.
The strength of the paper is the very detailed experimentation, involving microneurography and the sensory input provided in an amputee. These experiments are aided by giving a biomimetic or neuromorphic input. Further, modeling shows how microneurography and TIM electrodes work comparably. This type of modeling coupled to experiments shows that this paper represents thorough research. The cortical response further validates that sensory perception is observed – this is quite interesting and novel as well. Overall, the work has enormous promise for establishing a paradigm for restoring tactile sensation to patients with limb injuries.
Essential revisions:
First, and most importantly, previous work by Raspopovic, more or less lays out the tactile stimulation. Also, the authors have published their sensors work. The new aspect seems to be the biologically plausible model. But the model selected – Izh ikevich is not a biological model of receptors – does not produce receptor-spiking activity and no transduction is built in.
In the subheading “Mechano-neuro-transduction (MNT) process”, sensor to spike output from Izhikevich is not an ideal receptor to neural activity transformation. The authors should point out this simplification – actually a limitation. The periodic rate output is also a significant simplification. The model produces bursts vs single spikes, but it's not clear it is biomimetic. This interpretation also needs some caution.
Second, the paper does not clearly and cleanly present what is novel here – and differentiate from many prior papers by the authors. What is novel – the stimulus pattern? The computational model? Comparison with 4 normal, amputee? Distinguish from the prior work by the same group. Please emphasize this point in your response letter and revised manuscript. The question of novelty will be critical in making a final decision on the publication of this work.
Third, the work presents tactile encoding – in particular coarse + geometric -> shape sensing. This is not clear. Is it only the artifact of their stimulus (grating) or is this the result of perception by the subjects – having tested many stimuli. i.e. is the perceptual output of the given microneurographic access to nerves and penetrating electrodes' access to the fascicles? If so, how specific and representative is it?
Fourth, the statistical analysis of the data should be improved. The text and Figure 1 discuss the results of the study in terms of the overall success rate. Figure 3 shows the breakdown of the responses by subject. There is a conspicuous lack of analysis of the data. The data should be analyzed not by lumping together all of the subjects and reporting an aggregate successful discrimination rate, but rather by individual subject as a function of stimulus type. The coarser the stimulus, the greater the discriminatory power and the weaker the stimulus the less the discriminatory power. There should be an analysis showing that the performance by individual is greater than what would be expected by chance, where chance here is 0.33. If there is an approximate monotone relation among the stimuli then a logistic regression analysis could be performed. The analysis should be conducted using confidence intervals or Bayesian methods and not just simply by reporting p-values. An advantage of a Bayesian hierarchical analysis or a random effects model across subjects is that they provide a formal way of pooling information across subjects. The latter is crucial for this problem because the authors which to establish that they have been able to provide neuromorphically tactile sensation. This statement should be made with an assessment of the accuracy of the discriminatory power of the MNT technique for each subject as a function of stimulus type. It is likely that the results will reveal that more trials will be required to show that there is truly discriminatory capability.
The authors state that they observed no difference in the natural EEG activity and evoked stimulation in source topography, response timing and clustering. How were these assessments made? They involve negative results and require a power analysis to establish their validity.
Although the data are more compelling for the TIME technique applied to the transradial amputee patient, a similar analysis should be performed for these data as well.
The authors wish to establish from their simulation analyses that the response patterns under MNT and TIME are similar if not effectively the same. It is unclear where the uncertainty comes from in these analyses since they are simulation models. Therefore how can statistical assessments of uncertainty and formal statistical inferences be made? Moreover, the authors wish to infer that the stimulation responses of the two modalities do not differ. They report a p-value without a specific statement about the statistical test being used in the analysis. Because the authors wish to establish a negative result of no difference between the two stimulations modalities, they should report a power analysis stating what types and magnitudes of differences their investigations were calibrated to detect.
The authors do not have to develop an ancillary classifier model to understand the performance characteristics of the MNT and the TIME paradigms. If they build statistical models to analyze the data from the subjects that include covariates that has IBI they will get their answer with the same model being used to analyze the data. This approach would obviate the current analyses which lead to multiple comparisons corrections.
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Thank you for resubmitting your work entitled "Intraneural stimulation can restore discrimination of textural features by artificial fingertips in humans" for further consideration at eLife. Your revised article has been favorably evaluated by Timothy Behrens (Senior editor), a Reviewing editor, and one reviewer. The manuscript has been improved but there are a few remaining issues that need to be addressed before acceptance, as described below.
General Comments:
This paper has been revised, and more importantly, careful and thoughtful rebuttal to all the questions has been provided. Supplementary figures and additional statistical analysis are also appreciated. Overall, the paper is acceptable. The authors should note a few salient points and revise their paper accordingly (no further review is needed).
This is an outstanding piece of work. The paper has many novel aspects, now better clarified as requested, to point out the texture discrimination capability in humans, different stimuli and temporal coding, comparison with needle vs intrafascicular, etc. The cortical, neural topographic map, and source localization strengthen the paper. The claim about neuromorphic/Izhikevich is now moderated, although the authors still argue that this is representation of tactile receptors, bur arguably this is still a significant approximation based on the Izhikevich model.
The paper's distinction from the previous, Raspopovic paper is now made and is acceptable. The comments made concerning this, from texture to encoding model, are acceptable.
The novelty aspect has been responded to: a) Texture discrimination (but not "restoration" – see below);
b) Hybrid FEM model is very useful and demonstrates TIME's utility and various fiber and recruiting; c) Perceptual studies, both psychophysical and cortical, strengthen the paper.
Suggested Revisions:
1) The claim in the Abstract is still too broad: "Intraneural MNT-based stimulation restored discrimination of textural features, thus enhancing the user's tactile capabilities." The word "restore" is too strong. It implies permanent restoration/recovery. Here a texture perception has been mimicked, and performance has not been "enhanced."
2) Paragraph two, subheading “Analysis of neural coding strategies”: This information pertains to Figure 10 (e). The message that the rate (in e, worse) vs temporal code (in d, good) is getting lost in this paragraph. This is an important observation and clarification would help. The figure caption could be more explanatory too.
3) Paragraph two, subheading “Hybrid electrical-biophysical model of the median nerve for the comparison between microstimulation needle and implanted TIME”: As the fiber organization within different fascicles in the nerve is unknown, we assumed that fibers within the fascicles belong to same functionality.
Please clarify how nociceptive fibers were separated from sensory (e.g. see the comment pertaining to the model).Indeed, this issue or not being able to separate different fiber types is quite critical and must be marked as a limitation.
4) Figure 2: Traces and colors are not well explained.
5) Figure 6 caption: The X marker represents the targeted fascicle where the fiber activation per 9 different populations was calculated. What does this mean (what populations?) for different locations of microneedle and TIME? This procedure is carried out analogously for medium and small fascicles, confirming results. Please could you also clarify the procedure.
6) Figure 8: outcomes. Color-coding is not explained
7) Figure 10: This figure, slightly modified to clarify rate vs temporal, would be much clearer (and certainly this result is important).
Author response
Essential revisions:
First, and most importantly, previous work by Raspopovich, more or less lays out the tactile stimulation. Also, the authors have published their sensors work. The new aspect seems to be the biologically plausible model. But the model selected – Izhi kevich is not a biological model of receptors – does not produce receptor-spiking activity and no transduction is built in.
In the subheading “Mechano-neuro-transduction (MNT) process”, sensor to spike output from Izhikevich is not an ideal receptor to neural activity transformation. The authors should point out this simplification – actually a limitation. The periodic rate output is also a significant simplification. The model produces bursts vs single spikes, but it's not clear it is biomimetic. This interpretation also needs some caution.
We thank the reviewers for this comment, which allows us to clarify how the mechano-neuro-transduction actually works in our experimental design:
1) We modified the Methods section in order to better point out the limitations of our model. We highlighted that our model:
– Is a modified version of the original Izhikevich model, which was conceived to emulate neuron-to-neuron signaling, whereas in our modified implementation its input is haptic rather than synaptic.
– Does not account for the fine details of the complex spatio-temporal mechanical interaction between physical stimuli, soft tissues and receptors (Hayward et al., 2014).
2) Notwithstanding these limitations, as discussed in the revised paper, our implementation captures the fundamental properties of the firing behavior of SA1 afferents associated to Merkel receptors, a functionally relevant class of tactile units able to encode medium-coarse textures (Darian-Smith and Oke, 1980, Phillips and Johnson, 1981, Yoshioka et al., 2001, Weber et al., 2013), such as the surfaces evaluated in the present study. Specifically:
– In the study by Darian-Smith and Oke (e.g., in Figure 6 of such prior work) the spatial structure of moving gratings characterized by a temporal frequency in the order of 20 Hz is encoded by SA1 afferents into a coherent temporal spiking activity with about one spike per ridge of the stimulus (Darian-Smith and Oke, 1980). Coherently with such previous neurophysiological data, in our study stimuli with 0.5 mm spatial period (resulting in a 20 Hz temporal frequency since the sliding velocity is 10 mm/s) induce about one spike per ridge (Figure 9d of our manuscript).
– As shown in our manuscript in the raster plot in Figure 1b and in Figure 9, the encoding of our coarser surfaces (with 1.0 mm, 1.5 mm, 2.0 mm and 3.0 mm spatial period) reflects the physiological tendency of SA1 units to fire more than one spike per ridge for coarser spatial periods, with bursts properly modulated by the surface structure (towards longer IBI), as depicted in Figure 1 and Figure 2 of the previous study previously by Phillips and Johnson (1981).
– To evaluate such evidences in a more quantitative way, we added Figure 9—figure supplement 1, that shows the spatial modulation index calculated on the spike patterns of our study, to be compared with Figure 9 of the previous study by Phillips and Johnson (1981). The modulation index shown in the spike patterns produced by our artificial touch system reflects the trend (monotonically growing with the spatial period) that was previously reported with physiological data.
3) In the revised manuscript we also clarify that the periodicity of the bursts of neural stimulation delivered to the subjects is not inherent to the model, which is able to fire any kind of spike train patterns including the highly irregular ones induced by the contact with complex surfaces. The periodicity is instead a direct consequence of the experimented stimuli, which are periodic gratings and explored with constant velocity. Hence, the spatial structure of gratings is transduced in a temporal sequence that maps the textural/geometrical features of the stimuli. The capability of the implemented model to generalize with daily life naturalistic stimuli (either macroscopically aperiodic, such as latex and velvet, or with a fine periodic structure such as textiles) was recently demonstrated in a fully-robotic artificial touch study (Rongala et al., 2015), and such set of stimuli will be evaluated in the future protocols with amputees and chronically-implanted neural interfaces.
Second, the paper does not clearly and cleanly present what is novel here – and differentiate from many prior papers by the authors. What is novel – the stimulus pattern? The computational model? Comparison with 4 normal, amputee? Distinguish from the prior work by the same group. Please emphasize this point in your response letter and revised manuscript. The question of novelty will be critical in making a final decision on the publication of this work.
The main novelties of this work are hereafter summarized (and better explained in the manuscript):
1) This paper shows for the first time the possibility to restore in amputees, and induce in intact subjects, the real-time texture perception of surfaces with medium-coarse geometries, using a biologically-inspired mechano-transduction approach (see previous reply). We are thus moving from contact and pressure information (presented in Raspopovic et al., 2014) to texture recognition, which is a more sophisticated kind of tactile information (Saal and Bensmaia, 2015). A detailed comparison with previous works is shown in this letter in Author response image 1.
Induced tactile sensations are classified as in a recent review (Saal and Bensmaia, 2015).
The feasibility of our method is shown during experiments involving both a transradial amputee with implanted TIME neural interface and intact subjects with electrical nerve fibers stimulation via percutaneous insertion of needle microelectrodes. Such parallel experimental study of an acute (percutaneous microstimulation) and a chronic (implanted TIME) neural stimulation technique is novel in neuroscientific and neuroprosthetic research.
2) The innovative hybrid FEM-neural model of human median nerve shows needle microstimulation and TIME stimulation similar outcomes, justifying the translation of the paradigm from acute to chronic implementation. As a consequence, microneedle experiments can be used in the future as a methodology to test new stimulation patterns and innovative artificial fingers/sensors, that afterwards are translated to amputees with implantable intraneural interfaces.
3) Together with subjective perception, the quantitative analysis of EEG activity allowed to identify the brain areas mainly targeted by the different stimuli, supporting the hypothesis that the electrical stimulation was biologically plausible (i.e., similar to mechanical stimulation of the intact fingertip).
The first and main goal listed above has been achieved by changing the experimental setup described in the previous study (Raspopovic et al., 2014), as follows:
– Improving the quality of the sensors embedded in the fingertip to gather more complex tactile patterns during the tasks. This is of course a “pre-condition” to provide the complex tactile information to the user. Several recent papers (Zang et al., 2015, Tee et al., 2015) even if extremely interesting are able to provide only one-dimensional pressure information. This paper will be the first one showing in neuroprosthetics the potentials of triaxial sensors, that can resolve the normal and shear components of mechanical interaction (with a focus on the shear component, in this study) applying a computational model to achieve neuromorphic spiking transduction. This transduction mechanism is successfully used with human subjects for the first time in this paper, whereas the feasibility only in artificial touch studies had been previously shown (as necessary from scientific and ethical viewpoints to avoid exposing the human subjects to unnecessary invasive procedures). Here, we show that the human nervous system can make use of such information as machine-learning methods could with fully robotic experiments not involving humans in the loop.
– In conclusion, as shown in the “Encoding variable” column of Author response image 1, the way the stimuli encode in the pulses here is complementary to the approach followed in the previous study (Raspopovic et al., 2014). There, the delivery of current pulses to the nerve had a fixed frequency and duration within each session and was modulated in amplitude, thus temporal coding had not been implemented. In such case the number of afferent fibers recruited grew monotonically with the stimulation charge and hence the perceived force modulates accordingly, as discussed in a recent review (Saal and Bensmaia, 2015). Conversely, in the present study percepts about textural features were induced modulating the temporal structure of the spiking activity delivered through the nerve without modulating the pulses amplitude (nor the pure firing rate as shown by the analysis of Figure 10D-E in the manuscript).
– Overall, these are major hardware and computational differences with respect to our previous paper, and were introduced in this study in order to target different qualities of tactile perception, showing, for the first time, feasibility of restoration of texture recognition with “bionic” hand prostheses.
Third, the work presents tactile encoding – in particular coarse + geometric -> shape sensing. This is not clear. Is it only the artifact of their stimulus (grating) or is this the result of perception by the subjects – having tested many stimuli. i.e. is the perceptual output of the given microneurographic access to nerves and penetrating electrodes' access to the fascicles? If so, how specific and representative is it?
The goal of the study was to verify that using intraneural stimulation subjects can gather information about textural features of medium-coarse stimuli. As discussed before, the spiking activity is generated according to the type of mechanical stimuli presented and we expect generalization capability with different textures. The following points better clarify this aspect in details and discuss the changes made to the paper:
1) All the subjects were asked to qualitatively evaluate the experienced stimulation, and all could perceive the received electrical neural stimulation as similar to mechanical alternation between ridges and gaps. Such point was better clarified in the text.
2) To better clarify how we delivered the neural stimulation patterns to the subjects, we decided to change the labels of the experimented surfaces from the previous A1, A2, B1, B2, C1, C2, D1, D2 to ΔSP+/-, where ΔSP is the difference (in mm) in spatial period between the two halves of the surface and + or – indicates whether the spatial period is increasing or decreasing (with + or – for the constant stimulus having no effect, but being used because the surface was presented half times in one side and half of occurrences in the other side to avoid polarizing the experiment in case of fabrication asymmetries). Labeling of surfaces was then changed to: Δ2.5+, Δ2.0+, Δ1.0+, Δ0.0+, Δ0.0-, Δ1.0-, Δ2.0-, Δ2.5- (as shown in the revised manuscript in Figure 1b, Figure 3, Figure 9 and Figure 10).
3) Furthermore, the stimuli are now sorted with respect to such difference (with sign) between the two halves of the stimulus, so that the raster plot (Figure 1b) and confusion matrices (Figure 3a and Figure 10a) could now be more straightforward to read.
4) In Figure 1b of the manuscript, the raster plot represents the spikes that were generated while scanning the gratings on the sensorized region of the artificial fingertip, as represented by the mechano-neuro-transduction (MNT) process. Such spikes are delivered to the neural pathways to elicit artificial percepts by means of implanted TIME interface for the amputee subject or by means of the needle electrode for the intact subjects.
Fourth, the statistical analysis of the data should be improved. The text and Figure 1 discuss the results of the study in terms of the overall success rate. Figure 3 shows the breakdown of the responses by subject. There is a conspicuous lack of analysis of the data. The data should be analyzed not by lumping together all of the subjects and reporting an aggregate successful discrimination rate, but rather by individual subject as a function of stimulus type. The coarser the stimulus, the greater the discriminatory power and the weaker the stimulus the less the discriminatory power. There should be an analysis showing that the performance by individual is greater than what would be expected by chance, where chance here is 0.33. If there is an approximate monotone relation among the stimuli then a logistic regression analysis could be performed. The analysis should be conducted using confidence intervals or Bayesian methods and not just simply by reporting p-values. An advantage of a Bayesian hierarchical analysis or a random effects model across subjects is that they provide a formal way of pooling information across subjects. The latter is crucial for this problem because the authors which to establish that they have been able to provide neuromorphically tactile sensation. This statement should be made with an assessment of the accuracy of the discriminatory power of the MNT technique for each subject as a function of stimulus type. It is likely that the results will reveal that more trials will be required to show that there is truly discriminatory capability.
We carefully followed all such constructive comments made by the reviewers, as discussed below:
1) First, in the revised manuscript we computed a statistical analysis per individual subject, as a function of stimulus type, showing that the performance by individuals is greater than what would be expected by chance, conducted using confidence intervals. As shown in the new panels, added to Figure 3b (illustrated below), for all four subjects involved in the percutaneous electrical microstimulation protocol, we found that the 95% confidence interval for the fraction of correct answers (Clopper Pearson exact interval) was entirely above chance level for the surfaces characterized by 2.5 mm and 2.0 mm differences in spatial period SP between the two halves. For subject DAS implanted with TIME interface, the 95% confidence interval was largely above chance level for each single stimulus (see the new panels, added to Figure 10b, illustrated below). Significance is particularly evident in the results with the amputee subject, since the chronically-implanted interface allowed carrying out a quite high number of trials, whereas the percutaneous electrical microstimulation in intact subjects is a technique prone to interruption due to the possible sudden loss of contact between the electrode and the nerve. Hence, with acute microstimulation the number of trials could not be arbitrarily increased on a per-subject basis but still the current sets of trials provide evidence that the stimulations induced significant discriminatory capability in each subject.
2) We thank the reviewers for the constructive suggestion to perform a logistic regression analysis to investigate whether there was an approximate monotone relationship between the rate of correct response and the difference in spatial coarseness of the perceived stimuli. This analysis is shown in the new panels added to Figure 3c and Figure 10c.
The authors state that they observed no difference in the natural EEG activity and evoked stimulation in source topography, response timing and clustering. How were these assessments made? They involve negative results and require a power analysis to establish their validity.
We thank the reviewers for addressing this important issue. In the revised manuscript we now clearly specify each test applied to assess statistical significance. We also ran the requested power analysis for each comparison, to investigate the validity of the results.
More specifically, with respect to the response timing, the test used for the statistical comparison was a Montecarlo statistics with cluster correction for multiple comparisons. It is now specified in the Methods (in the section entitled EEG signal processing), as follows:
“Event-related potentials (ERPs) were time-locked to the onset of either the electrical microstimulation or of the sliding phase of the mechanical stimulation. ERPs were normalized for the standard deviation of the prestimulus (1000 ms). ERP’s statistical significance between conditions (electrical microstimulation vs. mechanical stimulation) was assessed using a Montecarlo statistics with cluster correction for multiple comparisons (triangulation and maxsum as clustered statistics)(Maris and Oostenveld, 2007), adapted from the FieldTrip toolbox (Oostenveld et al., 2011).”
As suggested by the reviewers, we computed a power analysis (GPower, Duesseldorf, Germany) on the ERPs (Figure 4—figure supplement 1) depicted in Figure 4 of the manuscript, with the following comparisons:
1) Sample size computation based on the effect size of the prestimulus and the ERP evoked by electrical microstimulation;
2) Sample size computation based on the effect size of the prestimulus and the ERP evoked by mechanical stimulation;
3) Sample size computation based on the effect size of the prestimulus recorded during the sessions with electrical microstimulation and of the prestimulus recorded during the sessions with mechanical stimulation;
4) Sample size computation based on the effect size of the ERP evoked during the sessions with electrical microstimulation and of the ERP evoked during the sessions with mechanical stimulation.
For the above conditions 1 (prestimulus vs. ERP evoked during electrical microstimulation) and 2 (prestimulus vs. ERP evoked during mechanical stimulation), with our sample size of 4 subjects the power of the test was 0.75 and 0.79, respectively, as illustrated below in the Figure 4—figure supplement 2 and Figure 4—figure supplement 3.
Conversely, the sample size required to reach a comparable power was greater than 75 subjects for both the above conditions 3 (prestimulus recorded during electrical microstimulation sessions vs. prestimulus recorded during mechanical stimulation sessions) and 4 (ERP evoked during electrical microstimulation sessions vs. ERP evoked during mechanical stimulation sessions), as illustrated below in the Figure 4—figure supplement 4 and Figure 4—figure supplement 5.
We therefore conclude that despite the small sample size, our data are sufficient to statistically highlight the activity evoked by both electrical and mechanical stimulations compared to the prestimulus baselines, and that the lack of significant differences in the ERP triggered by electrical and mechanical stimulations is as valid as it is the lack of differences between the corresponding baselines.
Although the data are more compelling for the TIME technique applied to the transradial amputee patient, a similar analysis should be performed for these data as well.
Unfortunately we do not have EEG recordings for the experiments with the amputee subject, but only those from the intact volunteers (a still completely original set of data). Furthermore, now the implanted electrodes have been removed from the stump of the amputee (though being still very well performing at the end of the experiments) as planned in the protocol approved by the ethical committee, preventing the possibility of further recordings with DAS subject.
The authors wish to establish from their simulation analyses that the response patterns under MNT and TIME are similar if not effectively the same. It is unclear where the uncertainty comes from in these analyses since they are simulation models. Therefore how can statistical assessments of uncertainty and formal statistical inferences be made? Moreover, the authors wish to infer that the stimulation responses of the two modalities do not differ. They report a p-value without a specific statement about the statistical test being used in the analysis. Because the authors wish to establish a negative result of no difference between the two stimulations modalities, they should report a power analysis stating what types and magnitudes of differences their investigations were calibrated to detect.
The present model is a realistic computational model of the human median nerve, implemented starting from the realistic anatomical picture and fiber distribution data (as illustrated in Figure 6—figure supplement 1).
The uncertainty for the two simulated devices is explained in the continuation: a total amount of 100 modeled fibers for each fascicle were placed randomly in the specific target fascicle, for several placements (Figure 8 of the manuscript, Author response image 2 and Figure 8—figure supplement 1 added to this letter). As the fiber organization within different fascicles in the nerve is unknown, we assumed that fibers within single fascicles belong to the same functionality. Moreover, since there is an inherent anatomical uncertainty in their placement and extension, for every position of stimulating electrode/needle we implemented nine populations having different extensions and centroids for the same fascicle (i.e., spanning from having the whole fascicle uniformly populated, to the case of very concentrated population where the fibers are almost touching each-other, Figure 8 of the manuscript, Figure 8—figure supplement 1). This is carried out since a-priori it is unknown whether the population of interest is concentrated in the small portion of a fascicle, or possibly uniformly distributed over it, and by simulating different populations placement we are emulating the inherent biological variability. Then the analysis has been performed for wide range of cases (for small, medium and big fascicles, with devices placed within, close, or far away from it), finally resulting in n=90 simulations for TIME and n=45 simulations cases for microneedle (this difference is due to the fact that in every position implemented for one needle tip the TIME electrode had 2 sites: right and left ones, at the same position).
The stimulations effects achieved by using microneedle and TIME interfaces (an example for one TIME active site is shown in this letter in Author response image 2) were simulated, in order to understand whether the results obtained with microneedle in intact subjects could be translated (in terms of recruited population and respective current necessary) to TIME interface in an amputee.
The model indicated that the stimulated portion of axonal population is similar for the two interfaces. In fact, we evaluated the statistical similarity between the injected charges necessary to reach the threshold of recruitment (defined at 10% of recruited fibers among the simulated population) for the two devices. The hypothesis of normality was studied for the groups resulting in non-parametric distributions. In order to compare non-parametric distributions we chose Kruskal-Wallis test (one-way analysis of variance by ranks). It is a non-parametric test, suitable for the case of populations with not equal number of samples, and we chose the threshold p-value of 0.05. Such information was now integrated in the revised manuscript.
Our statistical test is comparing the values of charge necessary to achieve a recruitment of 10% of fibers, and is tuned to take into account 5 nC as the magnitude of difference (that has been shown in the precedent stimulation studies to be the minimum range “differentiable”). For such a value, our sample of 135 overall simulations holds the power of test at 0.95 (G-Power, Dusseldorf, Germany).
The authors do not have to develop an ancillary classifier model to understand the performance characteristics of the MNT and the TIME paradigms. If they build statistical models to analyze the data from the subjects that include covariates that has IBI they will get their answer with the same model being used to analyze the data. This approach would obviate the current analyses which lead to multiple comparisons corrections.
We thank the reviewers for their comment that, jointly with the fourth point above, triggered us to complement the analysis of the psychophysical results and their correlation with electrophysiological data. As described in the reply to the fourth comment and figures therein we assessed the correlation between stimulus coarseness and the decoding performance for both microstimulation with intact subjects and implanted neural interface with transradial amputee. This is now shown in the revised version of Figure 3 and Figure 10, and discussed in the manuscript. In the manuscript we also report that a logistic fit of performance works for IBI but not for average firing rate since the latter does not correlate with coarseness.
[Editors' note: further revisions were requested prior to acceptance, as described below.]
Suggested Revisions:
1) The claim in the Abstract is still too broad: "Intraneural MNT-based stimulation restored discrimination of textural features, thus enhancing the user's tactile capabilities." The word "restore" is too strong. It implies permanent restoration/recovery. Here a texture perception has been mimicked, and performance has not been "enhanced."
We agree with the reviewers. Actually, the long-term restoration/recovery of tactile sensory skills for upper limb amputees is one of our next scientific and technological objectives. Hence, following the invitation to revise the wording, we changed “restore” to “elicit”. The changes were operated to the title, the Abstract and through all the manuscript when related to the specific findings of this work. Somewhere in the manuscript “restored” was left, particularly when referring to long-term objectives (e.g., “the restoration of sensory perception is the crucial step to achieve in the development of the next generation of artificial limbs and hand prostheses in particular”). The sentence “thus enhancing the user’s tactile capabilities” has been removed from the Abstract since it was an unnecessary comment.
2) Paragraph two, subheading “Analysis of neural coding strategies”: This information pertains to Figure 10 (e). The message that the rate (in e, worse) vs temporal code (in d, good) is getting lost in this paragraph. This is an important observation and clarification would help. The figure caption could be more explanatory too.
We thank the reviewers for triggering the revision of such paragraph in the manuscript in order to point out the message that the temporal structure of the injected spike trains can explain behavioural results, whereas a rate code could not. The revised text is reported hereafter.
“The abovementioned results show that DAS behavioral responses may be based on the temporal structure of stimulation. […]This further strengthens the hypothesis that the subject exploited the temporal structure of the response to discriminate the stimuli.”
Furthermore, the caption of Figure 10d-e was unified to induce a combined read of the two panels. We also modified the title of the two panels to made their content clearer.
3) Paragraph two, subheading “Hybrid electrical-biophysical model of the median nerve for the comparison between microstimulation needle and implanted TIME”: As the fiber organization within different fascicles in the nerve is unknown, we assumed that fibers within the fascicles belong to same functionality.
Please clarify how nociceptive fibers were separated from sensory (e.g. see the comment pertaining to the model). Indeed, this issue or not being able to separate different fiber types is quite critical and must be marked as a limitation.
We thank the reviewers for this very useful observation that triggered us to change the wording within the text accordingly. Indeed when stating that the fibers within the same fascicle belong to same functionality, we meant that they are innervating the same area (e.g. the tip of second finger), not that they are of the same type. Now this is corrected. While as regarding the nociceptive fibers, they are the ones with the smallest diameter and are therefore the last to be stimulated, as shown in previous studies (McNeal, 1976). Since in our study and model we were more interested about the recruitment at the threshold or at half-range of sensation, the nociceptive fibers stimulation is not induced. However, as suggested by reviewers, in the revised text we also introduced into the list of the limitations (called model assumptions), the one regarding the separation of the different fiber types. The related revised text is reported hereafter.
“A list of plausible assumptions had to be taken, during the model construction. […]One of the assumptions constraining the model was that the stimulation of different fiber types, such as nociceptive fibers, was not induced, at used current range.”
4) Figure 2: Traces and colors are not well explained.
The caption of Figure 2 was revised.
5) Figure 6 caption: The X marker represents the targeted fascicle where the fiber activation per 9 different populations was calculated. What does this mean (what populations?) for different locations of microneedle and TIME? This procedure is carried out analogously for medium and small fascicles, confirming results. Please could you also clarify the procedure.
This point was now better explained within the revised manuscript, in the Materials and methods section, in the caption of Figure 6, of Figure 8 and of Figure 8—figure supplement 1. Since there is an inherent uncertainty in the placement and extension of fibers innervating a specific hand district, for every position of stimulating electrode/needle we implemented 9 populations having different extensions and centroids for the same large fascicle (i.e., spanning from having the whole fascicle uniformly populated, to the case of very concentrated population where the fibers are almost touching each-other, Figure 8). Then the analysis has been performed for the range of significant fascicle sizes, defined as median size representatives of three groups: small (1 population implemented), medium (5 populations implemented) and large (9 populations). Different placements of the electrodes were simulated as well as illustrated in the figure. The captions of all figures related to this topic were changed accordingly.
6) Figure 8: outcomes. Color-coding is not explained.
Different colored dots (green, red, blue, black,…) indicate the positioning of different populations of the fibers implemented, which is emulating the unknown, possible dispersions of the positioning of fibers. This is now added to the caption of Figure 8. Figure 8—figure supplement 1 further clarifies the color-coding.
7) Figure 10: This figure, slightly modified to clarify rate vs temporal, would be much clearer (and certainly this result is important).
We thank the reviewers for pointing out this issue. The caption of Figure 10 and the related text in the manuscript were revised as discussed in the reply to point 2 above.
Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd
Citations & impact
Impact metrics
Citations of article over time
Alternative metrics
Smart citations by scite.ai
Explore citation contexts and check if this article has been
supported or disputed.
https://scite.ai/reports/10.7554/elife.09148
Article citations
Neural population dynamics reveals disruption of spinal circuits' responses to proprioceptive input during electrical stimulation of sensory afferents.
Cell Rep, 43(2):113695, 19 Jan 2024
Cited by: 0 articles | PMID: 38245870 | PMCID: PMC10962447
Biohybrid Robotic Hand to Investigate Tactile Encoding and Sensorimotor Integration.
Biomimetics (Basel), 9(2):78, 27 Jan 2024
Cited by: 0 articles | PMID: 38392124 | PMCID: PMC10886511
Neuromorphic hardware for somatosensory neuroprostheses.
Nat Commun, 15(1):556, 16 Jan 2024
Cited by: 1 article | PMID: 38228580 | PMCID: PMC10791662
Review
Free full text in Europe PMCIntegration of proprioception in upper limb prostheses through non-invasive strategies: a review.
J Neuroeng Rehabil, 20(1):118, 09 Sep 2023
Cited by: 0 articles | PMID: 37689701 | PMCID: PMC10493033
Review
Free full text in Europe PMCFrom rubber hands to neuroprosthetics: Neural correlates of embodiment.
Neurosci Biobehav Rev, 153:105351, 06 Aug 2023
Cited by: 1 article | PMID: 37544389 | PMCID: PMC10582798
Review
Free full text in Europe PMC
Go to all (108) article citations
Data
Data behind the article
This data has been text mined from the article, or deposited into data resources.
BioStudies: supplemental material and supporting data
Data Citations
- (1 citation) DOI - 10.5061/dryad.630hf
Similar Articles
To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.
Intensity Discriminability of Electrocutaneous and Intraneural Stimulation Pulse Frequency in Intact Individuals and Amputees.
Annu Int Conf IEEE Eng Med Biol Soc, 2020:3893-3896, 01 Jul 2020
Cited by: 5 articles | PMID: 33018851
Discriminability of multiple cutaneous and proprioceptive hand percepts evoked by intraneural stimulation with Utah slanted electrode arrays in human amputees.
J Neuroeng Rehabil, 18(1):12, 21 Jan 2021
Cited by: 6 articles | PMID: 33478534 | PMCID: PMC7819250
Case Studies in Neuroscience: Sensations elicited and discrimination ability from nerve cuff stimulation in an amputee over time.
J Neurophysiol, 120(1):291-295, 09 May 2018
Cited by: 11 articles | PMID: 29742031
Fascicle-Specific Targeting of Longitudinal Intrafascicular Electrodes for Motor and Sensory Restoration in Upper-Limb Amputees.
Hand Clin, 37(3):401-414, 01 Aug 2021
Cited by: 1 article | PMID: 34253313
Review
Strategies for providing upper extremity amputees with tactile and hand position feedback--moving closer to the bionic arm.
Technol Health Care, 7(6):401-409, 01 Jan 1999
Cited by: 37 articles | PMID: 10665673
Review
Funding
Funders who supported this work.
Directorate-General for Communications Networks, Content and Technology (3)
Grant ID: EU Grant FP7-NMP 228844 NANOBIOTOUCH project
Grant ID: EU Grant FET 611687 NEBIAS Project
Grant ID: EU Grant CP-FP-INFSO 224012 TIME project
Italian National Institute for Insturance against Industrial Injuries (1)
Grant ID: National project PPR2 (Control of hand prosthesis by invasive neural interfaces)
Ministero della Salute (1)
Grant ID: Italian NEMESIS (Neurocontrolled mechatronic hand prosthesis)
Ministero dell’Istruzione, dell’Università e della Ricerca (1)
Grant ID: Italian project PRIN/HandBot
Swiss National Competence Center in Research in Robotics (1)
Grant ID: NCCR Robotics
Wyss Center for Bio and Neuroengineering (1)
Grant ID: ENABLE Project