Frontiers in Neurorobotics
ISSN / EISSN : 1662-5218 / 1662-5218
Published by: Frontiers Media SA (10.3389)
Total articles ≅ 699
Latest articles in this journal
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.785808
With the continuous development of deep-learning technology, ever more advanced face-swapping methods are being proposed. Recently, face-swapping methods based on generative adversarial networks (GANs) have realized many-to-many face exchanges with few samples, which advances the development of this field. However, the images generated by previous GAN-based methods often show instability. The fundamental reason is that the GAN in these frameworks is difficult to converge to the distribution of face space in training completely. To solve this problem, we propose a novel face-swapping method based on pretrained StyleGAN generator with a stronger ability of high-quality face image generation. The critical issue is how to control StyleGAN to generate swapped images accurately. We design the control strategy of the generator based on the idea of encoding and decoding and propose an encoder called ShapeEditor to complete this task. ShapeEditor is a two-step encoder used to generate a set of coding vectors that integrate the identity and attribute of the input faces. In the first step, we extract the identity vector of the source image and the attribute vector of the target image; in the second step, we map the concatenation of the identity vector and attribute vector onto the potential internal space of StyleGAN. Extensive experiments on the test dataset show that the results of the proposed method are not only superior in clarity and authenticity than other state-of-the-art methods but also sufficiently integrate identity and attribute.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.788494
This paper investigates the EEG spectral feature modulations associated with fatigue induced by robot-mediated upper limb gross and fine motor interactions. Twenty healthy participants were randomly assigned to perform a gross motor interaction with HapticMASTER or a fine motor interaction with SCRIPT passive orthosis for 20 min or until volitional fatigue. Relative and ratio band power measures were estimated from the EEG data recorded before and after the robot-mediated interactions. Paired-samples t-tests found a significant increase in the relative alpha band power and a significant decrease in the relative delta band power due to the fatigue induced by the robot-mediated gross and fine motor interactions. The gross motor task also significantly increased the (θ + α)/β and α/β ratio band power measures, whereas the fine motor task increased the relative theta band power. Furthermore, the robot-mediated gross movements mostly changed the EEG activity around the central and parietal brain regions, whereas the fine movements mostly changed the EEG activity around the frontopolar and central brain regions. The subjective ratings suggest that the gross motor task may have induced physical fatigue, whereas the fine motor task may have induced mental fatigue. Therefore, findings affirm that changes to localised brain activity patterns indicate fatigue developed from the robot-mediated interactions. It can also be concluded that the regional differences in the prominent EEG spectral features are most likely due to the differences in the nature of the task (fine/gross motor and distal/proximal upper limb) that may have differently altered an individual's physical and mental fatigue level. The findings could potentially be used in future to detect and moderate fatigue during robot-mediated post-stroke therapies.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.734130
Background: Appropriate training modalities for post-stroke upper-limb rehabilitation are key features for effective recovery after the acute event. This study presents a cooperative control framework that promotes compliant motion and implements a variety of high-level rehabilitation modalities with a unified low-level explicit impedance control law. The core idea is that we can change the haptic behavior perceived by a human when interacting with the rehabilitation robot by tuning three impedance control parameters. Methods: The presented control law is based on an impedance controller with direct torque measurement, provided with positive-feedback compensation terms for disturbances rejection and gravity compensation. We developed an elbow flexion-extension experimental setup as a platform to validate the performance of the proposed controller to promote the desired high-level behavior. The controller was first characterized through experimental trials regarding joint transparency, torque, and impedance tracking accuracy. Then, to validate if the controller could effectively render different physical human-robot interaction according to the selected rehabilitation modalities, we conducted tests on 14 healthy volunteers and measured their muscular voluntary effort through surface electromyography (sEMG). The experiments consisted of one degree-of-freedom elbow flexion/extension movements, executed under six high-level modalities, characterized by different levels of (i) corrective assistance, (ii) weight counterbalance assistance, and (iii) resistance. Results: The unified controller demonstrated suitability to promote good transparency and render both compliant and stiff behavior at the joint. We demonstrated through electromyographic monitoring that a proper combination of stiffness, damping, and weight assistance could induce different user participation levels, render different physical human-robot interaction, and potentially promote different rehabilitation training modalities. Conclusion: We proved that the proposed control framework could render a wide variety of physical human-robot interaction, helping the user to accomplish the task while exploiting physiological muscular activation patterns. The reported results confirmed that the control scheme could induce different levels of the subject's participation, potentially applicable to the clinical practice to adapt the rehabilitation treatment to the subject's progress. Further investigation is needed to validate the presented approach to neurological patients.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.735177
There have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, large deformations, occlusions, low illumination, and inadequate focus occur. In addition, it is difficult and costly to obtain a large and accurate dataset on operational video anatomical structures, including arteries. In this study, we investigated cerebral artery segmentation using an automatic ground-truth generation method. Indocyanine green (ICG) fluorescence intraoperative cerebral videoangiography was used to create a ground-truth dataset mainly for cerebral arteries and partly for cerebral blood vessels, including veins. Four different neural network models were trained using the dataset and compared. Before augmentation, 35,975 training images and 11,266 validation images were used. After augmentation, 260,499 training and 90,129 validation images were used. A Dice score of 79% for cerebral artery segmentation was achieved using the DeepLabv3+ model trained using an automatically generated dataset. Strict validation in different patient groups was conducted. Arteries were also discerned from the veins using the ICG videoangiography phase. We achieved fair accuracy, which demonstrated the appropriateness of the methodology. This study proved the feasibility of operating field view of the cerebral artery segmentation using deep learning, and the effectiveness of the automatic blood vessel ground truth generation method using ICG fluorescence videoangiography. Using this method, computer vision can discern blood vessels and arteries from veins in a neurosurgical microscope field of view. Thus, this technique is essential for neurosurgical field vessel anatomy-based navigation. In addition, surgical assistance, safety, and autonomous surgery neurorobotics that can detect or manipulate cerebral vessels would require computer vision to identify blood vessels and arteries.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.831113
Editorial on the Research TopicNeuromechanical Biomarkers in Robot-Assisted Motor Rehabilitation Clinical evaluation of motor function is essential for tracking the evolution of patient's abilities during rehabilitation. A regular and accurate observation of neuromotor recovery allows therapists to adjust intensity, number of repetitions, or targeted motor activity during the treatment, in particular when delivered using advanced technological means. However, conventional evaluation methods are usually based on qualitative clinical metrics with low resolution (Hsieh et al., 2009) and, commonly, the correct interpretation of the relevant scales depends on the experience of the attending therapist. Consequently, the assessment of functional recovery suffers the risk of being incomplete or inaccurate. This editorial aimed to prompt and gather a collection of novel research efforts with the common goal of identifying relevant or promising neuromechanical biomarkers of neuromotor functions during robot-assisted rehabilitation in clinical settings. Indeed, the latest rehabilitation technologies allow, through the combination of sensors and robots, to measure the patient's kinematic and/or kinetic movement parameters with high precision (Bosecker et al., 2010; Colombo et al., 2015; Keller et al., 2015). These parameters can be used to assess motor learning or quantify improvements in targeted motor functions. Furthermore, the same technologies allow to investigate how the neuromuscular system is behaving from an electrophysiological point of view. As reviewed in detail by Garro et al. current research is mostly based on surface electromyography (EMG) and electroencephalography (EEG). Some studies already correlate these metrics with conventional clinical scales (Tang et al., 2018; Zhang et al., 2019), but their application to the optimization of robot-assisted rehabilitation has not yet been systematically explored. In this context, several examples of clinical estimation of neuromechanical biomarkers are explored in this collection. The aforementioned review by Garro et al. analyzes a number of non-invasive electrophysiological approaches to the computation of biomarkers from EEG and EMG recordings, particularly focused on stroke and robot-assisted rehabilitation. In Ye et al. data-driven models based on backpropagation neural networks (BPNN) are built from EMG data collected from chronic stroke individuals and correlated with the Fugl-Meyer Assessment scale (FMA) and the Modified Ashworth Scale (MAS). Ezaki et al. evaluate joint angles and muscle activity during gait before and after intervention with a HAL exoskeleton, reporting changes in acute and chronic patients with Ossification of the Posterior Longitudinal Ligament (OPLL) caused by myelopathy. In Reyes et al. variations in the Metabolic Equivalent of Task (METs) are found under different conditions of friction during walking activities using a Motor Assisted Hybrid Neuroprosthesis (MAHNP). Finally, in Longatelli et al. functional gait is assessed in terms of neuromuscular behavior during exoskeleton training, showing that patients treated with the robotic device regained a controlled rhythmic neuromuscular pattern in the proximal lower limb muscles. The presented articles in this editorial give insights into the estimation of neuromechanical biomarkers in clinical scenarios in a non-invasive way and with robustness similar or higher to that of conventional clinical scales. These studies illustrate some of the future directions in this field arguably indicating the trend of neuromechanical assessment in clinical motor rehabilitation. AÚ drafted the first version of the manuscript. All authors contributed to the critical discussion and revision of its contents. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher. Bosecker, C., Dipietro, L., Volpe, B., and Krebs, H. I (2010). Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabil. Neural Repair. 24, 62–69. doi: 10.1177/1545968309343214 PubMed Abstract | CrossRef Full Text | Google Scholar Colombo, R., Pisano, F., Micera, S., Mazzone, A., Delconte, C., Carrozza, M. C., et al. (2015). Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. IEEE Trans. Neural Syst. Rehabil. Eng. 13, 311–324. doi: 10.1109/TNSRE.2005.848352 PubMed Abstract | CrossRef Full Text | Google Scholar Hsieh, Y. W., Wu, C. Y., Lin, K. C., Chang, Y. F., Chen, C. L., and Liu, J. S (2009). Responsiveness and validity of three outcome measures of motor function after stroke rehabilitation. Stroke 40, 1386–1391. doi: 10.1161/STROKEAHA.108.530584 PubMed Abstract | CrossRef Full Text | Google Scholar Keller, U., Schölch, S., Albisser, U., Rudhe, C., Curt, A., Riener, R., et al. (2015). Robot-assisted arm assessments in spinal cord injured patients: a consideration of concept study. PLoS ONE 10:e0126948. doi: 10.1371/journal.pone.0126948 PubMed Abstract | CrossRef Full Text | Google Scholar Tang, W., Zhang, X., Tang, X., Cao, S., Gao, X., and Chen, X (2018). Surface electromyographic examination of poststroke neuromuscular changes in proximal and distal muscles using clustering index analysis. Front. Neurol. 8:731. doi: 10.3389/fneur.2017.00731 PubMed Abstract | CrossRef Full Text | Google Scholar Zhang, X., Tang, X., Zhu, X., Gao, X., and...
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.824592
Facial action unit (AU) detection is an important task in affective computing and has attracted extensive attention in the field of computer vision and artificial intelligence. Previous studies for AU detection usually encode complex regional feature representations with manually defined facial landmarks and learn to model the relationships among AUs via graph neural network. Albeit some progress has been achieved, it is still tedious for existing methods to capture the exclusive and concurrent relationships among different combinations of the facial AUs. To circumvent this issue, we proposed a new progressive multi-scale vision transformer (PMVT) to capture the complex relationships among different AUs for the wide range of expressions in a data-driven fashion. PMVT is based on the multi-scale self-attention mechanism that can flexibly attend to a sequence of image patches to encode the critical cues for AUs. Compared with previous AU detection methods, the benefits of PMVT are 2-fold: (i) PMVT does not rely on manually defined facial landmarks to extract the regional representations, and (ii) PMVT is capable of encoding facial regions with adaptive receptive fields, thus facilitating representation of different AU flexibly. Experimental results show that PMVT improves the AU detection accuracy on the popular BP4D and DISFA datasets. Compared with other state-of-the-art AU detection methods, PMVT obtains consistent improvements. Visualization results show PMVT automatically perceives the discriminative facial regions for robust AU detection.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.790060
User customization of a lower-limb powered Prosthesis controller remains a challenge to this date. Controllers adopting impedance control strategies mandate tedious tuning for every joint, terrain condition, and user. Moreover, no relationship is known to exist between the joint control parameters and the slope condition. We present a control framework composed of impedance control and trajectory tracking, with the transitioning between the two strategies facilitated by Bezier curves. The impedance (stiffness and damping) functions vary as polynomials during the stance phase for both the knee and ankle. These functions were derived through least squares optimization with healthy human sloped walking data. The functions derived for each slope condition were simplified using principal component analysis. The weights of the resulting basis functions were found to obey monotonic trends within upslope and downslope walking, proving the existence of a relationship between the joint parameter functions and the slope angle. Using these trends, one can now design a controller for any given slope angle. Amputee and able-bodied walking trials with a powered transfemoral prosthesis revealed the controller to generate a healthy human gait. The observed kinematic and kinetic trends with the slope angle were similar to those found in healthy walking.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.797147
Lower limb robotic exoskeletons have shown the capability to enhance human locomotion for healthy individuals or to assist motion rehabilitation and daily activities for patients. Recent advances in human-in-the-loop optimization that allowed for assistance customization have demonstrated great potential for performance improvement of exoskeletons. In the optimization process, subjects need to experience multiple types of assistance patterns, thus, leading to a long evaluation time. Besides, some patterns may be uncomfortable for the wearers, thereby resulting in unpleasant optimization experiences and inaccurate outcomes. In this study, we investigated the effectiveness of a series of ankle exoskeleton assistance patterns on improving walking economy prior to optimization. We conducted experiments to systematically evaluate the wearers' biomechanical and physiological responses to different assistance patterns on a lightweight cable-driven ankle exoskeleton during walking. We designed nine patterns in the optimization parameters range which varied peak torque magnitude and peak torque timing independently. Results showed that metabolic cost of walking was reduced by 17.1 ± 7.6% under one assistance pattern. Meanwhile, soleus (SOL) muscle activity was reduced by 40.9 ± 19.8% with that pattern. Exoskeleton assistance changed maximum ankle dorsiflexion and plantarflexion angle and reduced biological ankle moment. Assistance pattern with 48% peak torque timing and 0.75 N·m·kg−1 peak torque magnitude was effective in improving walking economy and can be selected as an initial pattern in the optimization procedure. Our results provided a preliminary understanding of how humans respond to different assistances and can be used to guide the initial assistance pattern selection in the optimization.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.801956
SLAM (Simultaneous Localization And Mapping) plays a vital role in navigation tasks of AUV (Autonomous Underwater Vehicle). However, due to a vast amount of image sonar data and some acoustic equipment's inherent high latency, it is a considerable challenge to implement real-time underwater SLAM on a small AUV. This paper presents a filter based methodology for SLAM algorithms in underwater environments. First, a multi-beam forward looking sonar (MFLS) is utilized to extract environmental features. The acquired sonar image is then converted to sparse point cloud format through threshold segmentation and distance-constrained filtering to solve the calculation explosion issue caused by a large amount of original data. Second, based on the proposed method, the DVL, IMU, and sonar data are fused, the Rao-Blackwellized particle filter (RBPF)-based SLAM method is used to estimate AUV pose and generate an occupancy grid map. To verify the proposed algorithm, the underwater vehicle is equipped as an experimental platform to conduct field tasks in both the experimental pool and wild lake, respectively. Experiments illustrate that the proposed approach achieves better performance in both state estimation and suppressing divergence.
Frontiers in Neurorobotics, Volume 15; https://doi.org/10.3389/fnbot.2021.783809
This paper explores the realization of a predefined-time synchronization problem for coupled memristive neural networks with multi-links (MCMNN) via nonlinear control. Several effective conditions are obtained to achieve the predefined-time synchronization of MCMNN based on the controller and Lyapunov function. Moreover, the settling time can be tunable based on a parameter designed by the controller, which is more flexible than fixed-time synchronization. Then based on the predefined-time stability criterion and the tunable settling time, we propose a secure communication scheme. This scheme can determine security of communication in the aspect of encrypting the plaintext signal with the participation of multi-links topology and coupled form. Meanwhile, the plaintext signals can be recovered well according to the given new predefined-time stability theorem. Finally, numerical simulations are given to verify the effectiveness of the obtained theoretical results and the feasibility of the secure communication scheme.