Nara Kasugano International Forum IRAKA
(Venue for rtFIN2017, Nov. 29- Dec. 1)
Advanced Telecommunications Research Institute International (ATR)
(Venue for Satellite Symposium and Technical Tour in ATR, Nov 28)
The human brain is a nonlinear dynamical system, which exhibits interesting dynamical features such as oscillations, synchrony, and noise-induced dynamics in brain activity. Here, I report our recent findings that resting-state and perturbation-induced brain dynamics measured by EEG account for individual-level differences in brain functions.
First, I show that the resting-state large-scale synchrony is associated with individual differences in functional recovery after stroke. We found that the degree of averaged phase synchrony of EEG, as well as the degree of fluctuations in moment-to-moment phase synchrony, was correlated with ADL-related clinical scores in stroke patients. The results suggest that the dynamic repertoire of spontaneous large-scale phase synchrony networks mediates functional networking associated with ADL recovery.
Next, I demonstrate our experimental evidence that noise-induced macroscopic human brain responses exhibit highly consistent temporal patterns to an identical noisy visual input within subjects. The results indicate that visual noise can harness EEG dynamics which can serve as a robust dynamical individual trait.
I also provide experimental results that a novel TMS-EEG co-registration technique can dissect individual differences in apparent motion perception, which is shaped by large-scale neural dynamics. By assessing the interhemispheric propagation of TMS-induced phase resetting of spontaneous oscillations in a resting condition, we succeeded in tracking individual differences in biases of apparent motion perception, which is associated with the capacity to integrate information across hemispheres. Taken together, our novel perturbational approaches are useful for the dissection of individual-level variations in nonlinear brain dynamics which causally mediate brain functions such as perception and cognition.
What methodology or tool would you like to introduce?
The exponentially growing computing needs of scientific research have increasingly been met with centralized compute clusters rather than local workstations, with large systems now housed within most academic institutions and available via commercial cloud services. We propose to review the state-of-the-art in scalable, high-performance computing and to discuss the promise this approach holds for neurofeedback research. Specifically, we will present case studies of how advanced neuroimaging analysis methods (e.g., multivariate, connectivity, etc.) can be implemented on clusters and the cloud, and what is involved in deploying these systems in real-time. We will introduce these methods via an open-source, Python-based software package that we have been developing with Intel Labs, called BrainIAK (https://github.com/IntelPNI/brainiak).
Why is this tool important for neurofeedback research?
The need to process functional imaging data in real-time for timely neurofeedback imposes constraints and requires simplifications in preprocessing and analysis. The promise of high-performance computing is that such compromises can be avoided because of the scale and speed of resources available. There are many benefits of this for neurofeedback research: First, sophisticated analyses that currently can only be run offline could be conducted in real-time, because of massive parallelization. Second, current real-time analyses could be accelerated via algorithmic optimization, providing faster neurofeedback. Third, parallelization could be exploited at the level of entire analyses, allowing multiple analyses of the same dataset to run at the same time, such as from different brain regions, using multiple models, sweeping parameter spaces, etc. Fourth, enabling real-time functional imaging analysis in the cloud will increase the accessibility of this technique, as any imaging facility with an internet connection could conduct cutting-edge studies without needing expensive local infrastructure or expertise. In sum, high-performance computing could be important for neurofeedback research by increasing the richness, power, and accessibility of real-time analyses.
What imaging modalities does this apply to?
The analytical techniques we will discuss related to high-performance computing, cloud services, and BrainIAK can be applied in principle to data from any imaging modality. • Is this a commercially available product, potentially commercially available, or open source? BrainIAK is open source, and more generally, all project- and workshop-related code and data will be shared publicly.
Speakers:
1. Nicholas Turk-Browne (Yale University)
2. Ted Willke (Intel Labs)
3. Kai Li (Princeton University)
4. Mihai Capota (Intel Labs)
Here we present a systematic review synthesizing information from 79 primary research fMRI-nf experiments—the bulk of currently available data. The vast majority of fMRI-nf findings suggest that self-regulation of specific brain signatures seems viable; however, replication of concomitant behavioral outcomes remains sparse. This state of affairs suggests that the interaction between neural self-regulation and behavioral control remains unclear.
To better understand the value of fMRI-nf as a clinical tool we would need to disentangle placebo influences and establish the specific effects of neurofeedback by using double-blind sham-controlled studies alongside rigorous and standardized statistical analyses. Of the 79 experiments we investigated, 32 (41%) used no control group and 19 (24%) used a control condition that precluded blinding and may have differed substantially from the primary treatment intervention in terms of expectation and motivation. Moreover, only 30 (38%) of the studies explicitly report accounting for respiration—an artifact that neurofeedback participants could leverage, even unconsciously, to modulate the BOLD signal.
Cortical plasticity of the sensorimotor cortex is attributed to intractable chronic pain after amputation or deafferentation of a limb [1]. However, direct evidence of a causal role of changes in the cortex is lacking. We have developed a robotic hand controlled by brain-machine interface (BMI) based on real-time magnetoencephalography signals to induce plastic changes in the sensorimotor representation of phantom hand movements in accordance with pain alteration [2]. However, the phantom pain might be affected by the prosthesis use or awareness to the phantom hand during the training. Here, we developed a novel BMI training to induce plastic changes in the motor cortex without any visual images associated with the hand and with instructions to not move the phantom hand, which minimized the patient’s awareness of the phantom hand.
This study included 14 phantom limb patients (12 brachial plexus root avulsions, 2 amputees). MEG signals were recorded while the patient grasped or opened the phantom hand. Cortical currents were estimated on the bilateral motor cortex. The decoder was trained with estimated currents contralateral or ipsilateral to the phantom hand to infer phantom hand movements. For each trained decoder, the probability of opening movement was evaluated for each 200 ms and presented to the patients as the size of disk. Patients controlled the size by thinking something without moving their body or phantom hands. Training effects were compared among the two decoders using a randomized cross-over trial.
Tobacco abuse is the leading preventable cause of disease and death in the world. Although the majority of smokers would like to quit and pharmacotherapy and behavioral therapy exists, the effectiveness still remains low. As a psychophysiological therapy, previous EEG-based neurofeedback protocols including alpha training, alpha/theta training and SMR/beta training have been employed in drug addiction for four decades. The majority of these protocols focused on helping participants to facilitate relaxation and reduce anxiety [1]. However, the clinical efficacy of EEG-based neurofeedback on drug addiction still remains dubious according to the Association for Applied Psychophysiology and Biofeedback and the International Society for Neurofeedback and Research [2]. One potential reason is that the regulated relation and anxiety are not specific to the core of addiction. On the other hand, smoking cue-reactivity is a core factor of quitting smoking and reducing brain reactivity to smoking-related cues has the potential to improve smoking cessation outcomes [3]. Based on these considerations, in the current study we developed an EEG neurofeedback method, by which brain activity patterns corresponding to smoking cue-reactivity were repeatedly down-regulated.
Balance impairment in the patients with brain injury severely affects their activities of daily living (ADL), and previous studies emphasize the vital role of the dorsomedial motor cortex including the supplementary motor area (SMA) for postural control in human, and suggested the SMA involvement in the gait and balance recovery after stroke[1]. Consistently, we hypothesized that SMA facilitation using functional near infrared spectroscopy mediated neurofeedback system (fNIRS-NFB) could augment the gait and balance recovery after stroke.
To investigate the clinical efficacy of the fNIRS-Neurofeedback (NFB) [2] on post-stroke balance recovery by a double-blinded RCT.
Subacute 43 subcortical stroke patients (59.8± 11.3 y.o., 115.0 days from onset) participated. Besides the usual rehabilitation up to 180min/day, they participated 6 sessions of motor imagery training of balance task concurrent with SMA-neurofeedback. Clinical measures including Berg Balance Scale (BBS) and 3m-Timed Up-and-go test (TUG) are assessed. Subjects are randomly assigned to REAL and SHAM groups. Resting-state functional MRIs (rsfMRI) were also assessed before and after intervention.
Noninvasive brain computer interfaces (BCIs) have been used to promote motor recovery in individuals after stroke with varying success1. These BCIs typically target brain activity in motor-related brain regions, and participants use strategies such as motor imagery to control them. However, neurofeedback is often provided on a screen in the form of a thermometer or other metric, which can be distracting during motor imagery and may lead to decreased learning.
To address this, we created a neurofeedback system that engages principles of action observation and implements them in head-mounted virtual reality (VR). Previous research has shown that action observation can increase brain activity in damaged motor regions2. In addition, studies with immersive VR have shown that embodiment of one’s virtual avatar can also increase sensorimotor neural responses3. We thus hypothesized that neurofeedback presented in the form of an embodied first-person avatar in VR should increase motor-related brain activity and thus improve performance on the BCI compared to on a screen.
Recent studies have indicated that electroencephalogram (EEG) neurofeedback (NFB) based on alpha-rhythm desynchronization may be a promising approach for normalizing resting-state activity in posttraumatic stress disorder (PTSD) associated with childhood abuse (Kluetsch et al., 2014; Ros et al., 2016) . However, these studies were designed to probe only single-session (i.e. short-term) neuromodulatory effects of NFB. It is also unclear whether abnormal resting-state EEG signatures are causally involved in the pathophysiology of PTSD. Here, we address these aspects by examining longer-term plasticity mechanisms and treatment outcomes, following administration of multiple sessions of EEG-based NFB in PTSD associated with childhood abuse.
The use of real-time fMRI for neurofeedback purposes has developed rapidly in recent years, and researchers are interested in more complex feedback forms that go beyond simple up/down modulation of single regions of interest (ROIs). An important aspect of this development has been a shift to network neurofeedback, in which feedback is given on connectivity between regions, rather than on their activations. However, instantaneous connectivity estimates are difficult to obtain in real time given the limited temporal resolution of fMRI acquisition, the sluggishness of the hemodynamic response, and the need for a minimum number of samples to compute stable correlation values. One proposed solution has been to calculate the correlations over blocks of time, and give intermittent feedback at the end of each block. However, this severely limits the number of feedback events that can be calculated for each run, diminishing the opportunities for learning.
Here we propose an alternative method using a proxy for Pearson’s correlations, which can be calculated over just two successive time points by examining the trend of the signal between these points. This proxy metric can be used to provide continuous feedback on a measure closely approximating connectivity between regions (see results to evaluate this claim).
Functional magnetic resonance imaging (fMRI) studies suggest that high-level cognitive tasks recruit a combination of spatially overlapping yet distinct frontoparietal networks [1–3]. However, understanding the exact functional role of these networks remains a challenge as the same network can be activated by inherently different tasks, e.g. [4]. The cost and difficulty of data acquisition with fMRI necessitates testing only a small subset of possible tasks. This is problematic as it can lead to misleadingly narrow functions being ascribed to a network that in reality has a broader role [2,5,6]. In the context of this many-to-many-problem, fully understanding the functional role of brain networks requires a more holistic approach that considers how brain activity changes in various task contexts [7]. While meta-analyses provide answers related to broad cognitive domains, they cannot extract information about finer-grained states [8]. Here we present a neuroadaptive closed-loop framework that combines real-time fMRI and Bayesian optimization to efficiently search across a many different cognitive tasks with the aim to optimally segregate two important frontoparietal networks. The results of this analysis were subsequently fed into another stage of optimization, to fine-tune the parameters of the optimal tasks, in order to gain further insights into the functional roles of these networks.
Decoded neurofeedback involves the online reinforcement of brain activity using multivoxel decoders trained to classify specific brain representations [1]. We hypothesized that, using a form of functional alignment known as Hyperalignment [2], fMRI data from multiple participants could be used to train such decoders. This method could potentially be used to build decoders with higher levels of accuracy that could be translated to new participants. Such Hyperalignment Decoders would facilitate novel applications, such as conducting fear reduction interventions in decoded neurofeedback [3] without having to present participants with feared objects that are highly aversive. Therefore, if the decoders of commonly feared animals could be built on one group of participants and then translated to new participants (i.e., Hyperalignment Decoders), it might be possible to conduct fully unconscious and nonaversive fear reduction interventions. To this end, we utilized Hyperalignment [2] and showed that Hyperalignment Decoders of 40 different animals and objects can be constructed on one group of participants (e.g., 29 individuals) and still yield high levels of accuracy (~ 82%) when tested on new participants (Fig. 1a). We then conducted a 5-day decoded neurofeedback experiment and showed that pairing a reward with the activation likelihood of the Hyperalignment Decoder of a feared animal (Target condition) decreased the physiological reactivity (i.e., amygdala reactivity in Fig. 1b and skin conductance reactivity in Fig. 1c) to the feared animal. Taken together, these results suggest that Hyperalignment Decoders could potentially be used to conduct decoded neurofeedback interventions, yielding a promising new technique with immediate clinical applications
White matter plasticity provides a hitherto overlooked route by which experience can shape brain structure and change behaviour [1]. Increasing neuronal activity via stimulation in animals can evoke increases in myelination [2], though effects of reducing activity remain untested. It is also currently unknown whether direct modulation of brain activity in humans can alter white matter. Here, we exploited an endogenous capacity to modulate activity using neurofeedback, to test the hypothesis that increases or decreases in sensorimotor activity produce rapid and directional effects on white matter structure in the human brain.
Realtime fMRI neurofeedback faces challenges as many participants are unable to selfregulate. The causes of this nonresponder effect are not well understood due to the cost and complexity of such investigation in the MRI scanner. Here, we investigated the temporal dynamics of the hemodynamic response measured by fMRI as a potential cause of the nonresponder effect. Learning to selfregulate the hemodynamic response involves a difficult temporal creditassignment problem because this signal is both delayed and blurred over time. Two factors critical to this problem are the prescribed selfregulation strategy (cognitive or automatic) and feedback timing (continuous or intermittent). Here, we sought to evaluate how these factors interact with the temporal dynamics of fMRI without using the MRI canner. We developed a simulated brain roughly reflecting the characteristics of early visual cortex (V1) and systematically tested four different temporal models. Simulated neural activity was convolved with the canonical hemodynamic response function (HRF) or one of three other linear filters: an impulse, a delay (6second), or a blur (10second moving average). We first used this model to examine the role of cognitive strategies by having participants learn to regulate the simulated neurofeedback signal using a unidimensional strategy: pressing one of two buttons to rotate a visual grating that stimulated our model of visual cortex. Under these conditions, continuous feedback led to faster regulation compared to intermittent feedback. Furthermore, performance within continuous feedback was modulated by the chosen temporal model: the greater the information delay, the longer it took for participants to regulate the signal (Figure 1, left). Yet, since many neurofeedback studies prescribe implicit selfregulation strategies, we created a computational model of automatic rewardbased learning to examine whether this result held true for automatic processing. When feedback was delayed and blurred based on the hemodynamics of fMRI, this model learned more reliably from intermittent feedback compared to continuous feedback (Figure 1, right). These results suggest that different selfregulation mechanisms prefer different feedback timings, and that these factors can be effectively explored and optimized via simulation prior to deployment in the MRI scanner.
One major form of learning is operant conditioning, in which the frequency of voluntary behavior increases or decreases after reward or punishment. Half a century ago, Fetz (1969) demonstrated a modified form of the operant conditioning; that is, by the use of neurofeedback training, motor cortical activity in monkeys was operantly reinforced in a given single neuron. More recently, Cerf et al. (2010) reported that humans can voluntarily control the firing rate of a specific neuron in the medial temporal lobe. However, no studies have addressed the synaptic dynamics that mediates the control of a single neuron’s firing. In the present work, we used in vivo patch-clamp techniques in the mouse hippocampus and an internal neural process-based paradigm with direct electrical stimulation of the reward system, which did not require an external reward, an explicit cognitive task, or a specific pretraining for the operant conditioning. Using reward stimulation through a real-time neurofeedback system (Fig. 1), we attempted to reinforce the event frequencies of the predefined patterns of spontaneous EPSCs (sEPSCs), spontaneous IPSC (sIPSCs), or spontaneous unit activity. In addition, we examined how this neural reinforcement learning was modulated by anesthesia or animal’s state. As a result, we demonstrated that mice could rapidly learn to receive more reward stimuli by increasing or decreasing the frequency of the predefined activity patterns in a single hippocampal CA1 pyramidal cell. This reinforcement learning did not occur in less motivated animals.
I personally appreciate the 250 attendees, 183 presenters, 14 exhibitors, and 8 sponsors who will be participating in rtFIN2017 in Nara, the oldest stable capital in Japan. rtFIN2017, the third international conference on neurofeedback, is the first one with a formalized program committee and the executive committee which raised funds from Europe, USA and Asia.
Scientifically, I will introduce 5-years efforts in the IMPACT program and the Japanese consortium AMED-DecNef for clinical and basic studies for big data-driven neurofeedback. As represented by deep neural networks, recent “AI” successes heavily depend on such big training data as tens of millions of images, tens of millions of GO games, and hundreds of millions of speeches in supervised learning. XNef is a common name for decoded neurofeedback (DecNef) and functional connectivity neurofeedback (FCNef). A target brain state for XNef can be defined from big data. In our recent studies, we used 108,000 multi-voxel patterns induced by images for DecNef (30 participants x 40 objects x 90 exemplars; Taschereau-Dumouchel V. Talk4, Special topics4; Peters MAK. Poster69), and 2,200 patient and control samples of resting state functional connectivity MRI data for FCNef (Posters 3, 4, 8, 11, 12, 13, 20, 24, 25, 32, 41, 64, and 108). We expect the analysis of big data to provide reliable neurofeedback targets for brain dynamics.
Main Question:
How can we define successful neurofeedback learning and how can we measure it?
This session will focus on a major issue in neurofeedback (NF) research; the neural measure of learning effect. A matter not yet decided both at the group and individual levels, leading to inconsistencies between studies and difficulties in establishing a standard for comparison. The significance of this topic is related to the well known fact that NF learning progression is different among individuals, leading to a need to better characterize and predict who is most likely to learn and benefit from an intervention. However, it is yet unclear when and why to use various possible indices of learning such as; the mean signal difference between conditions, the progression along different trials or change in resting state signal fluctuations. More so, it is debatable whether there is one correct way or perhaps we should combine different measurements.
In the proposed session we will discuss different methods of measuring NF learning and point to advantages and disadvantages of each method. Our goal is to open a discussion that may lead towards a consensus on this matter, thus enhancing research reliability and comparability and through that improving standard of care.
Speakers:
1. Mr. Noam Goldway, MA (Tel-Aviv Sourasky Medical Center, Affiliated to Tel-Aviv University. and Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel)
2. Prof. David Linden, MD, PhD (Faculty of Psychology and Neuroscience, Maastricht University, The Netherlands. Netherlands Institute for Neuroscience, Amsterdam, The Netherlands)
3. Leon Skottnik (Maastricht University, The Netherlands)
4. Ayumu Yamashita (Advanced Telecommunications Research Institute (ATR), Department of Cognitive Neuroscience, Kyoto, Japan)
What methodology or tool would you like to introduce?
In this workshop we would like to present various machine learning and signal processing tools that we and others developed to process EEG signals in real-time, and that could be used to substantially improve current standard EEG-based NeuroFeedback (NF) training processes. The tools we will present were mostly designed to build EEG-based Brain-Computer Interfaces (BCIs), i.e., systems that can decode users’ mental states in real-time from their EEG signals. As such, their signal processing tools promise to be very useful for EEG-NF research and practice as well. We notably plan to present various adaptive EEG spatial and temporal filters, which would enable NF practionners to find robust subject-specific NF features, in individually, and functionally relevant brain areas and frequency bands. Such tools notably include spatio-spectral filters such as inverse solutions (e.g., LORETA), common spatial patterns (CSP) and variants (RCSP, FBCSP, etc), Independent Component Analysis (ICA) or Source Power Comodulation (SPoC). We will also present briefly some more advanced, state-of-the-art processing tools such as Riemannian geometry-based EEG analysis or Tensor analysis and notably Tensor Regression. All these tools have the potential to extract much more specific and robust EEG signatures and features for EEG-NF.
Why is this tool important for neurofeedback research?
Most typical EEG-NF use very basic processing tools, typically the power in a single channel and single fixed frequency band as target NF feature. However, it is known that the EEG signal a single EEG channel reflects the brain activity from multiple brain areas and not a single one. Moreover, the spatial and spectral signature of a given cognitive function (e.g., attention, used for Attention Deficit Hyper Activity Disorder NF for instance) is known to vary substantially from one patient to the next. Thus using a fixed and single EEG channel and frequency band must be suboptimal, possibly not relevant for the current patient, and cannot be specific to the target cognitive function only. Thus, using more advanced signal processing and machine learning tools would make it possible to 1) really extract spatial and spectral features that are specific to the target cognitive function and not to another one, using dedicated spatial and spectral filters; and 2) to identify from EEG data (examples) the individual EEG signature of this target cognitive function for each user.
What imaging modalities does this apply to?
Most of the tools we plan to describe are specific to EEG signal processing. Some of them though, such as ICA, Riemannian geometry or Tensors and Tensor Regression, could be applied to any modality, including fMRI. Most of our examples will be on EEG.
Is this a commercially available product, potentially commercially available, or open source?
This is an ensemble of machine learning and signal processing tools. Most of them are actually freely available in open-source toolboxes (e.g. ICALab, EEGLab, etc) and real-time EEG processing software (e.g., OpenViBE). We will mention where to find them.
Speakers:
Fabien Lotte (Inria Bordeaux Sud-Ouest, France / RIKEN BSI)
Neurofeedback based on real-time functional magnetic resonance imaging (rt-fMRI) is a novel and rapidly developing research field. It allows for training voluntary control over brain activity and connectivity, and has demonstrated promising clinical applications.
I’ll present a novel GUI-based multi-processing open-source neurofeedback framework for real-time fMRI neurofeedback training, termed OpenNFT Äi0[1] (Fig. 1, http://opennft.org/). This framework is based on the platform-independent interpreted programming languages Python and Matlab to facilitate concurrent functionality, high modularity, and the ability to extend the software in Python or Matlab into new directions depending on programming preferences, research questions, and clinical application. Our goal was to balance best practices in interpreted Python and Matlab programming. Thus, the core programing engine is Python, which provides larger functionality and flexibility than Matlab. Based on this core, we integrate Matlab processes to add specific functions. The choice for Matlab is motivated by its popularity, availability, strong mathematical potential and relatively easy scripting language, as well as by the availability and the potential of Matlab-based toolboxes as such SPM, PRONTO and PTB. Notably, OpenNFT Äi0is not limited to the use of Matlab and Matlab-based toolboxes, as different parts or even all of the Matlab code can be replaced with Python code.
The combination of the outlined features makes it the first software of this type that supports a broad functionality asset for neurofeedback studies including real-time fMRI data watchdog, data preprocessing, data processing, feedback estimation and feedback presentation. Such as, I’ll showcase the framework computational pipeline and performance on selected three neurofeedback data sets (real-time fMRI Demo data) that use (1) continuously and periodically displayed (intermittent) activation-based feedback; (2) intermittent effective connectivity feedback estimated based on dynamic causal modelling (DCM); (3) continuous classification-based feedback based on support vector machine (SVM).
Our software is distributed via GitHub repository together with the Demo data set, an installation manual, simulation routines, test functions and video tutorials. The latter should facilitate the Workshop session, e.g. users could set OpenNFT Äi0on their own laptops prior to the Workshop and get ready with their specific questions. I’m supposed to be the single tutor, and a few of my colleagues could be available online for more specific questions.
We believe that a modular open-source rt-fMRI framework will facilitate interfacing and integration of new approaches to support the progress in the highly dynamic field of neurofeedback research. Such as, using the open-source opportunities provided through GitHub makes it easy to follow the software updates, make your own fork releases, and request the software team to integrate your own development into the next core release.
Neural activity is accompanied by a hemodynamic (vascular) response that is sensitive to a host of features of coordinated brain function. Relating these measures to the seemingly endless breadth of human behavior is a principal aim of many scientific investigations. Fortunately, brain processes associated with learning, language acquisition, sensory and motor functions, emotion, social interactions, and a variety of diseases can be explored using functional near-infrared spectroscopy (fNIRS) signal. There is fNIRS technology that is portable, safe and easy to use, resistant to motion artifacts and can be employed in a person’s natural environment.
A promising application of fNIRS is its use in the context of brain-computer interfaces (BCIs). Several functional brain-imaging methods are used for BCI purposes, each of them with their advantages and disadvantages. In this scenario, fNIRS is particularly suited because of its portability, high single-trial reliability, robustness to motion artifacts and noninvasiveness.
In this workshop, NIRx will be presenting its portable fNIRS imager, the NIRSport. The NIRSport is a portable, multi-channel, modular fNIRS platform which measures hemodynamic neuroactivation via oxy-, deoxy-, and total hemoglobin changes in the cortex. The NIRSport comes in a 16-probe (8 sources and 8 detectors) configuration, with a diverse array of possible headgear combinations. The acquired signal can be processed in real time for BCI purposes. Classification can be based on spatial, temporal and magnitudinal fNIRS-signal features.
A promising fNIRS-BCI application is the design of communication and control BCIs for the disabled. For example, in the so-called ‘locked-in’ state, fully conscious and awake patients are unable to communicate naturally due to severe motor paralysis. These patients are, however, able to voluntarily modulate their brain activation which can be measured by fNIRS and exploited for brain-based communication. Dr. Bettina Sorger will be presenting a recent study with healthy participants demonstrating the feasibility of a multiple-choice fNIRS-based communication BCI using differently timed motor imagery as information-encoding strategy .
Speakers:
1. Lamija Pasalic (NIRx)
2. Bettina Sorger (Maastricht University)
Main Question:
What are the various control groups/conditions being implemented in rtfMRI studies, and what the pros and cons of each ?
Importance:
To establish that behavioral/cognitive changes are directly related to rtfMRI neurofeedback training and establish causality, studies must implement control conditions. This session will provide an overview of control conditions currently being employed in rtfMRI neurofeedback designs as well as how to select one over the other to meet the particular study goals.
Debate:
There are many different control conditions being employed in rtfMRI-nf research. There is no consensus as to which condition is best, and the answer likely depends on what aspects of the task design one is trying to control for. This can range from determining whether participants can learn to control regional hemodynamic activity via rtfMRI-nf to determining whether the feedback signal is necessary for learning to regulate hemodynamic activity. This session is designed to present participants with the most commonly employed control conditions in rtfMRI neurofeedback designs and to discuss the pros and cons of using each. Conditions to be covered include strategy only, bidirectional control, sham, and feedback from a different region.
Speakers:
1. Frank Scharnowski (Department of Psychiatry, Psychotherapy and Psychosomatics, University of Zurich, Zurich, Switzerland)
2. Bettina Sorger (Departments of Vision and Cognitive Neuroscience, Maastricht University, Maastricht, 6200 MD, The Netherlands)
3. Kymberly Young (Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, USA)
4. Mariela Rance (Department of Radiology and Biomedical Imaging & Magnetic Resonance Research Center, Yale University School of Medicine, New Haven, CT, USA)
What is the main question?:
Historically, the general lack of double-blinded placebo controls in neurofeedback studies has always been a source of concern. This issue is regularly brought up in high profile reviews as a caveat for the method. And yet, it remains unclear if genuine double-blinded studies are feasible for most applications. Others may question whether placebo control may be truly necessary or just too conservative; one could argue if a treatment is better than passage of time or some other existing treatment, it does not have to be shown to be more effective than placebo-controlled sham treatments. On the other hand, it is the gold standard of modern medicine to implement double-blinded clinical trials. Should neurofeedback studies attempt meet this standard? What would be the best way for our community to establish common guidelines for future studies?
Why is this an important question to neurofeedback researchers?:
The issue of double-blinded control concerns not just acceptance for clinical trials. It is a matter of experimental rigor that can affect the broad image of the field as a legitimate scientific community of the highest standard. This can affect the publishability of our papers in top high-profile journals, funding, etc, and may have far reaching implications for the growth and prosperity of the field.
Is there extant literature to support different sides of the debate?:
The following are some example articles on the relevant controversies, which appeared recently in the literature and generated considerable impact:
Sitaram R, Ros T, Stoeckel L, Haller S, Scharnowski F, Lewis-Peacock J, Weiskopf N, Blefari ML, Rana M, Oblak E, Birbaumer N, Sulzer J. Closed-loop brain training: the science of neurofeedback. Nat Rev Neurosci. 2017 Feb;18(2):86-100.
Lofthouse N, Arnold LE, Hurt E. Current status of neurofeedback for attention-deficit/hyperactivity disorder. Curr Psychiatry Rep. 2012 Oct;14(5):536-42.
Young KD, Siegle GJ, Zotev V, Phillips R, Misaki M, Yuan H, Drevets WC, Bodurka J. Randomized Clinical Trial of Real-Time fMRI Amygdala Neurofeedback for Major Depressive Disorder: Effects on Symptoms and Autobiographical Memory Recall. Am J Psychiatry. 2017 Apr 14:appiajp201716060637
Schabus M, Griessenberger H, Gnjezda MT, Heib DPJ, Wislowska M, Hoedlmoser K. Better than sham? A double-blind placebo-controlled neurofeedback study in primary insomnia. Brain. 2017 Apr 1;140(4):1041-1052.
Speakers:
1. Tor Wager (University of Colorado, Boulder)
2. Vincent Taschereau-Dumouchel (UCLA)
3. Kimberly Young (University of Pittsburg)
Co-chair / Co-submitter:
Mitsuo Kawato (ATR, Kyoto)
What is the main question?:
What are the caveats of real-time neuroimaging research practices? Can we eventually arrive at a set of guidelines and recommendations for quality control, to be employed by all researchers across the field of real-time fMRI neurofeedback?
Why is this an important question to neurofeedback researchers:
With the recent exponential increase in rt-fMRI studies, especially from new research groups, it is easy for tacit quality assurance practices to be neglected.
A lack of commonly accepted standard recommendations means that a novel neurofeedback researcher has no precise guidelines to adhere to, in order to benefit from the accumulated experience of the field and ensure that even one’s first experiment will not be flawed by technological challenges that other researchers have already resolved in older labs. This is a problem that in the long run can become damaging to the entire field of real-time neurofeedback.
In this special session, we have invited a few experienced rt-fMRI researchers to share their experiences and approaches to overcoming common caveats of real-time practices, in an attempt to eventually integrate various approaches to real-time quality control into a coherent set of recommendations for best practices.
Is there extant literature to support different sides of the debate:
This is a special topic with different sides that are complementary rather than opposing one another. This is so, because it seems that different labs have placed differential emphasis on particular aspects of quality control, often depending on the particularities of the research questions addressed and the technological apparatus available.
The literature on this topic is relatively limited and this is precisely why the functional neurofeedback community must collaborate on establishing more precise criteria. This topic is controversial because most people agree on the need for quality assurance but there is no fixed set of recommendations for best practices.
Some of the ensuing issues, that our speakers have placed differential emphasis on, include: optimizing filtering procedures, controlling timing accuracy, maximizing SNR and CNR, controlling for physiological noise, ensuring data consistency, discarding outliers, selecting reproducible ROIs, correcting for field distortions and working in real-time at 7T.
Speakers:
1. Klaus Mathiak (Uniklinik RWTH Aachen)
2. Lydia Hellrung (University of Zürich)
3. Johan van der Meer (Otto-von-Guericke University / University of Amsterdam)
Neurofeedback training induces neuroplastic changes underlying behavior, but it is unclear by what mechanism it operates. Our lack of a mechanistic model for neurofeedback learning (Sitaram et al. 2016) prevents our understanding of why some of the most promising pilot studies fail to translate in clinical trials (Hawkinson et al. 2012). It is unclear how such self-modulated brain activation results in neuroplasticity underlying behavioral changes. The prevailing hypothesis is that self-regulation acts as endogenous stimulation, analogous to exogenous transcranial magnetic stimulation (TMS) (Ros et al 2010) or transcranial direct current stimulation (tDCS). According to this mechanism, participants should maximize activity in a neural circuit as much as possible. Some recent examples showing behavioral changes following such neurofeedback training include self-regulation of alpha waves (Okazaki et al. 2015) to improve visual sensitivity, increasing motor cortical BOLD to enhance fine motor control (Blefari et al. 2015), sensorimotor rhythm to enhance surgical skills (Ros et al. 2009), and up-regulation of DLPFC BOLD to improve working memory (Sherwood et al. 2016). Likewise, down-regulation of activation in the presence of a stimulus (e.g. Zilverstand et al. 2015; Nicholson et al. 2017) is assumed to be a type of inhibitory command. Nevertheless, we still lack sufficient empirical evidence to fully embrace the endogenous stimulation model. One confounding factor is the emerging evidence from brain stimulation studies indicating high intra-individual variability of induced plasticity following stimulation (e.g. LTP-like vs LTD-like) (Ridding and Ziemann 2010), suggesting that Hebbian-like plasticity following neurofeedback can be additionally influenced by homeostatic mechanisms (Kluetsch et al 2014).
Hence, instead of maximizing activity of a given substrate, another model of the efficacious mechanism of neurofeedback may be matching a set point of activation, for example, that of an expert or healthy control. This is most visible in studies that employ multivariate pattern analysis (MVPA)-based neurofeedback, such as matching a voxel-wise pattern of activity in visual cortex to manipulate visual perception (Shibata et al. 2011) or adaptive neurofeedback to manipulate attentional vigilance (deBettencourt et al. 2015). Non-MVPA-based studies that differentially regulate neural circuitry like motor cortical laterality (Nayedli et al. 2017) aim to normalize brain activity, in those with neurological dysfunction relative to healthy controls in order to restore more balanced network function (e.g. excitation/inhibition ratio, neural synchronization). In this Special Topics discussion, we will cover the evidence to support these competing, and probably coexisting, accounts of neuroplasticity in neurofeedback and offer suggestions on how to test these hypotheses.
Speakers:
1. James Sulzer, Ph.D. (University of Texas at Austin)
2. Tomas Ros, Ph.D. (University of Geneva)
3. Jarrod Lewis-Peacock Ph.D. (University of Texas at Austin)
TBD
Methods of cortical information flow analysis, based on non-invasive EEG/MEG recordings, provide a unique contribution to the human connectome, from the point of view that they span the whole cortex and give a directional, high-time resolution account of information transactions. Firstly, an inverse solution is used for the estimation of cortical signals of electric neuronal activity. Secondly, measures of “connectivity” (e.g. derived from autoregressive models or from cross-frequency coupling models) are applied to these signals for connectome inference.
However, these signals have low spatial resolution, such that any estimated signal is an instantaneous mixture of the true-unobserved signals across the cortex. False positive connections will result from low spatial resolution, thus producing false connectomes, even for connectivity measures thought to be immune to low resolution, such as cross-frequency phase-amplitude coupling and isolated effective coherence. One recent literature approach to solve this problem is called “leakage correction”, which is based on the assumption that the unobserved signals have zero cross-correlation at lag zero. We contend that this is a baseless assumption, and show how it leads to false connections under a very broad range of electrophysiological conditions.
We solve this problem with the method of “innovations orthogonalization”, which is based on the assumption that the multivariate autoregressive innovations are orthogonal. It is shown that under very broad conditions, the new method produces proper human connectomes, even when the signals are not generated by an autoregressive model.
Recent developments of resting state functional connectivity and diffusion MRI allow for investigating how our brain is organized as a network. These methods have revealed macro-scale network organization of human brain such as small world topology, existence of functional subnetwork (default mode, attention, sensory and motor systems), and individual difference of functional connectivity. However investigating dynamics on the brain network is challenging partially because of lack of measurement methodology. To address this issue, we have been developing fMRI-informed MEG/EEG source reconstruction method to visualize brain activities in milli-second temporal resolution (Sato et al. 2004, NeuroImage, MATLAB toolbox available from http://vbmeg.atr.jp/?lang=en). Effectiveness of the method has been demonstrated with basic neuroscience experiments (Yosioka et al, 2008, NeuroImage, Shibata et al. 2008, Cerebral Cortex) as well as BMI applications (Toda et al. 2011, NeuroImage, Yoshimura et al. 2012, NeuroImage, Yanagisawa et al, 2016, Nature comm). Recently we further developed extension of VBMEG to describe event-related brain network dynamics (Fukushima et al, NeuroImage, 2015). In this method, a dynamical generativeprocess of MEG is modeled via a current-source network dynamical model whose network structure is constrained by diffusion MRI. Algorithm to infer current sources and the dynamics model parameters from MEG and fMRI data is proposed. This method allows for visualization how brain activities are generated by interactions on structural network. In this talk I summarize our attempt to understand event-related brain dynamics using multi-modal integration approach.
One broad objective in neuroscience is to comprehend the mechanisms of large- scale, oscillatory neural dynamics: how they enable functions by shaping communication in brain networks, and how the earliest detection of their alterations in disease can contribute to improved healthcare prevention and interventions. We will review how the ubiquitous polyrhythmic activity of the brain has been approached empirically so far, with underlying mechanisms that remain not understood. This hinders our comprehension of how 1) perception and behaviour emerge from brain network activity, and 2) the pathophysiological developments of brain and mental- health disorders increasingly studied as network diseases, affect large-scale neural communication.
I will introduce how these difficult questions can benefit from a bottom-up approach: We aim to understand how basic physiological factors of neural integrity and function shape the dynamical structure of oscillatory brain rhythms, such as their interdependence across multiple frequencies through cross-frequency coupling. These phenomena represent a deep source of uncharted markers of neural excitability, activity and connectivity. I will illustrate these principles with our latest results concerning the resting brain, multimodal perception and pathophysiological markers of epilepsy and neurodegenerative syndromes.
Talma Hendler
My talk will focus on the intentions and hurdles in using NeuroFeedback (NF) as a non-invasive neuromodulation technique in neuropsychiatry by addressing three opened issues. First, possible therapeutic mechanisms of NF, considering the neural systems of reward, salience and control process their dependency in targeting. Second, the scope and limitations of existing clinical trials with NF that were aimed to alleviate depression, trauma related anxiety and emotion dysregulation. Third, ways and directions for improving scientific rigor and through that clinical effectiveness of NF training. Across these issues, the usage of personalized neural indications for training protocol, context specific immersive brain-computer-interfaces, and home-based NF platforms, will be pondered. Altogether this talk will provide a critical overview of current and future perspectives on the role of NF in harnessing the brain to heal itself.
Adam Gazzaley
A fundamental challenge of modern society is the development of effective approaches to enhance brain function and cognition in both the healthy and impaired. For the healthy, this should be a core mission of our educational system and for the cognitively impaired this is the primary goal of our medical system. Unfortunately, neither of these systems have effectively met this challenge. I will describe a novel approach out of our center at UCSF – Neuroscape – that uses custom-designed video games to achieve meaningful and sustainable cognitive enhancement via personalized closed-loop systems (Nature 2013; Neuron 4014). I will also share with you the next stage of our research program, which integrates our video games with the latest technological innovations in software (e.g., brain computer interface algorithms, GPU computing, cloud-based analytics) and hardware (e.g., virtual reality, mobile EEG, motion capture, physiological recording devices (watches), transcranial brain stimulation) to further enhance our brain’s information processing systems with the ultimate aim of improving quality of life.
Gerwin Schalk
Resective brain surgery is often performed in people with intractable epilepsy, congenital structural lesions, vascular anomalies, or neoplasms. Surgical planning of the resection procedure depends substantially on the delineation of abnormal tissue and on the creation of a functional map of eloquent cortex, i.e., cortex involved in motor or language function. Traditionally, different methodologies have been used to produce this functional map, most notably electrical cortical stimulation (ECS) and functional magnetic resonance imaging (fMRI), but each of these methods has important shortcomings (including increased morbidity, time consumption, expense, or practicality).
Patients undergoing invasive brain surgery would benefit greatly from a mapping methodology that is safe, can be rapidly applied, is comparatively inexpensive, is procedurally simple, and also is congruent to existing techniques (in particular to ECS mapping). Task-related changes detected in electrocorticographic (ECoG) recordings could provide the basis for a technique with those desirable characteristics. This approach seems particularly attractive because existing surgical protocols often already include the placement of subdural electrodes, and because a number of recent studies have shown that ECoG activity in the broadband gamma (70-170 Hz) band are directly reflective of activity of neuronal populations directly underneath the electrodes and can also be directly linked to the BOLD response detected using fMRI.
Over the past decade, we have been using and extending this understanding, and been applying it to develop a robust and practical ECoG-based procedure for presurgical functional mapping of eloquent cortex. This procedure is now readily available to others (cortiQ, www.cortiq.eu). We and others have shown that this procedure can produce a functional map of motor, language, or cognitive function within a few minutes, and that the results of this ECoG-based mapping are in strong congruence to the results derived using ECS mapping.
In this talk, I will be describing the neurophysiological and technical principles of this technique, and give examples of its clinical utility in the context of different types of invasive brain surgery. I will also be able to discuss the practical clinical relevance of this technique compared to ECS and fMRI.
Byron Yu
Learning has been studied at multiple levels, including behavior, brain regions, individual neurons, and synapses. However, little is known about how populations of neurons change their activity in concert during learning. Are there network constraints on the types of new neural activity patterns that can be achieved? We studied this
question using a brain-computer interface (BCI), which allows us to specify which population activity patterns lead to task success. We identified a simple network principle that can predict which types of activity patterns are easier or harder for the subject to learn to generate. This work provides a network-level explanation for why
learning some tasks may be easier than others.
Kazuhisa Shibata
Real-time fMRI neurofeedback can change local brain activity and its related behavior. Since the advent of fMRI neurofeedback, amazing progress was achieved by being equipped with an implicit protocol, external reward, multivariate analysis. In this talk, first I will summarize these three aspects. Second, I will describe how these aspects have been integrated into the new technology, decoded neurofeedback (DecNef), and how DecNef has advanced basic and clinical brain research. Third, I will discuss the potential problems of DecNef such as the one-to-many relationship from a voxel pattern to neuronal activity patterns and the curse of dimensionality, and propose theoretical solutions to these problems. Finally, I will present results of our meta-analyses and simulations based on recent DecNef studies in order to assess the validity of these theoretical solutions.