Independent component analysis-based source-level hyperlink analysis for two-person neuroscience studies

Yang Zhao; Rui-Na Dai; Xiang Xiao; Zong Zhang; Lian Duan; Zheng Li; Chao-Zhe Zhu

doi:10.1117/1.JBO.22.2.027004

8 February 2017 Independent component analysis-based source-level hyperlink analysis for two-person neuroscience studies

Yang Zhao, Rui-Na Dai, Xiang Xiao, Zong Zhang, Lian Duan, Zheng Li, Chao-Zhe Zhu

Author Affiliations +

Journal of Biomedical Optics, Vol. 22, Issue 2, 027004 (February 2017). https://doi.org/10.1117/1.JBO.22.2.027004

Abstract

Two-person neuroscience, a perspective in understanding human social cognition and interaction, involves designing immersive social interaction experiments as well as simultaneously recording brain activity of two or more subjects, a process termed “hyperscanning.” Using newly developed imaging techniques, the interbrain connectivity or hyperlink of various types of social interaction has been revealed. Functional near-infrared spectroscopy (fNIRS)-hyperscanning provides a more naturalistic environment for experimental paradigms of social interaction and has recently drawn much attention. However, most fNIRS-hyperscanning studies have computed hyperlinks using sensor data directly while ignoring the fact that the sensor-level signals contain confounding noises, which may lead to a loss of sensitivity and specificity in hyperlink analysis. In this study, on the basis of independent component analysis (ICA), a source-level analysis framework is proposed to investigate the hyperlinks in a fNIRS two-person neuroscience study. The performance of five widely used ICA algorithms in extracting sources of interaction was compared in simulative datasets, and increased sensitivity and specificity of hyperlink analysis by our proposed method were demonstrated in both simulative and real two-person experiments.

1. Introduction

Two-person neuroscience, a perspective in understanding human social cognition and interaction, has been recently introduced.¹^–³ Compared to traditional single-person neuroscience, two-person neuroscience involves designing immersive social interaction experiments rather than focusing on passive spectators and stimuli as well as simultaneously recording the brain activity of two or more subjects using electroencephalography (EEG), magnetoencephalography, functional magnetic resonance imaging, or functional near-infrared spectroscopy (fNIRS), a process termed “hyperscanning.”⁴ Using these newly developed imaging techniques, a phenomenon consisting of synchronized neural activity between two brains of a pair of humans, called interbrain connectivity or hyperlink, has been revealed in various types of social interaction, deepening our comprehension of the neural mechanisms involved in human social interactions.

Among the abovementioned imaging modalities used in hyperscanning studies, fNIRS offers both moderate spatial and temporal resolutions and high ecological validity, enabling more naturalistic environments for experimental paradigms of social interaction.⁵ Accordingly, fNIRS-hyperscanning studies have drawn much attention and have provided insights into two-person neuroscience. For example, Cui et al.⁶ performed a two-person button press experiment and found an increase in fNIRS-hyperlink (specifically interbrain coherence) during cooperation but not during competition. Holper et al.⁷ demonstrated that in a Socratic educational dialog, teachers and students showed an increase in fNIRS-hyperlink (specifically positive correlation of brain activity) when teaching was successful. Jiang et al.⁸ showed that fNIRS-hyperlink (specifically interbrain coherence) was significantly higher between leaders and followers than between followers and followers, suggesting that leaders emerge by synchronizing their brain activity with that of the followers.

However, most fNIRS-hyperscanning studies have computed fNIRS-hyperlinks by directly using sensor data and have ignored the fact that sensor-level signals contain not only neural activity but also various confounding noises.⁹ Body movement, facial expression, and head motion are difficult to eliminate during the process of complex brain to brain social interaction and, thus, may induce slippage of the probes on the scalp, leading to motion artifacts in raw fNIRS sensor data.¹⁰ Moreover, raw fNIRS sensor data are also mixed with physiological signals including cardiac pulsation, respiration, blood pressure variation, Mayer waves, and other low frequency variations.¹¹ All of the abovementioned contaminants within raw fNIRS sensor data may degenerate neural signals and reduce the sensitivity of sensor-level hyperlink computation. More importantly, it has been found that social tasks may induce physiological coupling, such as synchronization of cardiac pulse and respiration.¹² Trivial links may also originate from common physiological fluctuations induced by social-specific tasks.¹³ These spurious couplings can thus reduce the neural specificity of sensor-level hyperlink computation. Accordingly, an approach to hyperlink computation with better sensitivity and specificity would be beneficial for fNIRS-hyperscanning studies.

Independent component analysis (ICA) is a data-driven method for extracting and separating intrinsic components from mixed signals. It has been widely applied in traditional fNIRS-based single brain neuroscience studies to remove signal artifacts¹⁴ and to extract task-related neural activity,¹⁵ as well as to analyze resting state functional connectivity.¹⁶ In this study, we propose a three-step source-level analysis framework based on ICA to investigate the between-brain hyperlinks for two-person fNIRS neuroscience studies. First, fNIRS data of the interactors are decomposed into multiple temporal and spatial modes using ICA. The resultant temporal mode, as well as the corresponding spatial mode, is defined as a “source.” Second, sources of interaction (SoI), i.e., sources specific to neural processes of social interaction, can be determined with properly predefined criterion depending on the specific purpose of a particular study. Finally, the hyperlink between the SoI of a dyad can be computed. To determine the ICA algorithm and evaluate the performance of the proposed method, both simulative and real two-person interaction experiments were conducted.

2. Theoretical Analysis and Method

2.1.

Source-Level Hyperlink Analysis Framework

2.1.1.

Single brain independent component analysis decomposition

As the first step, temporal ICA is performed to decompose each interactor’s brain hemodynamic signals into a set of temporal modes and corresponding spatial modes. The general assumption made by ICA is that sources of neural activity are temporally independent from other sources, i.e., physiological noises and motion artifacts, which have been supported by many previous single brain fNIRS studies. For example, Katura et al.¹⁵ successfully extracted task-related neural sources as well as task-related N-shape physiological noise sources in a finger-tapping task. Zhang et al.¹⁶ performed an analysis of resting state functional connectivity using ICA and proved that it renders better results than seed-based correlation methods due to its powerful noise-separating capability. A basic ICA model for the $i$ th interactor in a dyad is as follows:

X^{i} = A^{i} \cdot S^{i},

which assumes that our observed time courses in

X^{i}

is a linear combination of the independent temporal components in

S^{i}

using a mixing matrix

A^{i}

. In detail:

[\begin{matrix} x_{1}^{i} (t) \\ ⋮ \\ x_{m}^{i} (t) \end{matrix}] = [\begin{matrix} a_{11}^{i} & \dots & a_{1 n}^{i} \\ ⋮ & ⋱ & ⋮ \\ a_{m 1}^{i} & \dots & a_{m n}^{i} \end{matrix}] \cdot [\begin{matrix} s_{1}^{i} (t) \\ ⋮ \\ s_{n}^{i} (t) \end{matrix}],

where

x_{j} (t) : (j = 1, 2, \dots, m, t = 1, 2, \dots, T)

are the data from the

i

’th fNIRS sensors.

s_{k} (t) : (k = 1, 2, \dots, n, t = 1, 2, \dots, T)

are

n

independent temporal modes

(n \leq m)

. The

k

’th column

[\begin{matrix} a_{1 k} & a_{2 k} & \dots & a_{m k} \end{matrix}]

contains

m

weights of the component

s_{k} (t)

corresponding to

m

sensors, representing the spatial mode. The temporal modes

s_{k} (t)

as well as its corresponding spatial modes

[\begin{matrix} a_{1 k} & a_{2 k} & \dots & a_{m k} \end{matrix}]

are defined as the

k

’th source. The question addressed by ICA is then how to derive both

A^{i}

and

S^{i}

using only information contained in the observed data

X^{i}

, which is also known as the blind source separation problem.¹⁷

In the first step of ICA decomposition, sphering was often performed on the fNIRS hemodynamic signals using principal component analysis (PCA) for two purposes. First, when the signals are sphered, we only need to perform ICA in an orthogonal space. Second, noise components can be discarded by retaining some number of top principal components, which will also determine the number of sources estimated in ICA. These will substantially reduce time costs and overlearning problems encountered by ICA.¹⁸ Specifically, for a $i$ ’th interactor in a dyad:

{\hat{X}}^{i} = V^{i} \cdot X^{i},

where

X^{i}

is a

M \times T

matrix containing

M

channels of hemodynamic signal with T time scans and it is multiplied by

V^{i}

, a

N \times M (N < M)

whitening matrix. This matrix multiplication will reduce the dimension of the data to

N

and make a

N \times T

matrix

{\hat{X}}^{i}

with

N

whitened signals. In our simulative experiment,

N

was determined manually, while in our real experiment, no dimension reduction was used

(N = M)

. After the PCA step, ICA was performed on the whitened matrix

{\hat{X}}^{i}

.

The basic idea of ICA is to change a matrix $W^{i} (N \times N)$ iteratively to maximize the independence of the components $s (t)$ ; it produces

W^{i} \cdot {\hat{X}}^{i} = {\hat{S}}^{i} .

Many ICA algorithms have been proposed to determine $W^{i}$ using different measures of statistical independence between components. In this study, to determine which algorithms are suitable for extracting SoI from two-person fNIRS data, the performance of five widely used algorithms—FastICA,¹⁹ SOBI,²⁰ Infomax,²¹ JADE²² and MILCA²³—was compared using simulative two-person datasets series 1 (code and parameters of each algorithm can be found in the Appendix). The performance of each algorithm was evaluated in both temporal and spatial aspects. Then, the algorithm with the highest performance was used in later analysis. After derivation of $W^{i}$ , the estimated mixing matrix ${\hat{A}}^{i} = {(W^{i} \cdot V^{i})}^{- 1}$ can be obtained from the above two equations.

2.1.2.

Sources of interaction selection

After ICA decomposition, the SoI can be selected either manually or automatically using prior information, such as anatomical region of interest (RoI)²⁴ or reference time courses²⁵ of social interaction. In the simulative experiment, we incorporated spatial templates to select the SoI. Specifically, the sources showing maximum correlation with spatial templates of RoI were selected for each interactor. In the real fNIRS study, the SoI was manually selected by investigating its spatial and temporal modes. Specifically, the spatial mode of the SoI should present a higher value in the RoI as compared to that outside the RoI, and its temporal mode should not be noise-like.

2.1.3.

Hyperlink computation

After SoI selection, both hyperlink strength and spatial pattern can be derived in source-level hyperlink analysis. In this study, the Pearson correlation coefficient, a simple hyperlink index,⁷ was used to estimate the linear hyperlink strength between the two selected SoI as follows:

ρ (s_{i}^{1}, s_{j}^{2}) = \frac{Cov (s_{i}^{1}, s_{j}^{2})}{σ_{s_{i}^{1}} σ_{s_{j}^{2}}},

where Cov stands for covariance and

σ_{s_{j}^{i}}

for standard deviation of

s_{j}^{i}

, which is the temporal mode of the

j

’th source from the

i

’th

(i = 1, 2)

interactor. This index was used in both the simulated and real fNIRS experiments. Other more complex hyperlink measurements, such as WTC⁶ or Granger Causality,²⁶ can also be used. To derive the spatial pattern of the dyadic hyperlink representing its channel position, it is reasonable to average the spatial modes of the two selected SoI of a dyad, which are usually similar to each other. Note that two steps are needed before averaging the individual spatial modes derived by ICA. First, the sign of the source was calibrated according to Ref. 16 since it was undetermined. Second, the spatial modes were transformed to z-maps.²⁷

2.2.

Sensor-Level Hyperlink Analysis

In this study, to compare the performance with source-level method, sensor-level hyperlink analysis with a Pearson correlation coefficient used as the hyperlink index⁷ was performed on both simulative and real datasets (Fig. 1). However, other indices can be used as well, similar to the hyperlink computation step in source-level hyperlink analysis. Sensor-level analysis renders a hyperlink map, which depicts the spatial pattern of the hyperlink index.

Fig. 1

Schematic comparison of a traditional sensor-level hyperlink analysis and an ICA-based source-level analysis framework.

2.3.

Simulative Two-Person Datasets

Simulative two-person datasets were generated to compare the performance of different ICA algorithms and measure the potential increase in sensitivity and specificity resulting from the proposed ICA-based source-level hyperlink analysis as compared to the traditional sensor-level method.

FNIRS data of a dyad were generated using the following forward model (Fig. 2):

D_{i} = D_{i}^{Neu} + D_{i}^{Phy} + D_{i}^{Mot} + D_{i}^{In} (i = 1, 2),

where

i

indexes the

i

’th interactor in the dyad.

D_{i}^{Neu}

,

D_{i}^{Phy}

,

D_{i}^{Mot}

, and

D_{i}^{In}

represent the sources of the neural signal, physiological noise, motion artifacts, and instrumental noise, respectively (Fig. 2). Each source was provided by the outer product of their spatial and temporal modes. For example:

D_{i}^{Neu} = T_{i}^{Neu} \otimes S_{i}^{Neu} .

Fig. 2

Spatial modes (left) and temporal modes (right) of a typical simulated dataset. Top row: neural sources. Middle row: physiological noise sources. Bottom row: motion artifact sources.

The sources used to generate the simulative datasets were defined as “true sources” for later use. The same temporal mode was set so that the two interactors ( $T_{1}^{Neu} = T_{2}^{Neu}$ ) would have a strong hyperlink, i.e., $ρ (T_{1}^{Neu}, T_{2}^{Neu}) = 1$ , and their corresponding spatial modes were set to be identical (Fig. 2, top row). The temporal mode of the neural signal was provided by simulated pseudorandom events convolved with the canonical hemodynamic response function (HRF).²⁸ The physiological noise, $T_{i}^{phy}$ , including breathing, heart rate, Mayer wave, and systemic physiological noise,²⁹ was simulated using sinusoidal waves at respective frequencies (Fig. 2, middle row). The spatial modes of $D_{i}^{Phy}$ were global and made using Gaussian random noise. Motion artifacts $T_{i}^{mot}$ were generated using a triangular pulse, whose amplitudes were 5 to 12 times the amplitudes of $T_{i}^{phy}$ (Fig. 2, bottom row). The spatial modes of $D_{i}^{mot}$ were generated in the same way as $D_{i}^{Phy}$ . Finally, instrumental noise $D_{i}^{In}$ was added to each channel using channel-specific Gaussian random noise. Details regarding the data generation process can be found in the Appendix.

2.3.1.

Simulative two-person datasets series 1: hyperlink contaminated by interactor-specific noise

Datasets with different amplitudes for interactor-specific physiological noise ( $T_{i}^{phy}$ ) and motion artifacts ( $T_{i}^{mot}$ ) were generated to compare the sensitivity between two methods (source-level versus sensor-level) in detecting the neural hyperlink. Specifically, we fixed the amplitudes of the neural signal and manipulated noise amplitudes by varying the standard deviations of $T_{i}^{phy}$ , $T_{i}^{mot}$ , and instrumental noise. Thus, a series of data with different noise amplitudes (0.1, 0.5, 1, 2, 4) wase generated. Ten datasets (10 dyads) were randomly generated for each value of noise amplitude.

2.3.2.

Simulative two-person datasets series 2: spurious link due to synchronized physiological noise

Considering that physiological synchronization may occur during social interactions,¹² we also simulated synchronized physiological noise by setting $T_{1}^{Phy} = T_{2}^{Phy}$ . Data with five different amplitudes (0.1, 0.5, 1, 2, 4) of synchronized physiological noise were generated (amplitudes of $T_{i}^{mot}$ were kept as 5 to 12 times $T_{i}^{phy}$ ). These spurious physiological synchronizations enabled us to compare the specificity of the two methods. As in datasets series 1, 10 dyads were generated for each amplitude setting of synchronized physiological noise.

2.4.

Real Two-Person Interaction Experiment

2.4.1.

Protocol

To validate our method, we used another real social interaction experiment conducted by our group in which a joint-tapping task was performed by the interactors while their brain activities were simutanously recorded using fNIRS hyperscanning [Fig. 3(a)]. About 48 right-handed participants [24 males and 24 females, mean age 22.77 years ( $SD = 2.19$ )] took part in this experiment and formed 24 same-gender dyads. This task involved a mutual adaptation condition in which interactors were instructed to tap synchronously with their partner. Only this condition is used for validation purpose. The experimental paradigm involves a block design [Fig. 3(c)], in which one block includes 10 s preparation, $\approx 10 s$ instruction of condition and hints of metronome (10 beats), 35 s joint tapping without metronome and $\approx 6 s$ rest (totally four blocks). Ten channels fNIRS hemodynamic signal for each participant were acquired by a multichannel fNIRS system (ETG-4000, Hitachi Medical Corporation) in the prefrontal cortex with a 3 cm source–detector separation $({DPF}_{695} = 6.51, {DPF}_{830} = 5.86)$ , where DPF is the differential path length factor [Fig. 3(b)]. The State Key Laboratory of Cognitive Neuroscience and Learning at Beijing Normal University approved the protocol, and informed consent was obtained from all participants.

Fig. 3

Experimental setup, probe montage, and experimental paradigm for the real fNIRS-hyperscanning. (a) Experimental setup; (b): FNIRS-probe montage; and (c) experimental paradigm, red circle indicates the period of interest.

2.4.2.

Data analysis

After data checking, data of three dyads were discarded due to very low SNR. In addition, bad channels (channel 6 of the second interactor in each dyad) were also omitted. First and second order detrends were first done to remove the linear and bilinear trends.¹¹ Since the condition was in a pseudorandomized order, the periods of interest [Fig. 3(c)] in mutual condition were taken and concatenated together. Only $Δ HbO$ data were used in this study. In the source-level method, SoI was selected manually according to two criteria: (1) its spatial mode should cover the medial prefrontal cortical areas, i.e., within the channels 2, 3, 5, 6, 7, 9, and 10 and (2) its temporal mode is not noise-like. Group-level hyperlink strength was derived by calculating the mean and the standard error of dyadic hyperlink strength. The group-level spatial pattern was derived by performing one-sample $t$ -tests on all dyadic spatial patterns.

To compare with the source-level hyperlink results, sensor-level channel-wise correlation analysis was also computed on the real fNIRS data. The group spatial pattern was derived by performing one-sample $t$ -tests on dyadic sensor-level correlation maps.

3. Results

3.1.

Simulative Two-Person Experiment

3.1.1.

Single brain decomposition

The performance of ICA algorithms was evaluated by correlating derived temporal and spatial modes of SoI with those of the true sources. We reduced the dimension of datasets to 10 using PCA for all ICA algorithms in this comparison. The accuracy of all ICA algorithms in extracting sources of social interaction decreased (Fig. 4) as the datasets became noisier. The results show that the temporal and spatial accuracy of Infomax and SOBI outperformed the other algorithms; while SOBI performed slightly better than Infomax, especially in noise amplitudes 2 and 4. These algorithms were also tested with other choices for the number of principal components, and the results remain similar. Therefore, SOBI was selected as the ICA algorithm in later analysis.

Fig. 4

Comparison of the performance of five ICA algorithms on the simulative datasets series 1. (a) Temporal accuracy of the five ICA algorithms under different noise amplitudes. (b) Spatial accuracy of the five ICA algorithms under different noise amplitudes.

3.1.2.

Comparison with sensor-level method

Simulative two-person dataset series 1

Results demonstrating increased sensitivity using the source-level analysis of a typical dyad under noise amplitude 1 are presented in Fig. 5. The temporal modes of the SoI are highly correlated with the true neural sources, and the averaged spatial modes of the SoI, i.e., hyperlink spatial pattern, are also located in the same channels as the true position of the hyperlink [Fig. 5(a)]. The correlation between the temporal modes of the SoI was 0.63. However, the sensor-level method was contaminated by both physiological and motion artifact noise, so the correlation of the channels containing the strongest neural signals in the RoI was only 0.31. The increase in sensitivity by the source-level method was further evaluated under different noise amplitudes (10 dyads for each noise amplitude setting). To compare the hyperlink strength of the two methods [Fig. 6(d)], for the sensor-level method, the correlation value between RoI channels with the strongest neural signals (the brightest channel, no. 28, in the spatial mode in Fig. 2, top row) was used. The source-level method produced higher mean and lower standard deviation values for the hyperlink strength than the sensor-level method [Fig. 5(c)]. The spatial pattern of the RoI could be successfully extracted as the representative spatial mode in all tested noise amplitudes using the source-level method (not shown). These results demonstrate the increase in sensitivity by ICA-based source-level hyperlink analysis.

Fig. 5

Results for a typical simulated dyad with noise amplitude 1. (a) ICA-based source-level results, left: averaged spatial mode of the SoI, the channel delimited by the dashed green line is the true hyperlink RoI. Right: temporal modes of the SoI, dashed green line represents the temporal modes of the true neural sources. (b) Time courses of the two interactors within the RoI channel. (c) Comparison of hyperlink strength derived by the two methods under each noise amplitude.

Fig. 6

Results of a typical simulated dyad, where two interactors have synchronized physiological noise with amplitude 1. (a) ICA-based source-level results, channel within the dashed green square is the hyperlink RoI; left: averaged spatial mode of the SoI, right: temporal modes of the SoI. (b) Synchronized physiological noise sources extracted by ICA; left: averaged spatial mode of synchronized physiological noise sources, right: temporal modes of synchronized physiological noise sources. (c) Sensor-level results; left: correlation map, right: timecourse of the two interactors in a channel with spurious physiological correlation.

Simulative two-person datasets series 2

The results for a typical dyad with synchronized physiological noise are shown in Fig. 6. Using the sensor-level method, the hyperlink strength in many channels outside the true RoI was spurious due to synchronization of the global physiological noise [Fig. 6(c)]. However, both SoI and synchronized physiological noise were successfully extracted by ICA, and their temporal modes showed high correlation when the source-level method was used [Figs. 6(a) and 6(b)]. The spurious link outside the true RoI was greatly reduced by the source-level method due to its capability for separating synchronized physiological noise from other sources [Fig. 6(b)].

Results of dyads with synchronized physiological noises of different amplitudes are shown in Fig. 7. As in the previous simulation, the spatial pattern of the RoI was successfully selected and the hyperlink strength was detected as significantly higher than 0 (not shown) when using the source-level method (two-tail one-sample $t$ -test, $p < 0.01$ ). However, many spurious correlations were found using the sensor-level method due to the synchronization of physiological noise, especially when the physiological noise was strong [Fig. 7(a)]. The spatial accuracy index, defined as correlation with the true spatial modes [Fig. 7(b)], and the receiver operating characteristic curves of the spatial pattern [Fig. 7(c)] were also computed. The source-level method showed a significantly higher spatial accuracy (two-tail paired $t$ -test, $p < 0.01$ ) and a larger area under the curve in all tested noise amplitudes. These simulation results show that the ICA-based source-level hyperlink analysis greatly reduced the effect of synchronized physiological noise and increased the specificity of hyperlink analysis.

Fig. 7

Results of simulated datasets where two interactors have physiological noise synchronization under different physiological noise amplitudes. (a) Comparison of source-level and sensor-level hyperlink spatial patterns, a channel within the dashed green square is in the hyperlink RoI. (b) Comparison of the spatial accuracy between the spatial patterns. Each bar represents the mean and the standard deviation of spatial accuracy indices of 10 dyads for each amplitude setting of the sychronized physiological noise. (c) Comparison of receiver operating characteristic curves of spatial patterns derived by the two methods. Red and blue curves represent the results using the source-level method and the sensor-level method, respectively; five receiver operating characteristic curves under different physiological amplitudes in (a) are drawn for each method.

3.2.

Real Two-Person fNIRS Experiment

Figure 8 shows representative noise sources separated by ICA in real fNIRS experiments. These sources can be interpreted as follows:

(A), (B), and (C): Physiological noise sources: The frequency of the temporal modes was around 0.1 Hz, and the spatial modes were diffuse, indicating that the sources were Mayer wave related.³⁰ Figure 8(b) shows a combined physiological noise source of both Mayer wave and heart beat with frequency peaks of around 0.1 and 1 Hz, respectively. Another source of physiological noise was respiration, with a frequency peak of around 0.3 Hz, as shown in Fig. 8(c).
(D) Motion artifact source: A short spike was found in its temporal mode (200 time samples) and was spatially localized in the right boundary of the probe montage, where the probe contact with the scalp is very sensitive to motion due to the tensile force of the holder.
(E) Noise source: This white-noise-like signal in a single channel suggests that the probe was not properly contacting the scalp.
(F) Task-induced common physiological noise sources: These N-shaped temporal modes were found to be scalp-vessel artifacts originating from the task-evoked vasomotion in the forehead.¹³ This spurious synchronization can be found in many dyads, suggesting a large influence on the hyperlinks computed using the sensor-level method.

Fig. 8

Representative noise sources separated by ICA in real fNIRS experiments. (a) Physiological noise source (Mayer wave); (b) physiological noise source (Mayer wave + heart beat); (c) physiological noise source (respiration); (d) motion artifact source; (e) noise due to poor contact; (f) sychronized physiological noise induced by a common task, black lines represent the onsets of the task.

Results derived from the two methods on three typical dyads are shown in Fig. 9. The hyperlink spatial pattern derived by averaging the spatial modes of the two selected SoI indicated that the location of the SoI was on the middle prefrontal cortex. However, the correlation maps derived by the sensor-level method were global, suggesting that the source-level method improved the specificity of hyperlink analysis over the sensor-level method. Group-level spatial patterns are shown in Fig. 10. For the source-level method, the channels with significant $t$ values (channels 5 and 9, one tail one-sample $t$ -test, $p < 0.05$ ) were within the medial prefrontal cortex [Fig. 10(a)]. There was also a significant channel (channel 10, one-tail one-sample $t$ -test, $p < 0.05$ ) in the mPFC found by the sensor-level method. However, the spatial pattern of sensor-level results was global in the PFC [Fig. 10(b)]. To compare the group hyperlink strength of the two methods, the hyperlink strength of channel 10 within the mPFC (channel 10) was taken as the hyperlink strength of the sensor-level method. The ICA-based source-level results showed a higher mean hyperlink strength than the sensor-level results.

Fig. 9

Comparison of results derived from the two methods on three typical dyads in real fNIRS experiments. Left: averaged spatial mode of SoI of ICA-based source-level results, mPFC is circled with a green dashed-line. Color bar represents the $z$ value of the averaged spatial mode. Right: correlation map derived from sensor-level results. Color bar indicates the correlation coefficents.

Fig. 10

Comparison of sensor and source-level spatial patterns in real fNIRS experiments. (a) Source-level $z$ -map, mPFC was circled with a green dashed line, significant channels within mPFC were circled with a blue solid line. (b) Sensor-level correlation map, mPFC was circled with a green dashed line, significant channels within mPFC were circled with a blue solid line. (c) Hyperlink strength derived by source-level and sensor-level methods, the error bar represents the standard error of the dyadic hyperlink strength.

4. Discussion and Conclusion

In this article, we proposed an ICA-based source-level hyperlink analysis method for two-person fNIRS neuroscience studies. Each interactor’s fNIRS data was first decomposed into a set of sources. Second, the SoI were selected. Finally, hyperlink between SoI of an interacting dyad was computed. Five widely used ICA algorithms for extracting SoI in simulative datasets were compared, and an increase in sensitivity and specificity for hyperlink analysis by our proposed method was demonstrated by both simulative and real two-person experiments.

To measure the potential increase in sensitivity for detecting hyperlinks by the ICA-based source-level hyperlink analysis method, a comprehensive set of noise sources, including physiological noise, motion artifacts, and channel specific instrumental noise, were simulated in simulative two-person datasets series 1. We manipulated the noise amplitudes of physiological and motion noise sources simultaneously. Due to noise contamination, the sensor-level method detected lower hyperlink strength in the true RoI than the ICA-based source-level method. In real fNIRS experiments, the types of noise mentioned above were also found by the source-level method, as shown in Figs. 8(a)–8(e). The group hyperlink strength calculated by the source-level method depicted a higher mean than that calculated by the sensor-level method. These results demonstrate the increase in sensitivity of hyperlink analysis by the source-level method.

The specificity of hyperlink detection is another challenge in two-person neuroscience.¹ Studies have found that physiological synchronization can happen during tasks with behavioral synchrony, such as among choir singers³¹ or between performers and related spectators in a fire-walking ritual.¹² These issues are even more important for two-person fNIRS studies because fNIRS is most sensitive to shallow tissue layers, whose hemodynamic signals contain mainly physiological fluctuations.¹³ We considered the above issues and designed a simulative two-person datasets series 2, where synchronized physiological noise, including heart rate, respiration, Mayer wave, and systemic physiological noises, was simulated. Our results show that the sensor-level method mostly found global spatial patterns. Being able to eliminate these synchronized physiological noise sources, the source-level method derived better spatial patterns. In the real two-person fNIRS experiment, spurious hyperlinks were also found by the source-level method, which were mainly due to the task-evoked vasomotion [Fig. 8(f)]. The hyperlinks found by the sensor-level method were also influenced by synchronized Mayer waves [Fig. 9(b)], and the global patterns shown in Figs. 9(b) and 10(b) suggest that the specificity of the sensor-level method was low. To summarize, both simulative and real two-person experiments indicated an increase in specificity by our proposed method.

The selection of the SoI is an important step in the source-level hyperlink analysis framework, which requires knowledge of the temporal or spatial features of the SoI. In our simulative two-person experiment, a template matching procedure was used to select the SoI. In the real fNIRS experiment, interactors attempted to synchronize their tapping by predicting each other’s behavior during the interaction; thus, the spatial modes in the mPFC, involved in the Theory of Mind or Mentalizing,³² were selected. Two things should be noted when using spatial features to select the SoI. First, channels with the same index or location in the montage across interactors do not necessarily record the same anatomical position, which is an issue in sensor-level channel-wise hyperlink analysis.³³ Thus, using common templates to select the SoI across interactors may cause a bias. Therefore, in future studies, interactor-specific templates should be used to select the SoI when anatomical information of each channel is available. Second, noise sources may also match the template well, so inspection of the temporal modes is required to exclude these. Temporal features can also be incorporated in the selection of the SoI. This may involve correlating temporal modes of sources to behavioral markers of interaction. However, explicit behavioral markers of natural social interaction may not always be available.¹^,³⁴ Thus, careful experimental design may be required to address this issue in future studies.

Many current two-person fNIRS studies also incorporate WTC in a sensor-level framework,⁶^,³⁵^,³⁶ which can analyze hyperlinks under different frequencies. Thus, synchronization of noise, such as heart beat and respiration, can be separated based on their specific frequencies. However, the signal recorded by fNIRS sensors also contains systemic physiological fluctuations that cover a broad frequency band.³⁰ At the same time, temporal events of natural social interaction are usually unpredictable and their frequency spectrum can overlap with that of systemic physiological fluctuations.¹^,³⁴ Therefore, the resulting spurious synchronized physiological noise cannot be discriminated using frequency-based methods only. ICA can separate neural sources from noise, regardless of the overlapping frequency, but only if the neural sources and noise sources are statistically independent from each other. After separation, noise sources can be identified and discarded according to their different spatial modes compared to neural sources.

To obtain neural SoI in this study, we compared five ICA algorithms, and second-order blind identification (SOBI) showed the best results in extracting SoI. The reason why SOBI outperformed other algorithms is that it focuses on the time structure of the signals. Specifically, it minimizes the cross-correlation between time courses.²⁰ The other four algorithms are generally based on prior knowledge about the probability distributions of signals, which are more likely to be affected by channel-specific Gaussian noise. For example, FastICA is based on maximizing the non-Gaussianity of components, which is more sensitive to channel-specific Gaussian noise because it can generate noise components having higher non-Gaussianity than SoI.³⁷ Thus, ICA algorithms considering Gaussian noise in their model could be considered in investigated studies.³⁸ On the other hand, Gaussian noise often results from bad contact by the probe or other hardware problems, influencing only a few channels. Thus, this issue can be addressed simply by excluding these channels before ICA decomposition. Another way of reducing Gaussian noise is to reduce the dimensionality of the datasets using PCA, before performing ICA.¹⁸ However, the number of dimensions to keep, i.e., how many sources to be retained, is another general question when using ICA. We retained 10 sources in simulative experiments, which is higher than the real number of sources. We also noticed that when more sources were retained, the time cost of algorithms became higher and the accuracy in extracting SoI by FastICA, JADE and MILCA worsened. More studies are required to investigate methods to estimate the number of sources.

Considering that most two-person neuroscience studies have mainly focused on the hyperlink between the same regions of two brains, SoI with the same spatial modes were selected and averaged in this study. The proposed framework also supports analyzing hyperlinks between different regions in a dyad by selecting SoI with different spatial modes. Moreover, being able to identify task-related functional networks,³⁹ accessing hyperlinks between the networks of two brains in social interaction may be an interesting avenue of research for future studies. Two additional interesting issues arise when we considered how we should hyperlink strength in a source-level hyperlink analysis framework. First, while we detected linear hyperlinks using Pearson correlation coefficient in this study, nonlinear hyperlinks may also exist during complex interactions. These may be detectable by a mutual information or maximal information coefficient. Second, directionality of the hyperlink, which may indicate the flow of information or the role of interactors in social interaction,⁴⁰ may be studied using cross-correlation or granger causality.²⁶ These changes can be easily incorporated in our proposed framework by substituting the correlation hyperlink index used in this study.

Appendices

Appendix:

Data Generation for Simulative Experiments

The temporal modes of all generated sources last 100 s with a sample frequency of 10 Hz, while spatial modes were $8 \times 8$ matrices containing spatial weights for the sources.

$D_{i}^{Neu}$

Temporal modes of neural sources, $T_{i}^{Neu}$ , were designed using pseudo-random events ( $δ$ function) convolved by the canonical HRF. The number of events was randomized from 1 to 12, and the times of events were also randomly selected within 100 s. Spatial modes of neural sources, $S_{i}^{Neu}$ , were made by setting the element at row 4, column 5 of the spatial matrix to 1 and setting all other elements to 0. Then, the matrix was smoothed by a $2 \times 2$ moving average spatial filter.

$D_{i}^{Phys}$

Temporal modes of physiological noise were generated by linear-combining sinusoidal waves with frequencies depicted in Table 1.

Table 1

Parameters of the generated physiological noise sources

Noise type	Frequency (Hz)	Amplitudes (std)
Systemic physiological noise	0.01–0.15	1
Mayer wave	$0.1 \pm 0.02$	0.2
Breathing	$0.2 \pm 0.03$	0.2
Heart beat	$1.1 \pm 0.1$	0.2

Spatial modes of physiological noise were simulated as Gaussian random noise assigned to each element of the spatial matrix and then smoothed by a $5 \times 5$ moving average spatial filter.

$D_{i}^{Motion}$

Temporal modes of motion artifacts were generated using triangular pulses whose amplitudes were 5 to 12 times larger than the amplitudes of $T_{i}^{phy}$ and whose span was 1 s. Motion artifacts randomly happened at any time during the 100 s with a total number from (for each simulated dataset) 1 to 10. Spatial modes of motion artifacts were simulated in the same way as spatial modes of physiological noise.

Source Code and Parameters of the Five ICA Algorithms

For the five ICA algorithms, only the parameters mentioned below were changed from their default values.

SOBI

Source: Implementation from EEGLAB: https://sccn.ucsd.edu/eeglab/download.php

FastICA

Source: http://www.cis.hut.fi/projects/ica/fastica

Parameter Value Description
$g$ tanh Chooses the nonlinearity used in the fixed-point algorithm
stabilization on This parameter controls whether the program uses the stabilized version of the algorithm.

Infomax

Source: Implementation from EEGLAB: https://sccn.ucsd.edu/eeglab/download.php

Parameter Value Description
extended 1 Automatically estimate the number of sub-Gaussian sources.

JADE

Source: http://perso.telecom-paristech.fr/~cardoso/Algo/Jade/jadeR.m

MILCA

Source: http://bsp.teithe.gr/members/downloads/Milca.html

Disclosures

The authors declare no relevant financial interests in the manuscript and no other potential conflicts of interest.

Acknowledgments

This work was supported by the National 973 Program (Grant No. 2014CB846100), the National Natural Science Foundation of China (Grants Nos. 61431002, 61273287 and 31221003), NCET (Grant No. 11-0046), the Major Project of the National Social Science Foundation (Grant No. 12&ZD228), and the National Key Scientific Instrument and Equipment Development Project of China (Grant No. 2012YQ120046).

References

1.

R. Hari et al., “Centrality of social interaction in human brain function,” Neuron, 88 (1), 181 –193 (2015). http://dx.doi.org/10.1016/j.neuron.2015.09.022 Google Scholar

2.

U. Hasson et al., “Brain-to-brain coupling: a mechanism for creating and sharing a social world,” Trends Cognit. Sci., 16 (2), 114 –121 (2012). http://dx.doi.org/10.1016/j.tics.2011.12.007 Google Scholar

3.

I. Konvalinka and A. Roepstorff, “The two-brain approach: how can mutually interacting brains teach us something about social interaction?,” Front. Hum. Neurosci., 6 1 –10 (2012). http://dx.doi.org/10.3389/fnhum.2012.00215 Google Scholar

4.

F. Babiloni and L. Astolfi, “Social neuroscience and hyperscanning techniques: past, present and future,” Neurosci. Biobehav. Rev., 44 76 –93 (2014). http://dx.doi.org/10.1016/j.neubiorev.2012.07.006 Google Scholar

5.

F. Scholkmann et al., “A new methodical approach in neuroscience: assessing inter-personal brain coupling using functional near-infrared imaging (fNIRI) hyperscanning,” Front. Hum. Neurosci., 7 813 (2013). http://dx.doi.org/10.3389/fnhum.2013.00813 Google Scholar

6.

X. Cui, D. M. Bryant and A. L. Reiss, “NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation,” Neuroimage, 59 (3), 2430 –2437 (2012). http://dx.doi.org/10.1016/j.neuroimage.2011.09.003 Google Scholar

7.

L. Holper et al., “The teaching and the learning brain: a cortical hemodynamic marker of teacher–student interactions in the Socratic dialog,” Int. J. Educ. Res., 59 1 –10 (2013). http://dx.doi.org/10.1016/j.ijer.2013.02.002 Google Scholar

8.

J. Jiang et al., “Leader emergence through interpersonal neural synchronization,” Proc. Nat. Acad. Sci., 112 (14), 4274 –4279 (2015). http://dx.doi.org/10.1073/pnas.1422930112 Google Scholar

9.

F. Scholkmann et al., “A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology,” Neuroimage, 85 6 –27 (2014). http://dx.doi.org/10.1016/j.neuroimage.2013.05.004 Google Scholar

10.

X. Cui, S. Bray and A. L. Reiss, “Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics,” Neuroimage, 49 (4), 3039 –3046 (2010). http://dx.doi.org/10.1016/j.neuroimage.2009.11.050 Google Scholar

11.

H. Obrig et al., “Spontaneous low frequency oscillations of cerebral hemodynamics and metabolism in human adults,” Neuroimage, 12 (6), 623 –639 (2000). http://dx.doi.org/10.1006/nimg.2000.0657 Google Scholar

12.

I. Konvalinka et al., “Synchronized arousal between performers and related spectators in a fire-walking ritual,” Proc. Natl. Acad. Sci. U. S. A., 108 (20), 8514 –8519 (2011). http://dx.doi.org/10.1073/pnas.1016955108 Google Scholar

13.

E. Kirilina et al., “The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy,” Neuroimage, 61 (1), 70 –81 (2012). http://dx.doi.org/10.1016/j.neuroimage.2012.02.074 Google Scholar

14.

A. V. Medvedev et al., “Event-related fast optical signal in a rapid object recognition task: improving detection by the independent component analysis,” Brain Res., 1236 145 –158 (2008). http://dx.doi.org/10.1016/j.brainres.2008.07.122 Google Scholar

15.

T. Katura et al., “Extracting task-related activation components from optical topography measurement using independent components analysis,” J. Biomed. Opt., 13 (5), 054008 (2008). http://dx.doi.org/10.1117/1.2981829 Google Scholar

16.

H. Zhang et al., “Functional connectivity as revealed by independent component analysis of resting-state fNIRS measurements,” Neuroimage, 51 (3), 1150 –1161 (2010). http://dx.doi.org/10.1016/j.neuroimage.2010.02.080 Google Scholar

17.

C. Jutten and J. Herault, “Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture,” Signal Process., 24 (1), 1 –10 (1991). http://dx.doi.org/10.1016/0165-1684(91)90079-X Google Scholar

18.

A. Hyvarinen, J. Karhunen and E. Oja, Independent Component Analysis, John Wiley & Sons, New York (2004). http://dx.doi.org/10.1002/0471221317 Google Scholar

19.

A. Hyvärinen, “Fast and robust fixed-point algorithms for independent component analysis,” IEEE Trans. Neural Netw., 10 (3), 626 –634 (1999). http://dx.doi.org/10.1109/72.761722 Google Scholar

20.

A. Belouchrani et al., “Second order blind separation of temporally correlated sources,” in Proc Int Conf, 2 –7 Google Scholar

21.

A. J. Bell and T. J. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution,” Neural Comput., 7 (6), 1129 –1159 (1995). http://dx.doi.org/10.1162/neco.1995.7.6.1129 Google Scholar

22.

J. F. Cardoso and A. Souloumiac, “Blind beamforming for non-Gaussian signals,” IEE Proc. F: Radar Signal Process., 140 (6), 362 (1993). http://dx.doi.org/10.1049/ip-f-2.1993.0054 Google Scholar

23.

H. Stögbauer et al., “Least-dependent-component analysis based on mutual information,” Phys. Rev. E, 70 (6), 066123 (2004). http://dx.doi.org/10.1103/PhysRevE.70.066123 Google Scholar

24.

M. D. Greicius et al., “Default-mode network activity distinguishes Alzheimer’s disease from healthy aging: evidence from functional MRI,” Proc. Natl. Acad. Sci. U. S. A., 101 (13), 4637 –4642 (2004). http://dx.doi.org/10.1073/pnas.0308627101 Google Scholar

25.

J. Markham, B. R. White and B. W. Zeff, “Blind identification of evoked human brain activity with independent component analysis of optical data,” Hum. Brain Mapp., 30 2382 –2392 (2009). http://dx.doi.org/10.1002/hbm.v30:8 Google Scholar

26.

L. Holper, F. Scholkmann and M. Wolf, “Between-brain connectivity during imitation measured by fNIRS,” Neuroimage, 63 (1), 212 –222 (2012). http://dx.doi.org/10.1016/j.neuroimage.2012.06.028 Google Scholar

27.

M. J. McKeown et al., “Analysis of fMRI data by blind separation into independent spatial components,” Hum. Brain Mapp., 6 (3), 160 –188 (1998). http://dx.doi.org/10.1002/(SICI)1097-0193(1998)6:3<>1.0.CO;2-S Google Scholar

28.

N. H. Berivanlou, S. K. Setarehdan and H. Ahmadi Noubari, “Evoked hemodynamic response estimation using ensemble empirical mode decomposition based adaptive algorithm applied to dual channel functional near infrared spectroscopy (fNIRS),” J. Neurosci. Methods, 224 13 –25 (2014). http://dx.doi.org/10.1016/j.jneumeth.2013.12.007 Google Scholar

29.

Y.-J. Zhang et al., “Determination of dominant frequency of resting-state brain interaction within one functional system,” PLoS One, 7 (12), e51584 (2012). http://dx.doi.org/10.1371/journal.pone.0051584 Google Scholar

30.

T. J. Huppert et al., “HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain,” Appl. Opt., 48 (10), D280 –D298 (2009). http://dx.doi.org/10.1364/AO.48.00D280 Google Scholar

31.

V. Müller and U. Lindenberger, “Cardiac and respiratory patterns synchronize between persons during choir singing,” PLoS One, 6 (9), e24893 2011). http://dx.doi.org/10.1371/journal.pone.0024893 Google Scholar

32.

U. Frith, “Mind blindness and the brain in autism,” Neuron, 32 (6), 969 –979 (2001). http://dx.doi.org/10.1016/S0896-6273(01)00552-9 Google Scholar

33.

A. L. Reiss et al., “NIRS-based hyperscanning reveals inter-brain neural synchronization during cooperative jenga game with face-to-face communication,” Front. Hum. Neurosci., 10 (March), 267 –11 (2016). http://dx.doi.org/10.3389/fnhum.2016.00082 Google Scholar

34.

P. Montague, “Hyperscanning: simultaneous fMRI during linked social interactions,” Neuroimage, 16 (4), 1159 –1164 (2002). http://dx.doi.org/10.1006/nimg.2002.1150 Google Scholar

35.

X. Cheng, X. Li and Y. Hu, “Synchronous brain activity during cooperative exchange depends on gender of partner: a fNIRS-based hyperscanning study,” Hum. Brain Mapp., 36 (6), 2039 –2048 (2015). http://dx.doi.org/10.1002/hbm.22754 Google Scholar

36.

L. Dommer et al., “Between-brain coherence during joint n-back task performance: a two-person functional near-infrared spectroscopy study,” Behav. Brain Res., 234 (2), 212 –222 (2012). http://dx.doi.org/10.1016/j.bbr.2012.06.024 Google Scholar

37.

A. Hyvarinen et al., “Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis,” Neuroimage, 49 (1), 257 –271 (2010). http://dx.doi.org/10.1016/j.neuroimage.2009.08.028 Google Scholar

38.

A. Hyvarinen and A. Hyvärinen, “Fast ICA for noisy data using Gaussian moments,” in 1999 IEEE International Symposium on Circuits and Systems, VLSI, 57 –61 (1999). http://dx.doi.org/10.1109/ISCAS.1999.777510 Google Scholar

39.

V. D. Calhoun et al., “fMRI activation in a visual-perception task: network of areas detected using the general linear model and independent components analysis,” Neuroimage, 14 (5), 1080 –1088 (2001). http://dx.doi.org/10.1006/nimg.2001.0921 Google Scholar

40.

E. Bilek et al., “Information flow between interacting human brains: identification, validation, and relationship to social expertise,” Proc. Natl. Acad. Sci. U. S. A., 112 (16), 5207 –5212 (2015). http://dx.doi.org/10.1073/pnas.1421831112 Google Scholar

Biographies for the authors are not available.

Citation Download Citation

Yang Zhao, Rui-Na Dai, Xiang Xiao, Zong Zhang, Lian Duan, Zheng Li, and Chao-Zhe Zhu "Independent component analysis-based source-level hyperlink analysis for two-person neuroscience studies," Journal of Biomedical Optics 22(2), 027004 (8 February 2017). https://doi.org/10.1117/1.JBO.22.2.027004

Received: 20 September 2016; Accepted: 13 January 2017; Published: 8 February 2017

Access the abstract

JOURNAL ARTICLE
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 8 scholarly publications.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Sensors

Independent component analysis

Neuroscience

Biological research

Computer simulations

Brain

Interference (communication)

1.

Introduction

2.

Theoretical Analysis and Method

2.1.

Source-Level Hyperlink Analysis Framework

2.1.1.

Single brain independent component analysis decomposition

2.1.2.

Sources of interaction selection

2.1.3.

Hyperlink computation

2.2.

Sensor-Level Hyperlink Analysis

Fig. 1

2.3.

Simulative Two-Person Datasets

Fig. 2

2.3.1.

Simulative two-person datasets series 1: hyperlink contaminated by interactor-specific noise

2.3.2.

Simulative two-person datasets series 2: spurious link due to synchronized physiological noise

2.4.

Real Two-Person Interaction Experiment

2.4.1.

Protocol

Fig. 3

2.4.2.

Data analysis

3.

Results

3.1.

Simulative Two-Person Experiment

3.1.1.

Single brain decomposition

Fig. 4

3.1.2.

Comparison with sensor-level method

Simulative two-person dataset series 1

Fig. 5

Fig. 6

Simulative two-person datasets series 2

Fig. 7

3.2.

Real Two-Person fNIRS Experiment

Fig. 8

Fig. 9

Fig. 10

4.

Discussion and Conclusion

Appendices

Appendix:

Data Generation for Simulative Experiments

DiNeu

DiPhys

Table 1

DiMotion

Source Code and Parameters of the Five ICA Algorithms

Disclosures

Acknowledgments

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years

$D_{i}^{Neu}$

$D_{i}^{Phys}$

$D_{i}^{Motion}$