1.

Introduction

Increasing concern for overall well being, aging population, rising number of people with the limited ability for independent dwelling, and rising cost of healthcare services present major challenges of modern society. Demographic trends suggest that by 2050 approximately 11% of the world population will be 80 or older.¹ Instead of hospitalization or institutionalization, the elderly people with limited abilities can be assisted in their home environment by using different smart devices. The concept of smart home must be cost effective and able for improving home care in an unobtrusive way. Such an approach leads to greater independence, maintains good health, and prevents social isolation.² Inexpensive low-power sensors, embedded processors, and wireless communications are available technologies that are typical building blocks for unobtrusive home healthcare.³

Advanced sensors have boosted signal processing methods for unobtrusive human vital signs detection, mostly heartbeat and respiration. Sensors may be coupled with the medium that a subject touches for a longer- or shorter-time interval, e.g., the bed or garments. Special high-impedance electrodes have been used, e.g., with no direct contact with the skin.⁴ The so-called load-cell pressure sensors also mean an option for detecting heartbeat and respiration.⁵ Another solution is the pressure sensor installed under the pillow.⁶ Static charge sensitive bed⁷ generates signals that contain components of motion, breathing, and heartbeat. The electromechanical film (EMF) force sensor has been developed⁸ as a thin biaxially oriented plastic film coated with the electric conductive layers that generate voltage proportional to the external pressure. An important advantage of these sensors is that they can also be applied as wearable sensors, such as parts of body area networks.

A deeper insight into the sensors’ properties sorts out the most adequate physiological features to be observed in unobtrusive home care. Respiration can be assessed by extracting the acceleration or motion of a person’s chest.^9,10 Authors of Ref. 11 proposed a technique for monitoring the respiration by changes of coupling between the input and the output sensor’s electrodes when the exhaled air induces micro-condensation of vapor particles. Heartbeat perception can be based on mechanical activity of cardiac muscle, the so-called ballistocardiography^12,13 or mechanocardiography,¹⁴ or on heart sounds, the so-called phonocardiography.¹⁵ These have been measured and identified by the Michelson fiber-optic interferometer in Refs. 1617.18.–19. To achieve adequate sensitivity, conventional interferometric principles were utilized depending on single-mode optical fibers. This also means relatively complex fiber coupled sources that operate at telecom wavelengths, which compromises cost efficiency of such systems. For a wider usage in home healthcare systems, a less expensive solution is needed. Speckle interferometry is one of the possible alternative approaches.

Fiber-optic speckle interferometer combines simple construction based on a minimum number of low-cost components readily used in consumer electronics systems with unobtrusiveness and the ability to detect a wide range of human vital-sign features simultaneously. We built a prototype sensor for heartbeat and respiration detection and published preliminary results in Ref. 20. Our latest research was focused on signal processing algorithms to extract human vital signs from speckle interferometric signals.

The article is organized as follows. In Sec. 2, fiber-optic sensor design and the methodology for data analyses are described. The experimental setup and protocol are explained in Sec. 3. Results are discussed in Sec. 4, and Sec. 5 concludes the article.

2.

Monitoring of Human Vital Signs by Using Fiber-Optic Speckle Interferometry

Speckle patterns can be observed in laser light at the output of coherently illumined multimode optical fibers. These speckle patterns are due to the interference among propagating fiber modes and are highly sensitive to the external fiber perturbations, such as strain or axial load. When the optical fiber is in direct or indirect contact with the human body, speckle images reflect mechanical and acoustic influence of human activity. Such a sensor can be unobtrusively embedded into various home devices and objects that come in physical contact with the observed person during daily living situations. The most common are beds, chairs, doors, handles of household appliances, floor carpets, slippers, shoes, etc.

2.1.

Sensor Design

The proposed sensor consists of only three low-cost, opto-electronics components already widely used by current consumer electronics market: (1) a 650-nm laser diode intended for use in digital-video-disc (DVD) systems, (2) section of standard plastic optical fiber (POF) with 980/1000 μm core/cladding and numerical aperture of 0.5, and (3) sampling device with 102-element linear optical sensor array, configured as shown in Fig. 1. The proposed system forms a speckle interferometer. To extract and distinguish useful life-signs information, a set of custom-tailored algorithms were developed to process data obtained from the linear array.

Fig. 1

Principle schema of speckle interferometer with an illustration of the interferometric signal transforms and processing methods.

The 650-nm laser diode was butt-coupled to the fiber without using any additional optical components. Few conditions need to be fulfilled in order to allow the usage of this simple proposed configuration. The large numerical aperture of commercial POFs produces large delay among the modes of the fiber. To provide interference of modes and consequent speckle generation at the fiber output, the coherence of the source shall be larger than the delay among interfering modes.

To obtain fully coherent conditions, where all modes of the fiber are able to interfere with each other, the delay among any pair of modes shall be less than the coherence length of the source, which can be approximated as $c{t}_d$, where $c$ stands for the vacuum speed of light and $t_d$ for the modal delay

Eq. (1)

$$L_c\ge c{t}_d.$$

By using well known expressions for $t_d$ in step index multimode fibers,²¹ we obtain

Eq. (2)

$$L_c\ge L{n}_{\mathrm{co}}\mathrm{\Delta }\text{or}{L}_c\ge \frac{L}{2n_{\mathrm{co}}}{\mathrm{NA}}^2,$$

where $L_c$ is the coherence length of the source, $L$ is the total fiber length, $\mathrm{\Delta }=(n_{\mathrm{co}}^2-n_{\mathrm{cl}}^2)/(2n_{\mathrm{co}}^2)$, NA is the numerical aperture of the fiber, and $n_{\mathrm{co}}$ and $n_{\mathrm{cl}}$ are the core and cladding indexes, respectively.

For the fiber with $\mathrm{NA}=0.5$, the coherence length of the source shall be about $0.086L$, which is a relatively stringent demand for Fabry–Perot laser source (each meter of sensing fiber requires almost an additional centimeter of the source coherence length). While DVD diodes support many longitudinal modes, one of these should dominate to fulfill the demanded coherence (highly multimode diodes were found inappropriate for this application). The full coherence among all the propagating modes is, however, unnecessary to obtain useful speckle pattern and the above condition can be thus partially relaxed. It is, however, important to recognize that the building of speckle interferometer using conventional POF requires relatively coherent sources. Thus, in a practical system, a laser diode coherence shall reside within a centimeter range (e.g., dominant laser line width shall be at the order of or below 0.05 nm), while the active length of the sensing POF shall be limited to a maximum of a few meters. A good, nearly single mode, low-cost DVD Fabry–Perot laser diode can meet this requirement, but it is also important to reduce the length of the POF to the minimum length required to achieve a desired sensing effect.

Speckle pattern images at the fiber output must be acquired by an imaging optical sensor. In general, it would be a digital camera, but in proposed low-cost solution a silicon linear optical sensor array, such as TSL3301,²² means a better trade-off. Losing the information on the whole speckle image is compensated by the sensor’s lower price, higher sampling frequency, and easily controlled settings. The array consists of 102 pixels, each measuring $77\mu \text{m}\times 85\mu \text{m}$, and was placed at the distance of 50 mm from the fiber end. The ATmega168 microcontroller was used to control and acquire data from TSL3301. An appropriate program was written to manage settings of the sensor array, such as amplification, time of integration, and sampling frequency. In the proposed application, the integration times can vary between 2 and 150 μs and are set up in the initial calibration to achieve optimum contrast of speckle images. A built-in A/D converter has 8-bit resolution, which proved acceptable in our experiments. The sampling frequency was set equal to 1000 lines of $102\text{pixels}/\mathrm{s}$.

2.2.

Data Model and Processing

Mechanical perturbations of optical fiber provoke changes in a speckle pattern and, consequently, changes in a time sequence of images $\mathbf{I}=\{I_0,I_1,\dots ,I_{N-1}\}$, where $I_n$ is the $n$’th image and $N$ stands for the number of images. The sensor described in the previous subsection is based on a linear optical array generating one-row images of dimensions $1\times L$, $L=102$. On a short-time scale, it can be considered that the spackle-image pattern is stable if no external forces press against the optical fiber, and vary according to the external forces when applied.

Denote a pixel value at coordinate $x$ in image $I_n$ by $p_n(x)$ and define a function $f$ that transforms pixels of two or more consecutive images into a one-dimensional (1-D) signal

Eq. (3)

$$s(n)=f_{m\in \mathbf{D}(x)\in \{0,\dots ,N-1\}}[p_m(x)],$$

where $\mathbf{D}$ stands for a set of indexes denoting the selected images. If function $f$ is properly chosen, signal $s(n)$ can discriminate different fiber-optic disturbances. Take, e.g., $f$ is an absolute sum of the differences of two consecutive images on a pixel level. When there is no external perturbation, this sum would approach 0, whereas it would rise to higher values when speckle patterns change due to pressures on the optical fiber.

The transform of speckle images into 1-D signals helps to decrease high two-dimensional computational complexity of the vital signs analysis, when taking into account the total number of consecutive rows of pixels in a longer time interval. Detection of cardiac and respiratory activities is, therefore, built on the analysis of constructed 1-D signals. A principal schema of data model, including reference signals, is depicted in the right-hand side of Fig. 1.

2.2.1.

Speckle image preprocessing

The developed optical sensor and acquisition hardware are connected to a personal computer (PC) where all the data processing and analysis algorithms run. A thousand speckle images of size $1\times 102\text{pixels}$ are acquired per second. Sequences of images are preprocessed by transforming their features into 1-D signal by function $f$, as defined in Eq. (3). Various possible transforms have been published recently: (1) mean-absolute speckle-intensity variation,²³ (2) a method based on contrast difference,²⁴ and (3) phase shifting by using three,²⁵ four,²⁶ or five²⁷ consecutive images. We implemented all these methods and assessed their efficiency when monitoring human vital signs. The method with phase shifting of three consecutive images gave slightly better results than others.

From three images in the sequence, a particular phase-shift distribution is calculated. Suppose one-row images $I_n$, $I_{n+1}$, and $I_{n+2}$ and pixels $p_n(x)$, $p_{n+1}(x)$, and $p_{n+2}(x)$, then the phase shift ${\theta }_n(x)$ is computed as follows:

Eq. (4)

$${\theta }_n(x)={\tan }^{-1}\left[\sqrt{3}\frac{p_{n+2}(x)-p_{n+1}(x)}{2p_n(x)-p_{n+1}(x)-p_{n+2}(x)}\right].$$

Values of phase shifts ${\theta }_n(x)$ are assumed proportional to the level of perturbation on the optical fiber. Gaussian zero-mean random noise is expected in image pixels $p_n(x)$, so the distortions of ${\theta }_n(x)$ tend to be zero-mean. A decrease of random phase-shift distortions can be obtained by averaging ${\theta }_n(x)$ over $x$, which results in 1-D features we are looking for

Eq. (5)

$$s(n)=\sum _{x=0}^{L-1}{\theta }_n(x),0\le n\le N-3$$

and the signal sampling frequency equals 1000 Hz, as it was derived from the linear-array images temporally sampled by this frequency.

2.2.2.

Heartbeat detection

Speckle images $\mathbf{I}$ mirror the changes caused by fiber-optic perturbations, among others, and also the phenomena under the consideration, i.e., heartbeat and breathing. Image preprocessing with the selected transform function $f$ guarantees the observed vital signs can be detected from the generated 1-D signal $s(n)$; $n=0,\dots ,N-3$. Both the heartbeat and breathing are periodic with rates in well-known ranges, with their minima and maxima limited by the human physiology. Therefore, a decomposition of $s(n)$ to the constituent vital functions can be based on separating their frequency bands. This involves spectral analysis that can be accomplished by Fourier transformation, wavelet transform, digital filtering, etc. As the heartbeat and respiration detection are related to only two well-defined frequency intervals, we decided to implement the band-pass filtering.

It is generally known that human heart rate varies between 45 and 70 beats per minute (bpm) during sleeping and up to 210 bpm under high physical efforts. Our research focused on monitoring the vital signs under normal physical efforts up to 120 bpm. Signal $s(n)$, generated from speckle images, contains frequency components induced by the mechanical activity of the heart. In our case, the limits for the heart rate were set equal to 0.75 and 2 Hz. Filtering by a Butterworth band-pass filter of order 3 and cut-off frequencies 0.75 and 2 Hz was deployed to extract the heartbeat component.

In filtered version of $s(n)$, the signal amplitude varies according to the heart rate. By using peak detection method, local maxima correspond to cardiac systoles. Signal shifts introduced by filtering were eliminated by using zero-phase filters.

2.2.3.

Respiration detection

A very similar approach was deployed for the respiration detection. Human respiration normally ranges between 5 and 60 per min, so the lower and upper frequencies are 0.08 and 1 Hz. A partial overlap with the heartbeat frequency contents is evident between 0.75 and 1 Hz. However, this does not cause misinterpretations because the heartbeat and respiration frequencies are proportionally related. Higher heart rate normally implies higher respiration rate and vice versa. When the respiration rate moves toward its upper limit, the heart rate cannot be maintained at its lowest limit. This means no frequency overlap of the two activities can happen in practice.

Unlike the heartbeat, the respiration influence on speckle images $\mathbf{I}$ and, thus, on signal $s(n)$ is much more unpredictable. According to our observations, the main reason is human control of the muscles involved in the respiration. While cardiac muscle cannot be controlled and produces rhythmic, equally strong, and comparable responses in fiber-optic system, at least on a short-time scale, breathing intensity can vary a great deal and can be even very shallow when a person is not relaxed. Any irregular activation of any muscles causes additional mechanical perturbations on the optical fiber, whose frequency contents in signal $s(n)$ may degrade the respiration frequency band. Using a single digital filter did not result in reliable signal decomposition. Therefore, we had recourse to the wavelet transform.

We used Morlet-wavelet-based continuous wavelet transform. To cover the frequency range of respiration, scale factors between 2000 and 8000 in steps of 100 were analyzed. An example of obtained time-scale representation is depicted in Fig. 2.

Fig. 2

Time-scale representation of 2-min long signal $s(n)$ with varying respiration frequencies exhibiting major energy between scales 2000 and 8000. The Morlet mother wavelet was used.

Inhaling and exhaling disturb an optical fiber mechanically and induce speckle changes at breathing frequencies. These changes provoke highest variations in speckle-image-based signal $s(n)$, which contribute local maxima in time-scale representation. Due to limited time and frequency resolution, the observed energy concentrations are spread over a range of time instants and scales. It is therefore necessary to apply some postprocessing in order to assess the most probable time of appearance of the phenomena looked for. The construction of maxima lines is adopted most often. We found out experimentally that in our case a simpler approach is entirely feasible.

We took maximum values from time-scale representation in every time instant and build up maxima signal $s_t(n)$

Eq. (6)

$$s_t(n)=\underset{a}{\max }{W}_s(n,a),$$

where $W_s$ stands for the time-scale representation of signal $s(n)$ at time instants $n$ and scales $a$.

Signal $s_t(n)$ was additionally smoothed by the third-order moving-average filter of 1000-sample window length. Signal shifts introduced by smoothing and wavelet transform were eliminated. This guaranteed in all experiments we performed unique signal extremes coinciding with respiration events: maxima correspond to exhaling and minima to inhaling.

3.

Experimental Results

3.1.

Experimental Protocol

Experimental protocol was design to verify proposed detection of heartbeat and respiration. Ten healthy nonhospitalized persons participated in the experiments. POF with the length of 6 m was inserted and spirally twisted in a thin mattress. Participants were asked to lie supine on the mattress and fiber-optic signals were acquired by a custom-made sampling device (see Fig. 1). For comparison reasons, the experiment duration was 2 min, while tested persons were supposed not to move and breathe normally.

Experimental measurements were accompanied by referential signals taken simultaneously with interferometric speckle images. Hardware synchronization was built in our sampling device.

3.1.1.

Referential signals

Referential signals for heartbeat were obtained by a standard Schiller ECG device. Time locations of R waves indicate heartbeats and were determined by Pan-Tompkins QRS detection algorithm.²⁸ An example of 30-s long reference ECG signal is depicted in the top panel of Fig. 3. Detected R waves are denoted by circles.

Fig. 3

Referential ECG signal with detected R waves (top) and cardiac interferometric signal with identified local maxima (bottom). Signals were sampled at 1000 Hz and aligned in time by hardware synchronization. Circles above the interferometric signal correspond to time instants of referential R waves.

Airflow temperature through the nostrils was measured by a temperature sensor MCP9700A. The temperature increases while subject exhales and decreases while subject inhales. Thus, the extremes of the respiratory reference coincide with the extremes of the respiration detected from interferometric speckle images. Obtained smoothed respiratory referential signal is depicted in the top panel of Fig. 4. Local maxima, detected by peak detector, are marked with circles. Referential signals and interferometric images were acquired by our own sampling device at a sampling rate of 1000 Hz.

Fig. 4

Referential respiration curve with detected exhales (top) and respiratory interferometric signal with identified local maxima (bottom). Signals were sampled at 1000 Hz and aligned in time by hardware synchronization. Circles above the interferometric signal correspond to time instants of referential respiration.

3.2.

Results for Heartbeat and Respiration Detection

Referential points on R-wave peaks and respiration-curve maxima depicted by circles in Figs. 3 and 4 correlate well with the fiber-optic detected by heartbeats and respiration. We evaluated obtained detections statistically. The following events were counted:

• • number of the first interferometric events detected in intervals between two consecutive referential events (true positive—TP);

• • number of all false identified interferometric events between two consecutive referential events, i.e., more than one detection in any interval (false positive—FP), and

• • number of unidentified interferometric events between two consecutive referential events (false negative—FN).

The efficiency of proposed approach is assessed by

• • sensitivity: $S=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}100[\%]$;

• • precision: $P=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}100[\%]$;

• • mean delays between referential and interferometric events; and

• • standard deviation of delays between referential and interferometric events.

Ten young, healthy persons with an average age of $31.8\pm 8.1\text{years}$, height $178.8\pm 3.52\mathrm{cm}$, and weight $81\pm 12.11\mathrm{kg}$ participated in our experiments. Sensitivity and precision results are gathered in Table 1 for cardiac and in Table 2 for respiratory activity.

Table 1

Detection efficiency of cardiac activity based on speckle interferometry and proposed method for 10 persons.

	TP	FP	FN	P (%)	S (%)
Person 1	143	0	0	100.0	100.0
Person 2	143	7	0	95.3	100.0
Person 3	172	0	0	100.0	100.0
Person 4	157	2	2	98.7	98.7
Person 5	168	0	0	100.0	100.0
Person 6	185	0	3	100.0	98.4
Person 7	192	0	0	100.0	100.0
Person 8	154	1	0	99.4	100.0
Person 9	160	0	0	100.0	100.0
Person 10	149	1	1	99.3	99.3
Average	$164\pm 16.8$	$1.2\pm 2.2$	$0.7\pm 1.1$	$98.8\pm 1.5$	$99.4\pm 0.6$

Table 2

Detection efficiency of respiration activity based on speckle interferometry and proposed method for 10 persons.

	TP	FP	FN	P (%)	S (%)
Person 1	33	2	3	94.3	91.7
Person 2	25	0	2	100.0	92.6
Person 3	24	1	2	96.0	92.3
Person 4	23	0	1	100.0	95.8
Person 5	18	0	0	100.0	100.0
Person 6	16	0	0	100.0	100.0
Person 7	26	1	2	96.3	92.9
Person 8	28	1	1	96.6	96.6
Person 9	22	1	1	95.7	95.7
Person 10	24	0	1	100.0	96.0
Average	$24\pm 4.8$	$0.6\pm 0.7$	$1.3\pm 0.9$	$97.9\pm 2.3$	$95.3\pm 3$

Variance of the time delays assessment between detected and referential events is important to infer on the methods’ accuracy. The mean and standard deviation of the delays are depicted in Figs. 5 and 6, where circles stand for means and vertical bars stand for standard deviations. Both figures show the results obtained in 10 tested persons.

Fig. 5

Means and standard deviations of delays between referential and interferometric heartbeats. Values of means are indicated by circles and values of standard deviations by vertical bars.

Fig. 6

Means and standard deviations of delays between referential and interferometric respiration events. Values of means are indicated by circles and values of standard deviations by vertical bars.

Computational complexity of proposed approach, including speckle-image and signal processing, allows for operation in real time. A laptop PC with 2.8 GHz 2-core Intel processor and 4 GB of memory processed 2-min long experimental data for about 9 s.

4.

Discussion

The proposed system for unobtrusive monitoring of human vital signs, based on POF, achieved high efficiency and accurate results, as shown in Tables 1 and 2. We tested its efficiency with different fiber lengths. Referring to Eq. (2), very long multimode fibers do not generate speckle patterns. On the other hand, short fibers lose an important sensitivity for weak perturbations of human vital signs. We found the best trade-off between 4 and 8 m of the fiber length. All the results reported in this article were obtained with a 6-m long fiber.

Average heartbeat sensitivity of $99.4\%\pm 0.6\%$ and precision of $98.8\%\pm 1.5\%$ prove the efficiency of the proposed measurement system and detection method. The same can be considered for means and standard deviations of delays between interferometric and referential heartbeat events. Results in Fig. 5 confirm the delays and their deviations, $116.6\pm 55.5\mathrm{ms}$ on average, and are in agreement with the previously reported heartbeat detection by using the Michelson interferometer.^18,29 Obtained delays also agree with the physiological measurements in Ref. 30.

The results from Table 2 show a lower efficiency of respiration detection compared to the heartbeat detection. However, average sensitivity of $95.3\%\pm 3\%$ and precision of $97.9\%\pm 2.3\%$ are comparable with the observations based on the Michelson interferometer.¹⁸ Means and standard deviations of delays in respiration events, as shown in Fig. 6, yield $1299.2\pm 437.3\mathrm{ms}$. Less accurate results may be attributed to fewer respiration events. In 2-min long experiments, the average number of heartbeats was 164, while an average number of exhalations was estimated to be 24. Hence, outliers corrupt the respiration statistics about seven times heavier than the heartbeat statistics. This can also be seen from the mean delays between detected heartbeats and exhalations compared to their referential counterparts. Obtained standard deviation of delays, $\pm 55.5\mathrm{ms}$, for heartbeats implies 7.6% variation according to the mean inter-beat interval, i.e., 731.7 ms, while the respiration standard deviation of $\pm 437.3\mathrm{ms}$ exceeds 8.7% variation according to the mean respiration period. The reasons for worse detections in respiration may be explained by the ability of humans to consciously control their respiration muscles. If tested persons are unfamiliar with the experimental environment or bothered by any other reason, they cannot feel relaxed and their respiration is affected. Also their muscle tension cause additional perturbations on optical fiber. These influence the analyzed interferometric signals and introduce more variability.

5.

Conclusion

A cost-efficient system based on POF for unobtrusive monitoring of human vital signs was proposed. Low-cost components, such as low-cost DVD Fabry–Perot laser diode, POF, and low-cost linear optical sensor array TSL3301, were used to build speckle interferometer. Transformation of obtained speckle images to 1-D signals proved most efficient when based on phase-shifting method with three sequential images. Digital filtering and wavelet transform using the Morlet wavelet extract signal local extremes that correspond to cardiac and respiratory activities when the scales are determined according to the proper frequency contents of both physiological phenomena.

Obtained results prove the sensitivity, precision, and accuracy of proposed speckle-interferometric sensor are suitable for a reliable, real-time monitoring of human vital signs. High-cost efficiency qualifies it as a universal sensor for in-home health care and surveillance in the widest sense.

Further research will address two related issues. First, monitoring of vital signs will be intentionally aggravated by measurements in noisy places, places with many people, etc. This is especially important to improve algorithms to cope with hospital environments and rehabilitation centers, where various disturbances are present. Second, some patients have cardiac or/and respiratory abnormalities, which have still not been verified with the proposed system enough. In the future, also the implementation of speckle interferometer needs further clinical assessments and improvements to increase its robustness.