Elsevier

Computers in Biology and Medicine

Volume 80, 1 January 2017, Pages 158-165
Computers in Biology and Medicine

Monitoring infants by automatic video processing: A unified approach to motion analysis

https://doi.org/10.1016/j.compbiomed.2016.11.010Get rights and content

Highlights

  • Unified approach to video-processing detection of specific neonatal diseases.

  • Relevant motion signal is extracted to analyze infant movements.

  • Presence/absence of periodic movements is exploited to detect clonic seizures or apneas.

  • Maximum-likelihood criterion is used to estimate periodicity of the movements.

  • System performance on real patients are compared with medical gold standard devices.

Abstract

A unified approach to contact-less and low-cost video processing for automatic detection of neonatal diseases characterized by specific movement patterns is presented. This disease category includes neonatal clonic seizures and apneas. Both disorders are characterized by the presence or absence, respectively, of periodic movements of parts of the body—e.g., the limbs in case of clonic seizures and the chest/abdomen in case of apneas. Therefore, one can analyze the data obtained from multiple video sensors placed around a patient, extracting relevant motion signals and estimating, using the Maximum Likelihood (ML) criterion, their possible periodicity. This approach is very versatile and allows to investigate various scenarios, including: a single Red, Green and Blue (RGB) camera, an RGB-depth sensor or a network of a few RGB cameras. Data fusion principles are considered to aggregate the signals from multiple sensors. In the case of apneas, since breathing movements are subtle, the video can be pre-processed by a recently proposed algorithm which is able to emphasize small movements. The performance of the proposed contact-less detection algorithms is assessed, considering real video recordings of newborns, in terms of sensitivity, specificity, and Receiver Operating Characteristic (ROC) curves, with respect to medical gold standard devices. The obtained results show that a video processing-based system can effectively detect the considered specific diseases, with increasing performance for increasing number of sensors.

Introduction

Monitoring neonatal movements may be exploited to detect symptoms of specific disorders. In fact, some severe diseases are characterized by the presence or absence of rhythmic movements of one or multiple body parts. Clonic seizures and apneas can be identified by sudden periodic movements of specific body parts or by the absence of periodic breathing movements, respectively.

Seizures are clinically defined as paroxysmal alterations of the neurological functions (i.e., behavioral, motor or autonomic function) and represent a distinctive symptom of acute brain dysfunction in the newborn [1]. Leading causes of neonatal seizures are intracranial haemorrhage, hypoxic-ischaemic encephalopathy and sepsis. As major symptoms of acute central nervous system impairment, almost 80% of these paroxysmal events occur in the first two days of life [2]. The estimated incidence of seizures is between 1‰ and 3.5‰ in full-term newborns and even higher in preterm infants [3]. Essentially, four main clinical seizure types can be recognized in neonates, namely: subtle, clonic, tonic and myoclonic [1]. Prompt and accurate detection of neonatal seizures is crucial to administer timely treatments that may prevent further seizure-induced brain damage. It is important to stress that the various categories of neonatal seizures are characterized by very different kinds of movements and require different analysis approaches. This article analyzes the monitoring of clonic seizures. Analysis systems for other types of neonatal seizures are discussed in the following articles: [4], [5] for subtle seizures, [6] for myoclonic seizures.

Apneas consist of the temporary absence of spontaneous respiration [7], which manifests itself by the lack of breathing movements for a certain time period. Seizures, cerebrovascular events [8] and congenital diseases, such as Congenital Central Hypoventilation Syndrome (CCHS) [9], are among the main causes of these events in the neonatal period. CCHS, also referred to as Ondine's curse, is a life-threatening disorder. Discovered as a disease caused by mutations in the paired-like homeobox 2B (PHOX2B) gene, CCHS manifests itself, in the neonatal period, especially during quiet sleep, with cyanosis, apnea events or even cardiorespiratory arrests [9]. CCHS is a relatively rare disorder: in fact, just approximately 1000 individuals with this condition have been identified. Researchers believe that some cases of Sudden Infant Death Syndrome (SIDS) may be caused by undiagnosed CCHS [10].

Monitoring of vital signs for the diagnosis of such type of diseases is possible almost exclusively in hospital environments. Currently, the standard monitoring systems are based on ElectroEncephaloGram (EEG) polygraphy and polysomnographic devices [11], composed of several wired sensors directly connected to the body of the patient: electrodes positioned on the scalp, chest and other body parts, elastic belt around the chest, nasal flow meter and pulse oximeter. The polysomnographic device allows to monitor the brain electrical activity, the cardiac and muscular activities, the respiratory movements, the breathing pattern and the blood oxygen saturation. These systems, typically used for short-term monitoring, are expensive, time-consuming, moderately invasive (especially for newborns), and require experienced medical staff, not always available full-time in a Neonatal Intensive Care Unit (NICU). For home monitoring, many systems are available, but they may require sensors attached to the body of the newborn (e.g., smart bed [12], wearable sensor system [13]). At the opposite, the objective of our research is to study a contact-less monitoring system. Thus, automatic, real-time and non-invasive equipment able to reliably recognize these diseases would be strategic in hospital environments, helping to monitoring 24/7 all the newborns present in a NICU, or even at home, in order to implement remote monitoring systems, reducing time and cost of hospitalization.

An attractive contact-less monitoring tool to automatically detect movement- or breathing-dependent diseases, such as seizures or breathing disorders, may rely on properly processing video signals, acquired through single or multiple video cameras. The movements of the newborn's body (e.g., limbs and chest), framed by the cameras, can be analyzed to detect specific behaviors which may be symptoms of neurological dysfunction.

In [14], the acquisition, through sophisticated video processing, of the motion strength was proposed as expedient to detect the presence of neonatal seizures. In [15], the authors used an optical flow-based technique to track and characterize the movements of a newborn during prolonged monitoring. Neural networks were then used to obtain a diagnosis based on a previous training phase. Taking into account the long monitoring time and the fact that the implementation of long reliable tracking of jerky movements of the newborn limbs may be very complex, this approach is not suitable for real-time detection and requires expensive hardware for accurate optical flow processing (especially for dense optical flow techniques). In [16], a real-time video processing-based approach to the detection of neonatal clonic seizures based on recognition of characteristic periodic movements, was proposed. This approach relies on a periodicity detector, based on hybrid pitch estimation, to analyze a motion signal extracted from the video. This method has also received attention in the medical literature [17].

In this paper, an improved method to estimate the periodicity of pathological movements, based on the use of the Maximum Likelihood (ML) criterion [18], is presented. In particular, motion signals from multiple digital cameras or depth-sensor devices (e.g., Kinect [19]) are extracted and properly processed in order to detect potential abnormal motor patterns. We propose a monitoring system based on the detection of pathological movements, characterized by the presence or absence of a significant periodic component (i.e. rhythmic movements). Thus, the field of application includes any disease presenting this type of symptoms: clonic seizures and CCHS are relevant examples. The novelty lies in combining known techniques (more or less recent) to create an innovative monitoring system that is, unlike existing systems, contact-less and low-cost. In fact, as an example, we were able to develop an Android application, denoted as Smartphone-based Contactless Epilepsy Detector (SmartCED), able to capture images through the phone camera and detect the occurrences of clonic seizures: the computing resources of modern smartphones have proven to be sufficient for our purpose [20]. This paper unifies and expands preliminary contributions [21], [22]. The approach behind these two studies, namely, detection and estimation of periodic movements, is common, so that a unified vision is here proposed.

The remainder of the paper is organized as follows. In Section 2, the method for the analysis of periodic movements is described. In Section 3, performance results are presented. Finally, in Section 4 conclusions are drawn.

Section snippets

Extraction of temporal motion signal

Extraction of a relevant motion signal from every sensor is the first key step of the proposed approach. The application to standard Red, Green and Blue (RGB) [23] cameras is initially presented, and the extension to depth sensors is later discussed.

We start considering a video (i.e., a sequence of frames) with sampling period T. Frames are numbered as i=1,2,, specifying the indexes related to time instants as integer multiples of T. Each frame is described by a matrix of W×H pixels. The

Performance analysis

The acquisition system used at the NICU of the University Hospital of Parma and some illustrative examples of images acquired from a few cameras are shown in Fig. 4. The system is composed of three cameras: the first two cameras are arranged orthogonally in order to frame the baby by the front and the side, the third one is attached to the cradle to frame the face of the newborn [4]. In the following, the use of the first camera only will be initially considered. Then, performance results using

Conclusion

This paper describes a unified wire-free, low-cost, non-invasive technique for automatic detection of abnormal paroxysmal events in newborns, namely, clonic seizures and life-threatening apnea events. According to the proposed approach, motion signals are extracted from RGB cameras or depth sensors in order to estimate whether a periodic component is present, which is the characteristic feature of clonic seizures or breathing movements. In the case of apnea detection, the video may be

Standard protocol approvals, registrations, and patient consents

In accordance with current practice at our Institution, an informed consent form was signed by parents of each newborn patient, and the informed consent was placed in the patients' hospital chart. The Ethic Local Committee approval to perform retrospective studies was obtained.

Conflict of interest statement

None declared.

References (33)

  • G.M. Kouamou Ntonfo, G. Ferrari, F. Lofino, R. Raheli, F. Pisani, Extraction of video features for real-time detection...
  • Mosby, Mosby's Medical Dictionary, 10th edition, Elsevier, Saint Louis, MO, USA,...
  • J.J. Tramonte et al.

    Temporal lobe hemorrhage in the full-term neonate presenting as apneic seizures

    J. Perinatol.

    (2004)
  • R.J. Martin et al.

    Indications and standards for cardiopulmonary sleep studies

    Sleep

    (1985)
  • W.B. Spillman et al.

    A ‘smart’ bed for non-intrusive monitoring of patient physiological factors

    Meas. Sci. Technol.

    (2004)
  • A. Pantelopoulos et al.

    A survey on wearable sensor-based systems for health monitoring and prognosis

    IEEE Trans. Syst., Man, Cybern.

    (2010)
  • Cited by (49)

    • Motion magnification algorithms for video-based breathing monitoring

      2023, Biomedical Signal Processing and Control
    • RGB-D scene analysis in the NICU

      2021, Computers in Biology and Medicine
      Citation Excerpt :

      Previous video-based studies in the NICU have focused on non-contact vital sign monitoring [1,2], behavioral and pain assessment from facial expression [3–5], and clonic seizure detection [6,7]. Other studies have integrated depth-sensing cameras to capture additional perspectives pertinent to monitoring algorithms [8–10]. For example, depth images have been used to monitor chest movement variations related to breathing patterns [8,9].

    View all citing articles on Scopus
    View full text