Monitoring infants by automatic video processing: A unified approach to motion analysis
Introduction
Monitoring neonatal movements may be exploited to detect symptoms of specific disorders. In fact, some severe diseases are characterized by the presence or absence of rhythmic movements of one or multiple body parts. Clonic seizures and apneas can be identified by sudden periodic movements of specific body parts or by the absence of periodic breathing movements, respectively.
Seizures are clinically defined as paroxysmal alterations of the neurological functions (i.e., behavioral, motor or autonomic function) and represent a distinctive symptom of acute brain dysfunction in the newborn [1]. Leading causes of neonatal seizures are intracranial haemorrhage, hypoxic-ischaemic encephalopathy and sepsis. As major symptoms of acute central nervous system impairment, almost 80% of these paroxysmal events occur in the first two days of life [2]. The estimated incidence of seizures is between 1‰ and 3.5‰ in full-term newborns and even higher in preterm infants [3]. Essentially, four main clinical seizure types can be recognized in neonates, namely: subtle, clonic, tonic and myoclonic [1]. Prompt and accurate detection of neonatal seizures is crucial to administer timely treatments that may prevent further seizure-induced brain damage. It is important to stress that the various categories of neonatal seizures are characterized by very different kinds of movements and require different analysis approaches. This article analyzes the monitoring of clonic seizures. Analysis systems for other types of neonatal seizures are discussed in the following articles: [4], [5] for subtle seizures, [6] for myoclonic seizures.
Apneas consist of the temporary absence of spontaneous respiration [7], which manifests itself by the lack of breathing movements for a certain time period. Seizures, cerebrovascular events [8] and congenital diseases, such as Congenital Central Hypoventilation Syndrome (CCHS) [9], are among the main causes of these events in the neonatal period. CCHS, also referred to as Ondine's curse, is a life-threatening disorder. Discovered as a disease caused by mutations in the paired-like homeobox 2B (PHOX2B) gene, CCHS manifests itself, in the neonatal period, especially during quiet sleep, with cyanosis, apnea events or even cardiorespiratory arrests [9]. CCHS is a relatively rare disorder: in fact, just approximately 1000 individuals with this condition have been identified. Researchers believe that some cases of Sudden Infant Death Syndrome (SIDS) may be caused by undiagnosed CCHS [10].
Monitoring of vital signs for the diagnosis of such type of diseases is possible almost exclusively in hospital environments. Currently, the standard monitoring systems are based on ElectroEncephaloGram (EEG) polygraphy and polysomnographic devices [11], composed of several wired sensors directly connected to the body of the patient: electrodes positioned on the scalp, chest and other body parts, elastic belt around the chest, nasal flow meter and pulse oximeter. The polysomnographic device allows to monitor the brain electrical activity, the cardiac and muscular activities, the respiratory movements, the breathing pattern and the blood oxygen saturation. These systems, typically used for short-term monitoring, are expensive, time-consuming, moderately invasive (especially for newborns), and require experienced medical staff, not always available full-time in a Neonatal Intensive Care Unit (NICU). For home monitoring, many systems are available, but they may require sensors attached to the body of the newborn (e.g., smart bed [12], wearable sensor system [13]). At the opposite, the objective of our research is to study a contact-less monitoring system. Thus, automatic, real-time and non-invasive equipment able to reliably recognize these diseases would be strategic in hospital environments, helping to monitoring 24/7 all the newborns present in a NICU, or even at home, in order to implement remote monitoring systems, reducing time and cost of hospitalization.
An attractive contact-less monitoring tool to automatically detect movement- or breathing-dependent diseases, such as seizures or breathing disorders, may rely on properly processing video signals, acquired through single or multiple video cameras. The movements of the newborn's body (e.g., limbs and chest), framed by the cameras, can be analyzed to detect specific behaviors which may be symptoms of neurological dysfunction.
In [14], the acquisition, through sophisticated video processing, of the motion strength was proposed as expedient to detect the presence of neonatal seizures. In [15], the authors used an optical flow-based technique to track and characterize the movements of a newborn during prolonged monitoring. Neural networks were then used to obtain a diagnosis based on a previous training phase. Taking into account the long monitoring time and the fact that the implementation of long reliable tracking of jerky movements of the newborn limbs may be very complex, this approach is not suitable for real-time detection and requires expensive hardware for accurate optical flow processing (especially for dense optical flow techniques). In [16], a real-time video processing-based approach to the detection of neonatal clonic seizures based on recognition of characteristic periodic movements, was proposed. This approach relies on a periodicity detector, based on hybrid pitch estimation, to analyze a motion signal extracted from the video. This method has also received attention in the medical literature [17].
In this paper, an improved method to estimate the periodicity of pathological movements, based on the use of the Maximum Likelihood (ML) criterion [18], is presented. In particular, motion signals from multiple digital cameras or depth-sensor devices (e.g., Kinect [19]) are extracted and properly processed in order to detect potential abnormal motor patterns. We propose a monitoring system based on the detection of pathological movements, characterized by the presence or absence of a significant periodic component (i.e. rhythmic movements). Thus, the field of application includes any disease presenting this type of symptoms: clonic seizures and CCHS are relevant examples. The novelty lies in combining known techniques (more or less recent) to create an innovative monitoring system that is, unlike existing systems, contact-less and low-cost. In fact, as an example, we were able to develop an Android application, denoted as Smartphone-based Contactless Epilepsy Detector (SmartCED), able to capture images through the phone camera and detect the occurrences of clonic seizures: the computing resources of modern smartphones have proven to be sufficient for our purpose [20]. This paper unifies and expands preliminary contributions [21], [22]. The approach behind these two studies, namely, detection and estimation of periodic movements, is common, so that a unified vision is here proposed.
The remainder of the paper is organized as follows. In Section 2, the method for the analysis of periodic movements is described. In Section 3, performance results are presented. Finally, in Section 4 conclusions are drawn.
Section snippets
Extraction of temporal motion signal
Extraction of a relevant motion signal from every sensor is the first key step of the proposed approach. The application to standard Red, Green and Blue (RGB) [23] cameras is initially presented, and the extension to depth sensors is later discussed.
We start considering a video (i.e., a sequence of frames) with sampling period T. Frames are numbered as , specifying the indexes related to time instants as integer multiples of T. Each frame is described by a matrix of pixels. The
Performance analysis
The acquisition system used at the NICU of the University Hospital of Parma and some illustrative examples of images acquired from a few cameras are shown in Fig. 4. The system is composed of three cameras: the first two cameras are arranged orthogonally in order to frame the baby by the front and the side, the third one is attached to the cradle to frame the face of the newborn [4]. In the following, the use of the first camera only will be initially considered. Then, performance results using
Conclusion
This paper describes a unified wire-free, low-cost, non-invasive technique for automatic detection of abnormal paroxysmal events in newborns, namely, clonic seizures and life-threatening apnea events. According to the proposed approach, motion signals are extracted from RGB cameras or depth sensors in order to estimate whether a periodic component is present, which is the characteristic feature of clonic seizures or breathing movements. In the case of apnea detection, the video may be
Standard protocol approvals, registrations, and patient consents
In accordance with current practice at our Institution, an informed consent form was signed by parents of each newborn patient, and the informed consent was placed in the patients' hospital chart. The Ethic Local Committee approval to perform retrospective studies was obtained.
Conflict of interest statement
None declared.
References (33)
- et al.
Central hypoventilation syndromes
Sleep Med. Clin.
(2014) - et al.
Congenital central hypoventilation syndrome and sudden infant death syndromedisorders of autonomic regulation
Semin. Pediatr. Neurol.
(2013) Detecting neonatal seizuresa challenge accepted!
Clin. Neurophysiol.
(2014)- et al.
Real-time automated detection of clonic seizures in newborns
Clin. Neurophysiol.
(2014) - et al.
Clinical testssensitivity and specificity
Contin. Educ. Anaesth. Crit. Care Pain
(2008) Neurology of the Newborn
(2008)The Epilepsies: Seizures, Syndromes and Management
(2005)- et al.
Controversies in neonatal seizure management
J. Child Neurol.
(2009) - et al.
Optimizing detection rate and characterization of subtle paroxysmal neonatal abnormal facial movements with multi-camera video-electroencephalogram recordings
Neuropediatrics
(2016) - G.M. Kouamou Ntonfo, F. Lofino, G. Ferrari, R. Raheli, F. Pisani, Video processing-based detection of neonatal seizures...
Temporal lobe hemorrhage in the full-term neonate presenting as apneic seizures
J. Perinatol.
Indications and standards for cardiopulmonary sleep studies
Sleep
A ‘smart’ bed for non-intrusive monitoring of patient physiological factors
Meas. Sci. Technol.
A survey on wearable sensor-based systems for health monitoring and prognosis
IEEE Trans. Syst., Man, Cybern.
Cited by (49)
Automatic prediction of therapeutic activities during newborn resuscitation combining video and signal data
2023, Biomedical Signal Processing and ControlMotion magnification algorithms for video-based breathing monitoring
2023, Biomedical Signal Processing and ControlMaximum likelihood speed estimation of moving objects in video signals
2022, Signal ProcessingPATRON: Exploring respiratory signal derived from non-contact face videos for face anti-spoofing
2022, Expert Systems with ApplicationsRGB-D scene analysis in the NICU
2021, Computers in Biology and MedicineCitation Excerpt :Previous video-based studies in the NICU have focused on non-contact vital sign monitoring [1,2], behavioral and pain assessment from facial expression [3–5], and clonic seizure detection [6,7]. Other studies have integrated depth-sensing cameras to capture additional perspectives pertinent to monitoring algorithms [8–10]. For example, depth images have been used to monitor chest movement variations related to breathing patterns [8,9].
Predicting apneic events in preterm infants using cardio-respiratory and movement features
2021, Computer Methods and Programs in Biomedicine