Arik Schwartz


Viterbi Faculty of Electrical Engineering, Technion

Medical monitoring using multi-modal cameras

In recent years, multi-modal cameras are increasingly used in commodity systems, including house-hold applications. Those include RGBD cameras (visual + depth), such as Microsoft's Kinect, Intel's RealSense or Apple's Primesense technology. RGBD cameras have been used for smart user-interface in gaming, virtual reality applications and face recognition technologies. In addition, numerous thermo-graphic cameras (RGB + thermal IR) are being developed, and are expected to become commodity in the near future. This work considers the application of multi-modal cameras in medical applications. We consider two scenarios: (1) monitoring system to help reduce infection spreading in hospital wards by enforcing the use of sanitary protocols, i.e. wearing gloves and face masks; (2) the application of thermal camera with RGB and depth for remote sensing of body temperature. The first part of the project describes and evaluates a complete system for sanitary protocol monitoring and enforcement in a hospital ward for contagious patients. The system is installed at the entrance of the ward, and registers all visitors. It clears the entrance only for visitors wearing gloves and a mask, alerting otherwise. The system relies on RGB and Depth cameras (using off-the-shelf Microsoft Kinect system). An algorithm to detect protective gloves and mask is based on color classification was developed. We performed a field trial of the system through a series of experiments at the entrance to an contagious patients room in internal Ward-A at Hadassah Ein Kerem hospital. Of the course of several days we were able to monitor real-life activities including in-use medical equipment. The system has shown promising preliminary results by achieving an average of 83% success rate over a series of 47 tests performed with 8 different testers. We analyze the results and propose a few future improvements. We also addressed challenges of the monitoring device which include (a) remote secure communication (b) long term image recording and storage (c) and fail-recovery mechanisms. The second part of the work deals with the essential tasks to enable remote thermal sensing. These include: (a) data acquisition from camera (b) camera calibration (c) modality fusion (d) and image processing of the acquired frames. The calibration poses unique challenges, such as designing new calibration boards with high contrast to thermal, RGB and depth measurements. We proposed a new thermal-RGB calibration design and algorithm. * Supervisors: Guy Gilboa and Mark Silberstein.

Date: Tue 10 Jul 2018

Start Time: 10:00

1061 | Electrical Eng. Building