almi
Posts: 21
Joined: Thu Jul 21, 2016 12:22 am

Non-contact tech for sleep monitoring

Wed Oct 19, 2016 4:24 pm

Let me know if this is something of interest to you.
This is still a research project.
My intention is to release it under an open source license at some point.

A brief example of what the ALT technology can do:
ALT can monitor pulse and respiration of a person even if the person is completely covered by a thick blanket.

Image
Figure 1. A person reclining in an armchair and completely covered by a blanket.

Image
Figure 2. Pulse and respiration ALT data for the person under the blanket in Figure 1. Several consecutive heartbeats are marked by the asterisks (*).

ALT is a non-contact technology. Current ALT hardware includes a Pi NoIR v2 camera connected to a Raspberry Pi 3B, and a Kinect for Xbox 360 unit which was used to illuminate the scene in a dark room shown in Figure 1.

You can find more details about the ALT tech here:

“What has happened while we slept?"
(https://www.linkedin.com/pulse/what-has ... r-misharin)
shows that ALT can detect heartbeats and respiration even when a person is completely covered by a thick blanket, and that ALT uses a motion detection algorithm which allows detecting small body movements.

“When your heart beats”
(https://www.linkedin.com/pulse/when-you ... r-misharin)
discusses potential applications of the ALT technology in medical imaging, including non-contact analog of the seismocardiography method of monitoring the mechanical activity of a heart.

“Non-contact real-time monitoring of heart and respiration rates using Artificial Light Texture"
(https://www.linkedin.com/pulse/use-arti ... t-misharin)
has some additional technical information about the ALT technology.

Alex
Last edited by almi on Sun Sep 17, 2017 6:06 pm, edited 1 time in total.

User avatar
SteveDee
Posts: 343
Joined: Thu Dec 29, 2011 2:18 pm
Location: Sunny Southern England
Contact: Website

Re: Non-contact tech for sleep monitoring

Tue Dec 06, 2016 1:47 pm

This is a really interesting project Alex.

Is there anything special about the infrared lighting? Could the light source be clustered close around the camera, or do you need a bit of side lighting to create shadows (enhance texture)? Is the system tolerant of the subject moving? (i.e. can the system track the subject's heart location in the image?).

If you decide to Open Source this project, please post back here with details.

almi
Posts: 21
Joined: Thu Jul 21, 2016 12:22 am

Re: Non-contact tech for sleep monitoring

Wed Dec 14, 2016 1:16 pm

Thanks!

The camera and projector can be pretty close to each other.

For example, I have used the light projector and camera of a F200 (Intel RealSense) unit for ALT monitoring of pulse/respiration.

Human eyes are mostly insensitive to infrared light of IR projectors used in different consumer electronics devices hence no disturbance when using ALT to monitor somebody's sleep at night.

ALT can work with non-IR light too.

ALT can detect pulse if the camera observes just your forearm, for example. And you can wear a shirt that covers the forearm.

There is no need to focus the camera on a particular part of a body like the heart area or to determine/track location of a particular part of a body (e.g. heart or forehead or eyes or an area of open skin etc.) in the images in order to enable pulse detection.

Note that one can process different parts of the video frames and extract ALT information (respiration/pulse/motion) specific to those parts, as I show in https://goo.gl/oBGD1s [edit: the link is https://www.linkedin.com/pulse/when-you ... r-misharin]. Therefore, you can monitor spatial characteristics of a process and how those characteristics change over time. For example, provided you have a camera with sufficiently high frame rate, you could detect the spread of a pulse wave from your heart to the different parts of your body.
Last edited by almi on Sun Sep 17, 2017 6:47 pm, edited 1 time in total.

almi
Posts: 21
Joined: Thu Jul 21, 2016 12:22 am

Re: Non-contact tech for sleep monitoring

Tue Jan 10, 2017 1:51 pm

Here's some ALT data from Intel's RealSense F200 and R200 cameras:
https://www.linkedin.com/pulse/alt-puls ... s-misharin

I wish Pi had USB 3 or F200/R200 allowed working with USB 2 devices.

almi
Posts: 21
Joined: Thu Jul 21, 2016 12:22 am

Re: Non-contact tech for sleep monitoring

Tue Jan 24, 2017 1:18 am

"How ALT works" is on GitHub: https://github.com/lvetech/ALT

martywinston
Posts: 1
Joined: Tue Apr 11, 2017 12:59 am
Location: Aiken, SC, USA
Contact: Website

Re: Non-contact tech for sleep monitoring

Tue Apr 11, 2017 1:19 am

This may seem trivial but it's a direct, if limited, adaptation of what you're trying to accomplish.

I simply want to accomplish a way of telling if a bed (or for a larger bed, each side of it) is occupied (by a human) or empty and whether somebody in the bed is awake and asleep.

(It's part of an ultra-automated house design project; it has to do with intelligently adjusting the lighting and power to the TV set, at least initially).

I am especially intrigued that your approach can see through covers; can it detect the hear rate no matter which way the person's facing?

I'm not sure if you explored the small, inexpensive radar heads (from old microwave field disturbance alarms used in some security systems) on Amazon - I was thinking that they, too, might be able to discern the tiny motions related to heartbeats; in their case, as in yours, I think the challenge is in separating the desired pattern from a field of noise.

The advantage of the radar head (in my mind) is that (I hope) it can be mounted under the bed. so no disruption to room decor.

I love the idea of having a single sensor (in your approach) monitoring two sleepers independently - is there a range limitation? Can the camera be a couple meters above the bed? Can one camera in the center of a room be reset to different sleeper positions when the bedroom furniture gets rearranged?

Is there enough information in something like a Panasonic Grid Eye (an 8 x 8 thermal measuring matrix of roughly 60 cm squares) to derive sleeping/waking data? Would your IR projector disrupt thermal measurements made by a Grid-Eye (used in my case to track warm-body positions)?

Marty Winston - Newstips - 40yearhouse
________/\/\.\/\/.
Marty Winston - Newstips - 40yearhouse
"Older than dirt and twice as smart"

almi
Posts: 21
Joined: Thu Jul 21, 2016 12:22 am

Re: Non-contact tech for sleep monitoring

Tue Apr 11, 2017 4:35 pm

martywinston wrote: … can it detect the hear rate no matter which way the person's facing?
Yes, it can detect the heart rate no matter which way the person is facing.
martywinston wrote: … is there a range limitation? Can the camera be a couple meters above the bed?
My ALT setup (Kinect + Pi camera + Pi) is @ about 2 meters distance from the bed when I monitor my sleep at night.
An ALT system can be designed for any distance range.
The range of the distances acceptable for an ALT system will depend on the characteristics of the ALT system elements such as its light emitter and video camera.
I am currently using light emitters (and cameras, too) of the off-the-shelf consumer electronics devices for ALT (Microsoft Kinect for Xbox 360, Intel RealSense F200 and R200 cameras).
martywinston wrote: Would your IR projector disrupt thermal measurements made by a Grid-Eye (used in my case to track warm-body positions)?
I think it should not.
IR projectors used in Kinects and RealSense cameras operate on wavelengths below 1 um, as I understand, while the "lens and the optical filter of GridEYE form a band-pass characteristic for wavelengths between 5 and 13um” (https://eu.industrial.panasonic.com/sit ... e_v1.0.pdf, page 7).
Also, I am not aware of any reports on using a Kinect or a RealSense camera for thermal imaging.
martywinston wrote: Is there enough information in something like a Panasonic Grid Eye (an 8 x 8 thermal measuring matrix of roughly 60 cm squares) to derive sleeping/waking data?
I do not have any experience working with GridEYE.
martywinston wrote: The advantage of the radar head (in my mind) is that (I hope) it can be mounted under the bed. so no disruption to room decor.
Design is important.

I think that there is a good chance that phones/tablets will be equipped with RealSence-like cameras to enable the “untethered” AR/VR/merged experiences.
One can put such a phone on a nightstand and use it as an ALT system for sleep monitoring at night.

Alex

Return to “Other projects”