Occupant Classification Technologies – No Child Left Behind

The interior radar-based occupant monitoring system represents a significant innovation in automotive safety, leveraging advanced technology to enhance passenger protection and comfort. Positioned at the top of the cabin, the interior radar utilizes sophisticated algorithms and sensors to analyze the cabin context effectively. This analysis enables the system to detect the presence of living beings in the car, including newborns and children up to six years old, ensuring their safety during travel, as well as animals.

EF Technology Nidec Elesys’ Electrical Field (EF) sensing technology works by emitting a very low level electrical field. The system is composed of a single-layered, flexible sensor assembly. The Elesys system fulfills the requirements set forth by FMVSS  208 issued by the National Highway Traffic Safety Administration in the United States.  Occupant Detection Technology the Occupant Position Detection Systems utilizes the same EF technology as ODS, but with sensors in the back cushion of the seat. In the event that a child’s head is detected in the deployment path of the side airbag, the ECU will suppress the side air bag in order to prevent serious or even fatal injury to the passenger.

PolarCoat™ patented technology, licensed by Corning can now apply a surface treatment with 99% of polarization efficiency (PE). This new coating, launched in early 2010, is applied on the backside of plano sunglass lenses.

ORNL keeps food and beverages refrigerated with an advanced evaporator, phase change materials, metal foam, direct-contact defrosting technology and a low global warming refrigerant. Credit: ORNL, U.S. Dept. of Energy.

Azcom Technology AWR1642 mmWave sensor, in combination with Azcom’s proprietary algorithms, can reliably recognize whether a seat is free or occupied. All processing, including Azcom Technology enhancements, runs on the sensor, while the graphical user interface on the host machine helps visualize the results.

The AWR1642 77GHz single-chip mmWave sensor with on-chip memory and a digital signal processor can sense very small movements, even breathing, that would indicate the presence of a person. The sensor is suspended from the sunroof for demonstration purposes, looking toward the back seat.

Fraunhofer IOSB detects the driver and all occupants equally. It recognizes the 3D body pose of all occupants and analyzes their movement behavior and classifies the activity of each individual person detected. This makes it possible to detect critical situations, such as a driver falling asleep, and to distinguish between different activities and the associated levels of distraction. Courtesy of Fraunhofer IOSB. The system distinguishes between up to 35 activities, including drinking, eating, sleeping, reading, and making phone calls. For this purpose, the machine learning algorithms process the 3D skeleton recognition of the occupants in combination with object detection and intelligent analysis of the movement behavior of all detected persons.

Perceptics Vehicle Occupant Detection (VOD) systems can be utilized in multiple different ways to help ensure safe driving practices and improve security. These systems utilize Perceptics cameras optimized for capturing images through windshields and software that is trained to identify humans. For security reasons, our VOD systems do not include any type of facial recognition or matching technology.

Each of our VOD configurations offers clear detection no matter the weather, configurable settings to meet your state and local regulations, optimized polarized lens and illumination to penetrate tinted windows and easy integration with your existing back-office software.

IVOD patented vehicle and passenger screening technology, security officers can see inside vehicles as they approach entry gates and know who is inside with accuracy and speed. The high-resolution machine vision camera is optimized for both day and night operation, the camera is paired with a powerful targeted array of LED lights that illuminate the driver and other occupants even through tinted glass, sun glare and at night. Using advanced image processing, the driver and passengers are highlighted while windshield glare from sunlight or overhead area lighting is removed through Gatekeeper’s advanced computer vision algorithms.

Tobii Autosense offers a holistic, and forward-looking approach to in-cabin safety. Powered by artificial intelligence, each passenger centric feature set relies on proprietary neural networks designed, trained, and tested by Tobi’s team of engineers, scientists and problem solvers. It is deployed using edge computing, without the need for cloud connectivity, meaning it is designed to enable all data to remain within the vehicle. The Tobii Autosense Occupant monitoring system brings together regulatory alignment and in-cabin differentiation, enabling the next-generation interior sensing.

Tobii Autosense single camera interior sensing (single camera DMS + OMS | SCDO) can change the perspective on interior sensing by targeting specific use cases categories – wellbeing, HMI, productivity, and entertainment – while being built on safety and privacy focused foundation.

04/2017 Update: Gortech, LLC www.gortechworks.com Dual-Carrier Non-Contact Vital Sign Detection

Growing interest in developing non-contact vital sign detection systems due to their promising applications in the field of vehicle cockpit and biomedical monitoring, health-care, astronauts physical monitoring and search and rescue after mudslides or earthquakes.

Various wireless solutions have been developed for vital sign detection:

1. Stepped frequency continuous wave (SFCW)
2. Frequency modulated continuous wave (FMCW)
3. Continuous wave doppler radar (CW)
4. Impulse-based ultra-wideband radar (UWB)

SFCW, FMCW and UWB bring the flexibility in controlling the frequency band to meet the application requirement on band and resolution, and at the same time provide the distance and image information.

CW uses the frequency or phase shift in a reflected radar signal to detect physiological movements which can reach a resolution of submillimeter, but respiration signal has a stronger power spectrum density than the heartbeat signal.

Current Vehicle Occupant Monitoring Technology Challenges

High requirements on bandwidth to detect small displacement in sub millimeters even millimeters.

As the vital sign is contained in the phase of microwave signals, phase noise of the received signal has a significant influence on the vital sign detection. It can raise the background noise floor, deteriorate the quality of signal, and impair the signal to noise ratio, which will decrease the detection accuracy and distance.

Phase locked loop is a promising method to suppress the noise, and a vital sign sensor based on phased locked loop in combination with the body proximity effect can effectively enhance the detection sensitivity by suppressing the phase noise of the received microwave signal, and reliably detect the vital sign signal in extreme and hostile environments.

Dual Carrier Phase Locked Loop Vital Sign Detection Technical Summary

A dual-carrier vital sign detection system with a noise suppression scheme based on phase locked loop is designed to automatically reduce the residual phase noise and path noise.

Through the phase discrimination between one carrier’s beat signal and the low-noise reference signal in the phase frequency detector, noises are extracted and then suppressed in the phase locked loop, providing a clean transmission path for the other carrier.

Vital sign signal contained in the phase of the second carrier can be obtained with low noise.

Experiments with locked and unlocked system have been carried out to compare the noise and detection performance.

System can successfully suppress the residual phase noise and path noise, improve the signal to noise ratio by about 50% at a detection distance of 50 cm, and significantly increase the detection distance of the weak heartbeat signal.

Demonstrated detection distance for heartbeat is at least 250 cm, more than double the distance of the unlocked system.

Experiments demonstrate effective measurements of vital sign in four physical orientations.

Current technology data is provided by a bench top component system to validate proof on concept.

Future application development will add a phase shifter to improve the efficiency of low noise amplifier and design a scheme to eliminate the influence of random physical movement, which will enhance its detection performance and practicability.

Automotive Application Strategy

Location: Recommend one sensor for each occupant location. Easily use sensor ID to discriminate between occupants on different seats. Each sensor will cover a specific area of the vehicle by designing an antenna with good directivity. Each sensor only detects the area that is covered by its own antenna.

Communication: Two types of communication involved: communication between multiple sensors in the monitoring system of a vehicle, and communication between the monitoring system and a terminal, which can be a specified terminal on a vehicle or a smart phone. Plan to use wired communication between multiple sensors and use wireless communication between the monitoring system and the specified terminal. Bluetooth (right) enabled for communication between the monitoring system and the terminal.

Size: Sensor package estimated to 5mm x 50 mm x 50mm

Key Potential Vehicle Applications:

Biometric driver identification detection. (Authorized driver enabled vehicle start)

Vehicle lock out with biometric identification detection (Timed authorization to enable restricted use, i.e. teenage driver)

Driver drowsiness detection. (Vital sign monitoring with alarm)

Driver heart condition detection and warning (Alarm and 911 call)

Occupant sensor and warning of extreme cabin temperature for children and pets. (Combined temperature sensor with alarm and automatic vehicle unlock).

 

09/15: Updates on In Car Child Presence Sensor

Below is updates to further investigates on in-car rear seat child presence sensors in the market or under innovation.

1. Evenflo SensorSafe Car Seat Alerts Parents to Presence of Child Still in the Car Consumer Reports evaluation shows tech has promise, but it still has some kinks to work out as per Emily A. Thomas, Ph.D. Evenflo Advanced SensorSafe Embrace DLX infant car seat now incorporates a sensor in the chest clip of the seat with a wireless receiver that connects to your car. Refer https://www.consumerreports.org/car-seats/evenflo-sensorsafe-car-seat-alerts-parents-to-presence-of-child-in-car/

2. NASA Engineer invested sensor – Very simple sensor located on the car seat which alerts you if child is left in the car seat. Of course, works directly with the child seat and some degree of awareness such as having the correct keychain with the receiver all the time.

Still step in the right direction.

3. On-Board Video Monitor as designed by Altitude and posted by their engineer Neil Bacon (http://www.altitudeinc.com/innovating-to-save-lives/) is a ceiling-mounted video monitor with temperature tracking capability that would monitor the interior of the car. Upon detection of a child left in the car with elevated temperatures, an embedded cellular communication would send an alert first to a parent’s registered smart phone. If no action has been taken within five minutes, Emergency Management Services would be contacted and deployed.

Remarks: A smaller micro version of this design concept is needed.

4. Another one currently going through crowd funding is the sensor with micro-controller based intelligent logic control system for accurate and reliable operation.

Refer (https://www.indiegogo.com/projects/babyproof-your-car-never-ever-forget-a-baby-in-car-kids#/).

It has BLE 4.0 Bluetooth built in to sync up with IOS and Android smart phone devices for app communication and Wireless RF receiver (MHz frequency) to communicate with child seat sensor device. Can be recharge anywhere using a Micro USB. Each charge can last up to three months. Get alerts far in advance via device & mobile app when battery is low. It utilizes state of the art optical transmitter and receiver components to determine whether the driver is about to exit the vehicle and will immediately alert them through a verbal notification via the built-in loudspeaker. Also includes a tri-color LED for visual status updates.

5. Another suggestion is to use ADAS help prevent this problem. Some examples are sensor is connected to the child seat sets off ringing / sends blue-tooth message until the child is successfully removed from the seat; cars with internal cameras can detect that the child is still in the seat and set off an alarm; detect through an infrared sensor if a human being is in a car; cars with power windows and doors that open automatically can be programmed so that when a child seat is occupied when the car ignition is off all the windows and doors open as well as any of the above after a certain time when the inside car temperature exceeds or 102° along with other alarms.

For information refer http://www.autoconnectedcar.com/2014/07/adas-needed-for-children-in-child-seats-in-hot-cars-use-app-or-childminder-for-now/

6. Sound Detection Sensor Module Sensor Intelligent Vehicle Compatible with Arduino by Atomic Market.

Also refer US Patent US 6922147 B1 “Warning system sensing child left behind in infant seat in vehicle.

Non-Contact Vital Sign Detection

Automobile Occupant Monitoring

Gortech, LLC

Dual-Carrier Non-Contact Vital Sign Detection: Growing interest in developing non-contact vital sign detection systems due to their promising applications in the field of vehicle cockpit and biomedical monitoring, health-care, astronauts physical monitoring and search and rescue after mudslides or earthquakes.

• Various wireless solutions have been developed for vital sign detection:
• Stepped frequency continuous wave (SFCW)
• Frequency modulated continuous wave (FMCW)
• Continuous wave doppler radar (CW)
• Impulse-based ultra-wideband radar (UWB)

SFCW, FMCW and Impulse radar bring the flexibility in controlling the frequency band to meet the application requirement on band and resolution, and at the same time provide the distance and image information.

Current Technology Challenges:

High requirements on bandwidth to detect small displacement in sub millimeters even millimeters, as range resolution is inversely proportional to the bandwidth.

Critical demand on ultra-wide bandwidth which is not easy to achieve through electrical devices.

Continuous-wave doppler radar (CW) uses the frequency or phase shift in a reflected radar signal to detect physiological movements which can reach a resolution of submillimeter, and is suitable for vital sign detection, including respiration and heartbeat, but respiration signal has a stronger power spectrum density than the heartbeat signal.

As the vital sign is contained in the phase of microwave signals, phase noise of the received signal has a significant influence on the vital sign detection. It can raise the background noise floor, deteriorate the quality of signal, and impair the signal to noise ratio (SNR), which will decrease the detection accuracy and distance.

Phase locked loop (PLL) is a promising method to suppress the noise, and a vital sign sensor based on PLL in combination with the body proximity effect can effectively enhance the detection sensitivity by suppressing the phase noise of the received microwave signal, and reliably detect the vital sign signal in extreme and hostile environments.

Large temperature variation during detection may influence the detection accuracy and its working range is limited to several centimeters.

Novel PLL Approach

A dual-carrier vital sign detection system using PLL to automatically suppress both the residual phase noise and transmission path noise.

Automatically adjust the feedback signals to suppress the noise when the environment changes.

Experimental results based on discrete components introduces no extra noise, has a 50% improvement in signal to noise ratio, and can effectively measure both respiration and heartbeat signals in four physical orientations at a distance of 50 cm.

By suppressing the residual phase noise and path noise, the detection distance of the weak heartbeat signal from the front of a human subject is extended to at least 250 cm, more than double the distance of a system without the noise suppression scheme.

Summary:

• A dual-carrier vital sign detection system with a noise suppression scheme based on PLL is designed to automatically reduce the residual phase noise and path noise.
• Through the phase discrimination between one carrier’s beat signal and the low-noise reference signal in the PFD, noises are extracted and then suppressed in the PLL, providing a clean transmission path for the other carrier.
• Vital sign signal contained in the phase of the second carrier can be obtained with low noise.
• Experiments with locked and unlocked system have been carried out to compare the noise and detection performance.
• System can successfully suppress the residual phase noise and path noise, improve the SNR by about 50% at a detection distance of 50 cm, and significantly increase the detection distance of the weak heartbeat signal.
• Demonstrated detection distance for heartbeat is at least 250 cm, more than double the distance of the unlocked system.
• Experiments demonstrate effective measurements of vital sign in four physical orientations.
• Future development will add a phase shifter to improve the efficiency of low noise amplifier and design a scheme to eliminate the influence of random physical movement, which will enhance its detection performance and practicability

For more information on this technology solution contact ‘John Gorniak at jgorniak@gortechworks.com.

In Car Rear Seat Child Sensor

In car rear seat child and occupancy detection should be a design imperative for all car lines. It is impossible to understand why it is not – most car lines seem to be waiting for NHTSA Guidelines before making a commitment.

And while cars now have features that remind drivers to buckle up, remember their keys in the ignition or turn their headlights off, no driver reminder chimes in to tell you your baby is snoozing in the back seat.

Even boy from Texas has invented a device which he hopes will save the lives of babies and young children if they become locked in a hot car.

39 toddlers died in 2016 because of being left inside sweltering cars after being forgotten about by their parents.

It was the highest number of deaths in three years with the state of Texas having the highest number in the nation with nine cases. Bishop’s device would see it attach to a car seat and it would detect if a child were left inside vehicle blowing cool air until parents and authorities are notified.

There are also multiple home security networks – view https://myplacesecurity.com/top/top-30-security-cameras-for-your-home for details.

The YI Home Camera Wireless IP Security Surveillance System (US Edition) Black is one such concept.

 

Vision technologies fused with other technologies such as RADAR, LIDAR, and far infrared sensors, the automobile is well on its way to becoming fully self-driving. By looking at a person’s face, that same vision system can also detect whether a person is drowsy or, by using facial recognition, who is driving the car and adjust systems – audio, seat position, HVAC, mirrors, or even whether the car will start.

Using such a system you could configure to detect child in rear seat.

Innoviz aftermarket LiDAR product for self-driving car development can also be developed for this application.

Light Detection and Ranging (LiDAR) is a light-based system that utilizes high-speed optical time-of-flight surveying to bounce off objects and generate a high-resolution map of the surroundings in real time. LeddarTech has developed concept systems, which consist of 360° view micro-LiDAR systems located in the headlights and taillights of a vehicle. This location not only eliminates reduction in aesthetics caused by the sensors, but also prevents sensor blind spots.

Image sensors and infrared light sources are used in occupant detection systems to detect and monitor occupants in terms of their position in the seat, or the driver’s activity or fatigue.

With the variety of airbags used in cars today, this feature is important to ensure the best possible protection for each occupant. The airbag control unit decides in the case of an accident, if it is helpful to deploy the airbags and with which level of force, or if there is a higher probability to cause severe injury to a passenger.

Besides airbags, occupant detection and monitoring is also necessary for other advanced driver assistance systems (ADAS). Some cars, for example, use seat-belt tightening as a pre-crash safety measure in the case of an accident, which requires occupant information. In addition, fatigue detection can be performed by monitoring eye movements and blinking frequency using a camera. Refer (http://www.hamamatsu.com/us/en/index.html).

Volvo testing new technology enabling cars to detect kangaroos… as figures show they cause 20,000 accidents every year – what about technology that detects child.

Will potentially avoid instances of Mother in New Zealand left her newborn baby in the car with a note saying she was shopping and to call her if the infant needed anything.

A device that will immediately alert a driver that has left their child in the car through a mobile application & speakers developed by http://www.sensealife.com/.

Their device utilizes patented technology to sense whether the parent and child are both present in the vehicle. It then determines when the parent is about to exit the car through optical sensors and triggers an audio alert with a verbal message notifying the driver to remove the child from the car seat.

If for some reason the driver is distracted and does not react to the verbal alert, the system triggers a secondary notification to a Bluetooth synced mobile phone through our proprietary mobile app alerting the driver.

GMC tries to prevent hot car deaths with rear seat reminder by adding “Rear Seat Reminder” as a standard feature to its 2017 Acadia SUV. GMC feature monitors the car’s rear doors and activates when it senses they are opened and closed within 10 minutes of the car starting, or while it’s running. The next time the engine is turned off, the system chimes and displays a message for the driver that reads, “Rear Seat Reminder, Look in Rear Seat.”

Vision-based system is one key opportunity for improved passenger sensing. Occupant detection for airbag deployment is currently implemented via in-seat pressure sensors or e-field sensors. Occupancy tracking for the back seat can provide very useful information to parents of small children, as is the related no-leave-behind reminder that sleeping, silent kids are still inside when the vehicle is parked and exited. And speaking of children, gesture-based interfaces are of intuitive use for all vehicle occupants, not just the driver.

Walmart Selling Car Seat That Alerts Parents When a Child Is Left Behind. The Evenflo Advanced SensorSafe Embrace infant seat uses a wireless receiver that plugs into a car on board diagnostic port, and syncs with a chest clip that goes around the baby (other companies’ products rely on Bluetooth or cellular technology. Once the car turns off, if the chest clip is still buckled, a series of tones will ring out and alert the driver.

Another example – PARC’s radar digital eye is discriminate humans in congested environments, e.g., detecting a child running around in a parking lot, or mapping out a cyclist on a curve.

Edward Modlin, Terry Mack and William Edwards, all NASA employees, developed a keychain alarm that sounds if a child is left in a car – Credit NASA Langley Research Center. The NASA device is simple. When a child is placed in the car seat, a sensor under the cushion, working through a module mounted on the side of the seat, establishes communication with an alarm on the driver’s key ring. If the driver walks away from the car while the child is still in the seat, the alarm sounds — and can be turned off only by removing the child.

The NASA device was designed to be added to existing cars, and the research center is looking for a commercial partner to further develop and market a product based on the technology.

Mr. Edwards said he believed an automaker could offer a similar device as a factory-installed feature.

Creators of the popular Israeli GPS app Waze decided to add a new feature to remind parents to check everyone has exited the vehicle. The feature, still in beta, provides a customizable notification when users arrive at their intended destination. Users can choose to enable or disable the warning and customize the message.

Matthew Sheets, 26, has designed a weight-activated seat sensor which notifies a parent’s mobile phone when their child is still in their baby seat.

The alert appears on their phone once they step outside of a 20ft (6 meters) radius from the baby seat if they’ve forgotten to take their child out of its seat.

It is designed to work on both Apple and Android smartphones via Bluetooth. As per design when you place your child on the Starfish device, it automatically pairs with your smartphone, sending you a notification that your child is in his or her seat.

‘When activated, Starfish then creates a ‘geo-fence’ radius around itself.

Three undergraduate engineering students at Johns Hopkins have turned technology from a popular video game player into a detector for children left behind in dangerously overheated vehicles. The young inventors tinkered with parts from a Kinect motion-sensing device, normally used with the Xbox 360 game console, and came up with the heart of a new system aimed at “seeing” children left in locked cars and summoning help.

Baby Alert’s ChildMinder SoftClip System Digital Wireless Technology Monitor replaces the plastic chest clip on the baby’s car seat safety harness. You carry the other part with you as a key fob. Once the baby’s chest device is clipped together, it connects digitally to the caretaker’s device. If the two devices are separated by more than 15 feet for more than 6 minutes, an alarm goes off.

Another possible tech solution is using a GPS tracker or wearable device to help monitor your child. GPS trackers that are currently marketed to alert you to lost glasses, wallets, and keys can be adapted to help you keep track of your child, even when a nanny or someone else is with them.

Most parents never forget their children. Unfortunately, in a world filled with distractions, they sometimes do resulting in tragedy. Design HMI LLC is investigating different technologies suited for fast to market introduction in current and 2017 MY+ MY cars.

Please refer Consumerist for more information.

First Technology’s Thermal Imaging Occupant Sensor (TIOS) developed to provide a low cost thermal sensor can be used to classify vehicle occupants to reduce the possibility of injury being caused by deployment of the airbag.

The sensor is an array of bolometers that are sensitive to energy in the 8μm – 14μm waveband. An image is captured and then image processing is performed to classify the occupants and determine their position. This information will then be available to the airbag system to assist in the deployment decision.

Per their WEBSITE the key features are:

1. Will help meet smart airbag regulations (FMVSS208).
2. Can offer additional functionality, such as intruder detection and thermal comfort applications.
3. Low production and investment costs.
4. Small package size and will work under different illumination levels.
5. Small package size.
6. Will classify between Normal adult, 5^th percentile female, 6-year-old child and infant in baby seat.
7. Could be evaluated for NO CHILD LEFT BEHIND application.

General Specifications

• Operating Voltage 9 to 16 Volts
• Current Draw 150mA at 13.5V
• Operating Temperature -40°C to 120°C
• Storage Temperature -40°C to 125°C
• Seat Occupancy Detection within 10 seconds
• Tracking Accuracy ± 25mm

The other is IEE SA VitaSense Radio-based technology to sense occupants. Per IEE VitaSense is a system that detects children left in vehicles and provides this information to the vehicle’s warning systems. It detects occupants, including new born babies based on their movements or breathing – even when they are sleeping.

Using 24 GHz low-power radio technology VitaSense transmits a signal that is then reflected by occupants or objects. As, due to their movement, occupants modify these reflected signals differently, the system can discriminate between them and inanimate objects.

IEE SA VitaSense technology is already automotive validated. VitaSense detects the presence of occupants based on their movements or breathing, even when they are sleeping. The sensing unit is integrated invisibly behind the vehicles headliner.

VitaSense uses well-known low-power radio technology. The reflected signal is received by the detector, and analyzed by the built-in algorithm. VitaSense works independently of any CRS and is not influenced by clothing, blankets, sunshades.

Another technology which has been investigated is solution that would consist of a linear multi-channel Leddar® Device installed inside the vehicle facing the passenger’s seat.  Powered by Leddar® technology and infrared light, the sensor will then provide measurement information from each of its segments, which, in term, could be used to estimate the size and shape of the person seated by subtracting the measured values of an empty seat.

 

Using detection and ranging technology for this application assumes the passenger is properly seated. Volume estimation also does not provide weight.

Both with current cars and especially with autonomous vehicles rear seat occupant detection becomes essential – specially to reduce / eliminate instances of child left behind’.

There are multiple technologies which can be applied for this application as listed below:

1. IS D360-24 is a highly sensitive ceiling mount occupancy sensor for indoor use. Sensor is passive infrared and responds to heat radiated from human body motion.
2. Infineon REAL3™ Image Sensor.
3. Hamamatsu TOF CMOS sensor works with an indirect time of flight (TOF) measurement as a 3D sensor and is designed to deliver information about the occupants’ position or about the driver’s activity or fatigue.
4. BeBop’s smart fabric OCS takes continuous full seat pressure images, providing real time data frames containing pressure information from the entire seat used to measure all aspects of physical contact between the occupant and the seat.
5. Echoflex RVS is a solar powered, passive infrared, wireless vacancy sensor. Optimized for spaces with ceiling heights of 8 to 10 feet, the RVS provides automatic OFF control, meeting today’s strictest energy codes.