STMicroelectronics Introduces High-Dynamic Range Image Sensors for Automotive Driver Monitoring
Someday soon, your car may watch you drive. These two new image sensors are designed for computer vision in automotive environments.
Someday soon, your car may watch you drive. This week, STMicro announced two new image sensors that are designed for computer vision in automotive environments, including driver monitoring.
Engineers are cramming more and more sensors into automotive applications, which require high levels of reliability, precision, and robustness.
While many of these applications are being developed for outward-facing monitoring—that is, for environment sensing for assisted and autonomous driving systems—some of these image sensors are being employed looking inward, as well.
This is one of the featured applications put forth by STMicroelectronics as they launch their Vx5y61 series of image sensors for automotive.
Image modified from STMIcroelectronics
The VG5661 and VG5761, announced this week, can operate in the near-infrared (NIR) range in the area of 940nm, beyond the human visible spectrum. This makes it possible to access not only the driver’s level of fatigue and attention but also conditions relating to passenger comfort and child behavior.
Such an application requires exceptional acuity to allow an autonomous-driving AI system to better make that most critical decision of whether or not the human driver is, in the moment, able to take over the vehicle’s operation from the autonomous controller.
Automotive 1.6 - 2.3 megapixel high-dynamic global shutter image sensor. Image from STMicroelectronics
Camera Features Designed for High-Precision
In their press release, STMicro cites that driver fatigue is a factor in up to 25% of all traffic accidents. In light of this and other factors, Europe’s New Car Assessment Program, Euro NCAP’s 2025 Roadmap, has recognized monitoring a driver’s instantaneous fitness as a vital safety feature. Most importantly, it is an absolute prerequisite for automated driving.
The VG5661 and VG5761 sensors are designed to capture images illuminated by the camera, which is intended to eliminate errors caused by the unpredictable effects of sunlight and street lighting. The devices utilize global shuttering technology, which, among other benefits, minimize the need for power-intensive illumination even when the units operate in their high dynamic range mode.
What Is Global Shuttering?
Global shuttering, a key factor in the architectures of the VG5661 and the VG5761, is a technique whereby all the pixels of a sensing device are sampled at the same instant.
This is contrasted with rolling shuttering, which involves sampling one row of pixels and then, in sequence, the next row. The finite time delay can cause blurred, distorted images, from which AI controllers, such as driver monitoring systems, can make erroneous conclusions about the situation being observed.
Rolling shutter vs. global shutter. Image from Oxford Instruments
The VG5661 has a resolution of 1.6 megapixels, (1464 x1104) at 75 frames per second (fps) in a 1/3” optical format. The VG5761 has a resolution of 2.3 MP (1944 x1204) at 60 fps in a 1/2.5” optical format. Both come in Automotive Qualified IM2BGA plastic packages. If bare die is preferred, the VD5661 and VD5761 are available.
Image from STMicroelectronics
The devices are qualified to AEC-Q100 grade 2. They conform to automotive safety standard ISO 26262 in regards to ASIL-B camera systems.
Collaboration with Jungo
STMicro’s new image sensors were demonstrated at electronica 2018 in combination with the driver-monitoring product CoDriver from automotive software developer Jungo. Aymeric Fuchet, Differentiated Imaging Solutions Business Line Director at STMicroelectronics states that “Our successful cooperation with Jungo has shown how these sensors can enhance road safety and extend the performance of autonomous driving systems.”
Comparing Image Sensor Options
As might be expected, there are many entries into the hot field of image sensors. The specs of these sensors are mostly quite similar, so, as usual, the designer will have to spend time poring through the datasheets to find one just right for the application at hand. Here are a few options:
Sony Corporation offers a suite of devices, including the IMX324, which offers a resolution of 7.42 megapixels. Maximum frame rate is 40 fps. And, it too, conforms to ISO 26262 and AEC-Q100 grade 2.
ON Semiconductor’s offerings include the AR0233AT. Its main features include a resolution 2.6 megapixels with an optical format of 1/2.5”.
Samsung offers three devices, the S5K2G1/SK53B5/S5K4A1. These three units sport resolutions ranging from 7.4 to 2.9 megapixels, optical formats from 1/1.7 to 1/3.8, and frame rates of 40fps or 30fps.
You can also read about ams's use of global shuttering in our look at the CSG14K image sensor for automated optical inspection (AOI).
What experience can you share about working with image sensors? Do you work with them for automotive projects or other applications? Please share your thoughts on the subject in the comments below.