Sony’s Image Sensor Roundup: Automation, Auto LiDAR, AI, and More
What's at the heart of Sony's flurry of new CMOS image sensors? To start, copper-to-copper connection, back-illuminated structure, and an EVS mechanism.
Sony has had a busy start to the second half of 2021, announcing a slew of new CMOS image sensors. Application solutions range from machine vision for security to LiDAR and industrial automation.
These new image sensors include:
- IMX636 and IMX637, two types of stacked event-based vision sensors
- IMX459, a stacked SPAD-depth sensor for automotive LiDAR
- IMX585, a 1/1.2-type 4K-resolution CMOS image sensor for security cameras
The IMX636/IMX637 EVS. Screenshot used courtesy of Sony
First, we will take a look at the sensors themselves, detailing the basic specifications and potential applications of each. We'll also take a look at the technology behind these three new image sensors to get a sense of how they function.
IMX636 and IMX637 Event-based Vision Sensors
Sony claims that the IMX636 and IMX637 event-based vision sensors have the industry’s smallest pixel size—4.86 µm—for detecting subject change (pixel luminance changes).
The two devices share the same proprietary cu-cu (copper to copper bonding) stacked technology, in which the pixel section is placed atop the logic section. However, they differ in image size, with a format of 1/2.5 at 0.92 effective megapixels for the IMX636 and 1/4.5 at 0.33 effective megapixels for the IMX637.
The IMX636 (left) and IMX637 (right) are event-based vision sensors (EVS). Image used courtesy of Sony
Pixel size is one of the three main features for this new EVS, but what really makes it interesting is the way Sony employs it to address power challenges and limit event data to required elements only. Building on a partnership with Prophesee, the new EVS can filter event data at 30 fps. The image shown below is said to show a reduced data volume of 92 percent (single frame).
Reduction of captured data through luminance thresholds and filtering. Image used courtesy of Sony
Next, Sony has another stacked cu-cu image sensor designed for LiDAR applications.
IMX459 Automotive-grade SPAD-depth Sensor
The IMX459 is a single-photon avalanche diode (SPAD) image sensor, which utilizes the same cu-cu stacked technology as the IMX636. However, it also takes advantage of Sony’s back-illuminated structure and a light incident plane.
This architecture provides a 24 percent photon detection efficiency, operating at a recommended wavelength of 905 nm. The sensor has a 1/2.9 image format with an effective pixel count of 100,000.
Examples for Sony’s stacked technology (left) and LiDAR point cloud (right). Image used courtesy of Sony
The IMX459 is currently undergoing AEC-Q100 Grade 2 certification. Additionally, Sony’s newest processes are said to be compliant with ISO 26262 for automobile functional safety.
IMX585 Provides 4K Single-exposure Imaging for AI Processing
The IMX585 is the final device in this image sensor roundup. The IMX585 is a 1/1.2-type sensor with 4K resolution, which utilizes Sony’s proprietary STARVIS 2 technology to deliver high sensitivity and dynamic range.
Using Sony's back-illuminated pixel technology, the sensor is said to provide an 88 dB dynamic range, which is eight times that of a conventional sensor with the same format.
A key feature of STARVIS 2 and the IMX585 is the elimination of image artifacts through the HDR single exposure process. Artifacts can interfere with AI processing of an image, leading to incorrect inferences.
Improvements between STARVIS (left) and STARVIS 2 (right) in artifact suppression. Image used courtesy of Sony
These three new image sensor platforms share a common technology base. Here is a quick overview of the common technologies employed in these CMOS sensors.
A View Into Sony’s Image Sensor Technology
Three key technologies are incorporated into these image sensors: the cu-cu connection, the back-illuminated structure, and (for the IMX636/637) an EVS mechanism.
First, the stacked die technology is said to provide a cu-cu connection between pixels and logic die. On the IMX636/637 and IMX459, this allows Sony to reduce package size while maintaining high effective pixel counts.
Second, and key to security applications, the STARVIS 2 back-illuminated structure claims to support superior low-light performance.
Significant improvements in low-light performance through structure changes (right). Image used courtesy of Sony
The metal wiring in a front-illuminated image sensor suffers from reflections that reduce the total received photons.
Finally, a brief look at the EVS mechanism for the IMX636/637 reveals how this feature changes luminance according to a scene. The EVS process captures only changing pixel luminance and processes it with respect to coordinates, time, and polarity.
Capturing a ball’s movement in time and space with EVS. Image used courtesy of Sony
Asynchronous pixel operation is a key feature of EVS. Each pixel contains its own light-receiving units and luminance-detection units to control the amount of data that is output from the device.
Sony’s newest CMOS image sensors are bringing forth a host of performance enhancements for AI processing through STARVIS 2 technology and the event-based vision mechanism.
With applications ranging from automotive LiDAR and security to industrial automation, designers may have the opportunity to evaluate one of these Sony CMOS image sensors for future computer vision applications.