News

Sony’s Image Sensor Roundup: Automation, Auto LiDAR, AI, and More

September 30, 2021 by Adrian Gibbons

What's at the heart of Sony's flurry of new CMOS image sensors? To start, copper-to-copper connection, back-illuminated structure, and an EVS mechanism.

Sony has had a busy start to the second half of 2021, announcing a slew of new CMOS image sensors. Application solutions range from machine vision for security to LiDAR and industrial automation. 

These new image sensors include:

 

IMX636/IMX637 EVS

The IMX636/IMX637 EVS. Screenshot used courtesy of Sony

 

First, we will take a look at the sensors themselves, detailing the basic specifications and potential applications of each. We'll also take a look at the technology behind these three new image sensors to get a sense of how they function.

 

IMX636 and IMX637 Event-based Vision Sensors

Sony claims that the IMX636 and IMX637 event-based vision sensors have the industry’s smallest pixel size—4.86 µm—for detecting subject change (pixel luminance changes). 

The two devices share the same proprietary cu-cu (copper to copper bonding) stacked technology, in which the pixel section is placed atop the logic section. However, they differ in image size, with a format of 1/2.5 at 0.92 effective megapixels for the IMX636 and 1/4.5 at 0.33 effective megapixels for the IMX637. 

 

The IMX636 and IMX637

The IMX636 (left) and IMX637 (right) are event-based vision sensors (EVS). Image used courtesy of Sony

 

Pixel size is one of the three main features for this new EVS, but what really makes it interesting is the way Sony employs it to address power challenges and limit event data to required elements only. Building on a partnership with Prophesee, the new EVS can filter event data at 30 fps. The image shown below is said to show a reduced data volume of 92 percent (single frame).

 

Reduction of captured data

Reduction of captured data through luminance thresholds and filtering. Image used courtesy of Sony

 

Next, Sony has another stacked cu-cu image sensor designed for LiDAR applications.

 

IMX459 Automotive-grade SPAD-depth Sensor 

The IMX459 is a single-photon avalanche diode (SPAD) image sensor, which utilizes the same cu-cu stacked technology as the IMX636. However, it also takes advantage of Sony’s back-illuminated structure and a light incident plane. 

This architecture provides a 24 percent photon detection efficiency, operating at a recommended wavelength of 905 nm. The sensor has a 1/2.9 image format with an effective pixel count of 100,000. 

 

Examples for Sony’s stacked technology (left) and LiDAR point cloud (right)

Examples for Sony’s stacked technology (left) and LiDAR point cloud (right). Image used courtesy of Sony

 

The IMX459 is currently undergoing AEC-Q100 Grade 2 certification. Additionally, Sony’s newest processes are said to be compliant with ISO 26262 for automobile functional safety

 

IMX585 Provides 4K Single-exposure Imaging for AI Processing

The IMX585 is the final device in this image sensor roundup. The IMX585 is a 1/1.2-type sensor with 4K resolution, which utilizes Sony’s proprietary STARVIS 2 technology to deliver high sensitivity and dynamic range. 

Using Sony's back-illuminated pixel technology, the sensor is said to provide an 88 dB dynamic range, which is eight times that of a conventional sensor with the same format.

A key feature of STARVIS 2 and the IMX585 is the elimination of image artifacts through the HDR single exposure process. Artifacts can interfere with AI processing of an image, leading to incorrect inferences.   

 

Improvements between STARVIS (left) and STARVIS 2 (right) in artifact suppression  
Improvements between STARVIS (left) and STARVIS 2 (right) in artifact suppression. Image used courtesy of Sony 

 

These three new image sensor platforms share a common technology base. Here is a quick overview of the common technologies employed in these CMOS sensors.

 

A View Into Sony’s Image Sensor Technology

Three key technologies are incorporated into these image sensors: the cu-cu connection, the back-illuminated structure, and (for the IMX636/637) an EVS mechanism. 

First, the stacked die technology is said to provide a cu-cu connection between pixels and logic die. On the IMX636/637 and IMX459, this allows Sony to reduce package size while maintaining high effective pixel counts. 

Second, and key to security applications, the STARVIS 2 back-illuminated structure claims to support superior low-light performance. 

 

Significant improvements in low light performance through structure changes (right)

Significant improvements in low-light performance through structure changes (right). Image used courtesy of Sony

 

The metal wiring in a front-illuminated image sensor suffers from reflections that reduce the total received photons.

Finally, a brief look at the EVS mechanism for the IMX636/637 reveals how this feature changes luminance according to a scene. The EVS process captures only changing pixel luminance and processes it with respect to coordinates, time, and polarity. 

 

Capturing a ball’s movement in time and space with EVS

Capturing a ball’s movement in time and space with EVS. Image used courtesy of Sony

 

Asynchronous pixel operation is a key feature of EVS. Each pixel contains its own light-receiving units and luminance-detection units to control the amount of data that is output from the device. 

Sony’s newest CMOS image sensors are bringing forth a host of performance enhancements for AI processing through STARVIS 2 technology and the event-based vision mechanism. 

With applications ranging from automotive LiDAR and security to industrial automation, designers may have the opportunity to evaluate one of these Sony CMOS image sensors for future computer vision applications.