A New Dual RGB-IR Sensor Put to Use in Endoscopes for Cancer Detection

OmniVision, a company that develops digital imaging solutions, has recently introduced its newest sensor, the OH02A1S. This company says this RGB-IR sensor is the first device of its kind to be used in medical imaging applications—specifically endoscopes—according to the press release.

 

The Bayer format vs. the RGB-IR format. Image used courtesy of OmniVision

 

Before we dive into the specifics of the new sensor, it may first be helpful to review what exactly an RGB-IR sensor is, how it’s made, where it can be used, and how it's different from an RGB sensor.

 

A Brief Review of RGB Sensors

According to an article explaining the technology by FRAMOS, RGB-IR sensors are an enhancement of normal RGB sensors. Normally, RGB sensors are composed of a matrix of pixels, each pixel being a monochrome sensor.

This means each pixel is a sensor that captures all incoming light regardless of wavelength. Each pixel is then covered with a color filter mask in order to only let one color go through to the pixel, which in the case of RGB sensors, means the mask will let in either red, green, or blue light only.

The mask is commonly referred to as a Bayer mask or Bayer filter, and the colors are in a pattern of blue-green rows and red-green rows alternating with one another in the matrix, as shown in the following image:  

 

Depiction of the Bayer mask, in which green is most visible. Image used courtesy of FRAMOS

 

Green pixels are favored because the human eye has the greatest color resolution in this area of the visible light spectrum.

 

How are RGB-IR Sensors Distinct From RGB Sensors?

Now, for RGB-IR sensors, the red-green row is replaced by a red-IR row, so the number of green pixels is halved. Now, there is an equal number of blue, red, green, and IR pixels.

This distribution will negatively affect the color resolution because the number of “color pixels” (red, blue, and green) are decreased. As such, RGB-IR sensors are ineffective in environments with low IR radiation or environments where color resolution and measurement must be maximized.

Still, the IR pixels give the sensor a great number of benefits. The IR pixels are able to pick up ambient IR radiation, and thus the brightness of the whole image will be enhanced as both visible and IR light are absorbed. That way, the sensor can create clearer images in darker areas or areas with bright spots near dark spots.

 

Application circuit for a sensor from Maxim Integrated, the MAX44005, that includes seven sensors—including an RGB sensor and an IR proximity sensor. Image used courtesy of Maxim Integrated

 

Furthermore, in normal RGB sensors, the IR light can damage the color separation since it is not directly measured. Traditionally, this issue was solved by adding an IR cut filter in front of the sensor. No filter is required for low light conditions where IR radiation is present; however, a proper lens must be chosen for the system since RGB light and IR light are measured simultaneously.

With RGB-IR sensors, one can now subtract the exact amount of IR light measured from the RGB signals before creating the image through a method known as the de-bayering process.

 

The dual RGB and IR capabilities allow devices to capture images in both day and night. Image used courtesy of OmniVision

 

The resultant image is brighter while maintaining vibrant color. The device can also adapt to changing brightness in its environment, such as outdoor monitoring during the day and night.

 

The Problem of Two Sensors in Endoscopes

With what we now know about RGB-IR sensors, we can look into its importance in medical imaging.

Currently, IR light is used to detect precancer or cancer in the body while RGB light is used to confirm any abnormalities detected with the infrared. As such, endoscopes include two separate image sensors: one for IR light and one for RGB light. However, the inclusion of both sensors made endoscopes so large that they were unable to monitor certain parts of the body. Endoscopes would also heat up more because of the power requirements of two separate image sensors.

 

Endoscope imaging has many advantages over other reusable devices, which often have issues with sterilization. Image used courtesy of OmniVision

 

Because of these complexities, endoscopes are not disposable because the cost and complexity of the scopes are just too high.

 

How the New RGB-IR Sensor is Used in Medical Imaging

OmniVision hopes to address these constraints with the new RGB-IR sensor, the OH02A1S. Housed in a 3.8 mm by 2.9 mm package, this device is said to feature low power consumption, topping off at 90 mW.

The sensor is in a ⅙-inch optical format, which is especially relevant for ever-smaller endoscope tips to better access more areas in the body. This small size also means less heat dissipation and more efficiency.

 

A dual band-pass filter and a dual focal lens are required to capture both visible and IR images. Image used courtesy of OmniVision

 

OmniVision claims that because this single sensor is easier to integrate, disposable scopes may be a possibility in the future.

The sensor features a 1.4-micron PureCel pixel architecture and 4 x 4 binning. The 4 x 4 matrix of pixels act as one “superpixel,” allowing for a greater signal-to-noise ratio. The company also says the sensor offers resolutions of 1080 p at 60 fps and 720 p at 90 fps through a 2-lane MIPI serial output. Additionally, the device features two kilobits of one-time programmable memory on-chip.

 

The RGB-IR Sensor's "Quantum Efficiency"

The sensor is said to have exceptional quantum efficiency (QE), which is a ratio of the number of carriers collected by a sensor to the number of photons of a given wavelength incident to the solar cell.

In English, the QE determines how much of the light the sensor collects vs. how much light the sensor is exposed to. A high quantum efficiency entails that the sensor is collecting a large amount of the light incident. In other words, the "effectiveness of an imaging device to convert incident photons into electrons," according to Teledyne Photometrics.

 

The QE of a sensor with 95% efficiency at various photon wavelengths. Image used courtesy of Teledyne Photometrics

 

The result of a high QE is a vivid image that allows surgeons to switch between IR and RGB measurements in real-time or display both images either side-by-side on the monitor or overlaid with one another.

On both the device and system level, OmniVision’s OH02A1S sensor aims to advance the medical imaging industry. The sensor is a testament to the capabilities of RGB-IR sensor technology, which, still being relatively young, may significantly shape imaging technology in the future.