Home
Technical Articles
Understanding Color Models Used in Digital Image Processing

Technical Article

Understanding Color Models Used in Digital Image Processing

August 17, 2018 by Robert Keim

Learn about digital-signal-processing concepts that help us to store and manipulate color information.

What is color? The question is far less straightforward than it might appear. For people, color is a visual characteristic that is deeply connected to emotional, aesthetic, and cultural aspects of the human experience. In the context of physics, color is merely a convenient way of referring to different wavelengths of electromagnetic radiation. In the purely physical realm, then, color doesn’t really exist. Light waves exist, and the wavelength of this light can indeed vary, but color—green, blue, red, and all the rest—is a delightful fiction created by the human brain, which for some reason looked at the horizon and decided that the mishmash of 600 nm-ish electromagnetic radiation should look something like this:

This discussion highlights the difficulty of translating color into the digital realm. Color is an inherently subjective phenomenon: wavelengths are objective things, but the visual appearance connected to a particular wavelength is established by the biological system that is receiving and interpreting the electromagnetic radiation. And since computers, robots, and sensor systems are typically used by people, the goal of digital color processing is to create information and images that are consistent with human vision.

What Is a Color Model?

The partially psychological way in which human cognition formulates colored imagery based on wavelength is not something that can be duplicated in a machine. Instead, machines use a color model, which I would describe as a mathematical approximation of the inherently nonquantifiable nature of human visual perception.

[Author’s note: The above information has proved to be much more controversial than I expected! Please take a look at the comments at the end of this article, and feel free to contribute to the discussion and to form your own opinion on the fascinating interaction between the physical characteristics of electromagnetic radiation and the human experience of color.]

Many color models exist, and presumably they all have advantages and disadvantages that make them more or less suitable for a given application. In this article we’ll discuss the two that are most commonly used in the context of digital image processing: RGB and HSI.

The RGB Color Model

As you probably know, RGB stands for red, green, blue. The RGB color model is additive: red, green, and blue light are added together in varying proportions to produce an extensive range of colors.

It’s important to keep in mind that real-life colors are not actually a mixture of red, green, and blue. Purple, for example, is purple, not a vector that extends 33 units in the red direction, 39 units in the green direction, and 127 units in the blue direction. Nonetheless, the RGB model has been wildly successful and is frequently used in sensor and image-processing applications.

In my opinion, the RGB model is, overall, quite intuitive. The key to understanding RGB image processing is recognizing that an RGB image is simply a composite of three independent grayscale images that correspond to the intensity of red, green, and blue light.

These three images can be processed separately and then recombined into a single image that human beings will perceive as having color.

Gathering RGB data is also straightforward, again, because RGB imagery boils down to intensity. A photosensor that measures light intensity becomes an R, G, or B sensor if you combine it with an optical filter, which means that you can generate RGB data using a unit consisting of three photosensors and three optical filters. In fact, I used a sensor like this for a project series that I wrote a couple years ago.

It would be reasonable to assume that color photographs or video imagery would require three full CCD or CMOS image sensors, but it turns out that a system can generate high-quality color imagery from one image sensor by using a Bayer filter and then applying specialized processing algorithms to the resulting data.

The arrangement of colors in a Bayer filter. Image created by Cburnett and taken from Wikimedia Commons.

The HSI Color Model

HSI stands for hue, saturation, intensity. This model is interesting because it can initially seem less intuitive than the RGB model, despite the fact that it describes color in a way that is much more consistent with human visual perception.

It’s true that the RGB model draws upon our familiarity with mixing primary colors to create other colors, but in terms of actual perception, RGB is very unnatural. People don’t look at a grapefruit and think about the proportions of red, green, and blue that are hidden inside the somewhat dull, yellowish-orangish color of the rind or the shinier, reddish flesh. Though you probably never realized it, you think about color more in terms of hue, saturation, and intensity.

Hue is the color itself. When you look at something and try to assign a word to the color that you see, you are identifying the hue. The concept of hue is consistent with the way in which a particular wavelength of light corresponds to a particular perceived color.
Saturation refers to the “density” of the hue within the light that is reaching your eye. If you look at a wall that is more or less white but with a vague hint of peach, the hue is still peach, but the saturation is very low. In other words, the peach-colored light reaching your eye is thoroughly diluted by white light. The color of an actual peach, on the other hand, would have a high saturation value.
Intensity is essentially brightness. In a grayscale photograph, brighter areas appear less gray (i.e., closer to white) and darker areas appear more gray. A grayscale imaging system faithfully records the intensity of the light, despite the fact that it ignores the colors. The HSI color model does something similar in that it separates intensity from color (both hue and saturation contribute to what we call color).

HSI is closely related to two other color models: HSL (hue, saturation, lightness) and HSV (hue, saturation, value). The differences between these models are rather subtle; the important thing at this point is to be aware that all three models are used and that they all adopt the same general approach to quantifying color.

The following three images are screen captures from a graphics program called Inkscape; they indicate the H, S, and L components of the three colors shown above (in the RGB section).

Click to enlarge.

Conclusion

We’ve covered some basic information related to color-representation systems that are useful in machine-vision applications. If you’ve done image-processing work that was not based on the RGB or the HSI/HSL/HSV color model, feel free to leave a comment and tell us about your experiences.

Learn More About:

8 Comments

R
Raymond Genovese August 17, 2018

The author does not, in my view, understand nor appreciate the role of neuronal processes which are integral to any discussion of color. While the models of color seem to be presented well enough, much of the introduction is unnecessary for the central purpose of the article, inaccurate and, frankly, sophomoric.

Color is a neuronal event. It is produced by the CNS. These points have been completely missed by the author when making statements like, “In the purely physical realm, then, color doesn’t really exist.” Neuronal processes are absolutely in the physical realm.

Furthermore, statements such as “The mysterious and somewhat immaterial way in which human cognition formulates colored imagery based on wavelength is not something that can be duplicated in a machine…” are equally errant. If the intended definition of “immaterial” is “irrelevant or not important”, then the statement is wrong because the neuronal processes are obviously essential to the experience of color. If the intended definition of “immaterial” is “spiritual, rather than physical”, then the statement is also wrong because the neuronal processes are clearly physical. Moreover, there is no logical reason why a machine cannot duplicate those processes, at least eventually.

The fundamental lack of understanding by the author is compounded in the following statement, “Instead, machines use a color model, which I would describe as a mathematical approximation of the inherently nonquantifiable nature of human visual perception.” There is no logical reason to believe that the nature of human visual perception is “inherently nonquantifiable” and scientists have worked for years to do just that and have made great strides in understanding. To deny this strikes me as being no less than antiscience.

Indeed, in some cases, light is not even necessary to experience color. It is common for most people to report that they dream in color. Many studies have shown that electrical brain stimulation produces color. The phenomena of chromesthesia, in which sound produces or is accompanied by color (particularly in musicians) is well established.

It is unfortunate and disappointing that the author, who is obviously a skilled EE and writer, chose to use so much of the article to attempt to explain color without commensurate knowledge, when the space could have been used to add to the quite reasonable explanation of models of color.

Like. Reply
- R
  Robert Keim August 20, 2018
  
  Thanks for the comment. As I mentioned in our previous discussion, I realized that it would be a good idea to avoid the use of the word "immaterial," but I wasn't aware of the publishing schedule and the article was published before I made the revision. I changed that sentence to use "partially psychological," a description that is confirmed by the university-level textbook on which this article is based. I have provided solid arguments in favor of the nonquantifiable nature of human color perception, and you have not responded directly to those arguments. Furthermore, my textbook states that "one of the key factors in describing color sensation" is "practically impossible to measure." If just this one factor is "practically impossible to measure," and considering the other points that I have adduced, I am comfortable stating that human vision is nonquantifiable. The fact that light is not necessary for the experience of color seems to me a confirmation of my perspective, namely, that there is a fascinating separation between the wavelength of electromagnetic radiation and the human experience of and interaction with color, and that digital color models serve as a sort of bridge between wavelength in the physical realm and color in the physiopsychological realm. I admit that it is rather provocative to say that "color doesn't really exist" in the purely physical realm, but part of my argument is this: If psychological processes are involved in the formulation and interpretation of color imagery, and if psychological processes are considered to be outside of the purely physical realm, then from the perspective of human beings color comes into existence in a way that is not completely physical, and before that it is simply wavelength.
  Like. Reply
- - R
    Raymond Genovese August 20, 2018
    
    You’re welcome. I read your “rebuttal” and it is distinctly underwhelming. You have danced around some issues, offered defensive rationales, swapped out some terms, omitted others (e.g., inherently), and attempted to turn things around and place the burden on me. The insufficiency (in this context) of your, apparently, faith-based theory of color persists. It is not surprising that an EE with no training and SME in neuroscience, or a related field, does not understand or appreciate the complexity of the CNS required to write in detail on the instant topic. It is, however, both surprising and disappointing that they would continue to insist that they are “right” and to do so based on their consulting a book. I have an old copy of Horowitz and Hill on my bookshelf, but it does not make me an EE. As I told you before this article was published, there are serious scientific errors in the article that are typical when an author decides their expertise extends into areas that they know little about. This is such a case and it is unfortunate, because those errors did not need to be made in order to discuss models of color.
    Like. Reply
  - - R
      Robert Keim August 20, 2018
      
      I'll continue to ponder this and try to understand why the information that I have written—which is based on an authoritative resource and straightforward observations—is objectively erroneous rather than simply an expression of a valid (though perhaps unpopular or uncommon) perspective on a phenomenon that is not yet fully understood (as you yourself have said). It seems to me that the most problematic phrase was "mysterious and somewhat immaterial," and that has been removed. I don't recall ever claiming that I was "right." If I did I was mistaken; I do claim, though, that my perspectives are valid based on our currently incomplete knowledge of human vision and color perception. I am certainly not an expert in neuroscience, and I would never claim to be. I also would not attempt to write "in detail" on this topic. My discussion of human color perception is limited to generalities that were intended to provide an interesting and engaging introduction to the subject of digital color models. As I said, I'll continue to think about this, and I'll also consider the ideas and facts offered by other commenters. In the meantime, I included a note toward the beginning of the article to ensure that readers are aware of the debate and to make clear that they are in no way obligated to agree with my perspectives.
      Like.
    - A
      analog_headache August 24, 2018
      
      Bit harsh Raymond.
      Like.
L
lmatte August 20, 2018

I’m a bit disagreeing too on the common description of color as a fictional characteristic, invented by our brain. I’ve seen it repeated all the time, when someone introduces color in image or light processing.
The fact that I label as “red” some light doesn’t make it more fictional than an instrument determining it to be a “700 nm” wavelength. If “red” is just a perception, so it’s that “700 nm” unit, as it’s expressed in “meters”, which is just a fictional measuring unit that we invented - and a wavelength or a mountains height do exist even if we don’t invent numbers to measure them.
Quite different is when we refer to “white” as a color, as it’s the sum of different wavelengths, so it’s a perception of a more complex phenomenon. Then are similarly fictional all the musical notes from practical instruments, as they aren’t just a single frequency pure tones.
The fact we classify complex physical effects with simple names doesn’t mean they don’t exist. And that let’s me drink a glass of “water” even if it’s a fictional name for hydrogen and oxygen, in a specific ratio, plus a lot of metals, as other elements, bound together in a very complex way. Trust me, even “water” exists, no matter how I simplify it.

Like. Reply
- R
  Robert Keim August 20, 2018
  
  There's no doubt that this issue is influenced by how we interpret words such as "color" and "music." If color is simply another way to refer to physical characteristics of electromagnetic radiation, then color is certainly a physical reality, just like water and sound. In this article, though, I'm focusing on color in the context of a human being's perception of, interpretation of, and response to the physical characteristics of electromagnetic radiation. Within this realm of perception, interpretation, and response, color is created (based on the physical characteristics of visible EM radiation) by the human brain. I would make a similar argument for music. Sound waves certainly exist in the physical realm, and "sound" exists if you think of it as just a simple name for physical characteristics of sound waves. "Music," on the other hand, is fundamentally bound to a human being's physiopsychological perception of, interpretation of, and reaction to combinations and sequences of physical sound waves. In this context, music comes into its full existence within the person who is perceiving, interpreting, and reacting. According to my perspective, then, music (like color) has both objective characteristics and subjective characteristics. The objective characteristics are purely physical and quantifiable; the subjective characteristics are not.
  Like. Reply
A
adx August 24, 2018

Colour vision does tend to be a hot topic, I personally find many descriptions (not pointing at the article or comments specifically) to be overcomplicated as an explanation of the process.

My understanding is that the way human colour vision works is basically the same as the pictured Bayer array - the eyes of most humans “analyse” a spectrum into tristimulus values of red green and blue. These 3 bands can be combined in any mix of proportions, resulting in a description of colour that works equally well in a computer, a TV set, and our brains.

The “imaginary” label arises because the sensation of some of these mixtures is quite surprising to us - we see “purple” or “orange” as distinct colours, as if we are perceiving the spectrum directly, not something decoded from the simple measurement of only 3 channels of information. But there’s nothing more to it (in a philosophical sense) than that. This approach is typical of senses, whether in the brain or a machine - for example the way a stereoscopic image is decoded into a depth map and 3D sense of space. It’s just the way it’s done. Sure, philosophical musings on the emotional and aesthetic qualities of the result will tend to float off into the aether, but the process of how colours are sensed and decoded is easily explained by basic signal processing concepts.

Animals and machines can have less or more channels, with peak responses around different wavelengths. Insects often have a channel in the UV. An entry level spectrometer might break the spectrum into 128 bands. None of these devices or creatures are capable of resolving fine spectral structure, even if they are all capable of determining the wavelength of a single spectral line.

One colour model not mentioned in the article but of biological relevance is the YUV or luminance (white-black) and chrominance (yellow-blue and red-green bipolar channels). This is the result of the “debayering” process that occurs in the first layers of neural processing in the eye. Luminance is transmitted with higher resolution than chrominance, a natural result of debayering. This model does have perceptual relevance, and is why we can’t conceive of colours that are both light and dark, yellow and blue, or red and green (for example there is no reddy-green, but there is a reddy-blue).

The HIS model is more the decoding of discrete spectral channels into a notional place on the physical reality of a spectrum. Apart from white light, I don’t think this has any direct relation to the wavelength (unless we have some primordially wired understanding of rainbows), more just that this is the colour space available, and the real objects we see in our perception of reality exist with a range of different colours.

I know I have slightly oversimplified this, but I don’t think that greatly.

Like. Reply

Load more comments