Technical Article

Understanding Color Models Used in Digital Image Processing

August 17, 2018 by Robert Keim

Learn about digital-signal-processing concepts that help us to store and manipulate color information.

Learn about digital-signal-processing concepts that help us to store and manipulate color information.

What is color? The question is far less straightforward than it might appear. For people, color is a visual characteristic that is deeply connected to emotional, aesthetic, and cultural aspects of the human experience. In the context of physics, color is merely a convenient way of referring to different wavelengths of electromagnetic radiation. In the purely physical realm, then, color doesn’t really exist. Light waves exist, and the wavelength of this light can indeed vary, but color—green, blue, red, and all the rest—is a delightful fiction created by the human brain, which for some reason looked at the horizon and decided that the mishmash of 600 nm-ish electromagnetic radiation should look something like this:



This discussion highlights the difficulty of translating color into the digital realm. Color is an inherently subjective phenomenon: wavelengths are objective things, but the visual appearance connected to a particular wavelength is established by the biological system that is receiving and interpreting the electromagnetic radiation. And since computers, robots, and sensor systems are typically used by people, the goal of digital color processing is to create information and images that are consistent with human vision.

What Is a Color Model?

The partially psychological way in which human cognition formulates colored imagery based on wavelength is not something that can be duplicated in a machine. Instead, machines use a color model, which I would describe as a mathematical approximation of the inherently nonquantifiable nature of human visual perception.

[Author’s note: The above information has proved to be much more controversial than I expected! Please take a look at the comments at the end of this article, and feel free to contribute to the discussion and to form your own opinion on the fascinating interaction between the physical characteristics of electromagnetic radiation and the human experience of color.]

Many color models exist, and presumably they all have advantages and disadvantages that make them more or less suitable for a given application. In this article we’ll discuss the two that are most commonly used in the context of digital image processing: RGB and HSI.


The RGB Color Model

As you probably know, RGB stands for red, green, blue. The RGB color model is additive: red, green, and blue light are added together in varying proportions to produce an extensive range of colors.



It’s important to keep in mind that real-life colors are not actually a mixture of red, green, and blue. Purple, for example, is purple, not a vector that extends 33 units in the red direction, 39 units in the green direction, and 127 units in the blue direction. Nonetheless, the RGB model has been wildly successful and is frequently used in sensor and image-processing applications.

In my opinion, the RGB model is, overall, quite intuitive. The key to understanding RGB image processing is recognizing that an RGB image is simply a composite of three independent grayscale images that correspond to the intensity of red, green, and blue light.



These three images can be processed separately and then recombined into a single image that human beings will perceive as having color.

Gathering RGB data is also straightforward, again, because RGB imagery boils down to intensity. A photosensor that measures light intensity becomes an R, G, or B sensor if you combine it with an optical filter, which means that you can generate RGB data using a unit consisting of three photosensors and three optical filters. In fact, I used a sensor like this for a project series that I wrote a couple years ago.

It would be reasonable to assume that color photographs or video imagery would require three full CCD or CMOS image sensors, but it turns out that a system can generate high-quality color imagery from one image sensor by using a Bayer filter and then applying specialized processing algorithms to the resulting data.


The arrangement of colors in a Bayer filter. Image created by Cburnett and taken from Wikimedia Commons.

The HSI Color Model

HSI stands for hue, saturation, intensity. This model is interesting because it can initially seem less intuitive than the RGB model, despite the fact that it describes color in a way that is much more consistent with human visual perception.

It’s true that the RGB model draws upon our familiarity with mixing primary colors to create other colors, but in terms of actual perception, RGB is very unnatural. People don’t look at a grapefruit and think about the proportions of red, green, and blue that are hidden inside the somewhat dull, yellowish-orangish color of the rind or the shinier, reddish flesh. Though you probably never realized it, you think about color more in terms of hue, saturation, and intensity.

  • Hue is the color itself. When you look at something and try to assign a word to the color that you see, you are identifying the hue. The concept of hue is consistent with the way in which a particular wavelength of light corresponds to a particular perceived color.
  • Saturation refers to the “density” of the hue within the light that is reaching your eye. If you look at a wall that is more or less white but with a vague hint of peach, the hue is still peach, but the saturation is very low. In other words, the peach-colored light reaching your eye is thoroughly diluted by white light. The color of an actual peach, on the other hand, would have a high saturation value.
  • Intensity is essentially brightness. In a grayscale photograph, brighter areas appear less gray (i.e., closer to white) and darker areas appear more gray. A grayscale imaging system faithfully records the intensity of the light, despite the fact that it ignores the colors. The HSI color model does something similar in that it separates intensity from color (both hue and saturation contribute to what we call color).

HSI is closely related to two other color models: HSL (hue, saturation, lightness) and HSV (hue, saturation, value). The differences between these models are rather subtle; the important thing at this point is to be aware that all three models are used and that they all adopt the same general approach to quantifying color.

The following three images are screen captures from a graphics program called Inkscape; they indicate the H, S, and L components of the three colors shown above (in the RGB section).


Click to enlarge.



We’ve covered some basic information related to color-representation systems that are useful in machine-vision applications. If you’ve done image-processing work that was not based on the RGB or the HSI/HSL/HSV color model, feel free to leave a comment and tell us about your experiences.

  • R
    Raymond Genovese August 17, 2018

    The author does not, in my view, understand nor appreciate the role of neuronal processes which are integral to any discussion of color. While the models of color seem to be presented well enough, much of the introduction is unnecessary for the central purpose of the article, inaccurate and, frankly, sophomoric.
    Color is a neuronal event. It is produced by the CNS. These points have been completely missed by the author when making statements like, “In the purely physical realm, then, color doesn’t really exist.” Neuronal processes are absolutely in the physical realm.

    Furthermore, statements such as “The mysterious and somewhat immaterial way in which human cognition formulates colored imagery based on wavelength is not something that can be duplicated in a machine…” are equally errant. If the intended definition of “immaterial” is “irrelevant or not important”, then the statement is wrong because the neuronal processes are obviously essential to the experience of color. If the intended definition of “immaterial” is “spiritual, rather than physical”, then the statement is also wrong because the neuronal processes are clearly physical. Moreover, there is no logical reason why a machine cannot duplicate those processes, at least eventually.

    The fundamental lack of understanding by the author is compounded in the following statement, “Instead, machines use a color model, which I would describe as a mathematical approximation of the inherently nonquantifiable nature of human visual perception.”  There is no logical reason to believe that the nature of human visual perception is “inherently nonquantifiable” and scientists have worked for years to do just that and have made great strides in understanding. To deny this strikes me as being no less than antiscience.

    Indeed, in some cases, light is not even necessary to experience color. It is common for most people to report that they dream in color. Many studies have shown that electrical brain stimulation produces color. The phenomena of chromesthesia, in which sound produces or is accompanied by color (particularly in musicians) is well established.

    It is unfortunate and disappointing that the author, who is obviously a skilled EE and writer, chose to use so much of the article to attempt to explain color without commensurate knowledge, when the space could have been used to add to the quite reasonable explanation of models of color.

    Like. Reply
    • R
      Robert Keim August 20, 2018
      Thanks for the comment. As I mentioned in our previous discussion, I realized that it would be a good idea to avoid the use of the word "immaterial," but I wasn't aware of the publishing schedule and the article was published before I made the revision. I changed that sentence to use "partially psychological," a description that is confirmed by the university-level textbook on which this article is based. I have provided solid arguments in favor of the nonquantifiable nature of human color perception, and you have not responded directly to those arguments. Furthermore, my textbook states that "one of the key factors in describing color sensation" is "practically impossible to measure." If just this one factor is "practically impossible to measure," and considering the other points that I have adduced, I am comfortable stating that human vision is nonquantifiable. The fact that light is not necessary for the experience of color seems to me a confirmation of my perspective, namely, that there is a fascinating separation between the wavelength of electromagnetic radiation and the human experience of and interaction with color, and that digital color models serve as a sort of bridge between wavelength in the physical realm and color in the physiopsychological realm. I admit that it is rather provocative to say that "color doesn't really exist" in the purely physical realm, but part of my argument is this: If psychological processes are involved in the formulation and interpretation of color imagery, and if psychological processes are considered to be outside of the purely physical realm, then from the perspective of human beings color comes into existence in a way that is not completely physical, and before that it is simply wavelength.
      Like. Reply
      • R
        Raymond Genovese August 20, 2018
        You’re welcome. I read your “rebuttal” and it is distinctly underwhelming. You have danced around some issues, offered defensive rationales, swapped out some terms, omitted others (e.g., inherently), and attempted to turn things around and place the burden on me. The insufficiency (in this context) of your, apparently, faith-based theory of color persists. It is not surprising that an EE with no training and SME in neuroscience, or a related field, does not understand or appreciate the complexity of the CNS required to write in detail on the instant topic. It is, however, both surprising and disappointing that they would continue to insist that they are “right” and to do so based on their consulting a book. I have an old copy of Horowitz and Hill on my bookshelf, but it does not make me an EE. As I told you before this article was published, there are serious scientific errors in the article that are typical when an author decides their expertise extends into areas that they know little about. This is such a case and it is unfortunate, because those errors did not need to be made in order to discuss models of color.
        Like. Reply