Sensing color with the TAOS TCS230
The TAOS TCS230 is a small, highly integrated color nsing device packaged in a clear plastic 8—pin SOIC. It reports，as analog frequency, the amount of shortwave (blue), mediumwave (green)，longwave (red)，and wideband （white）optical power incident onto the device。It can be ud in a variety of color nsing applications。Details of the device can be found in its datasheet。This white paper details the concepts and calculations involved in color nsing using the TCS230。
We will u the ColorChecker chart as an optical stimulus to work through a numerical example of color nsing。The chart，depicted in Figure 1, is manufactured and distributed by GretagMacbeth. The chart measures approximately 13 inches by 9 inches （330 mm by 230 mm)；it contains 24 colored patches arranged in a 6 by 4 array。Figures 2 through 5 overleaf show the spectral reflectance of the patches in each of the four rows of the chart – that is，the fraction of incident light that is reflected （with respect to an ideal diffu reflector），as a function of wavelength from 350 nm to 750 nm.
Figure 1 The ColorChecker contains 18 colored patches and a 6—step gray ries.
Figure 2 ColorChecker spectra, top row。
Figure 3 ColorChecker spectra, cond row.
Figure 4 ColorChecker spectra, third row.
Figure 5 ColorChecker spectra, bottom row （neutral ries）Figure 6 Cone nsitivities of cone photoreceptors are shown。Theshortwave-nsitivephotoreceptors are much less nsitive than the other two types. The respons of the mediumwave and longwave photoreceptors have a great deal of overlap。Vision is not nsitive to the preci wavelength of the stimulus: Whatatters is optical power integrated
under each respon curve。
Introduction to color vision
Photoreceptor cells called cones in the retina are responsible for human color vision. There are three types of cone cells，nsitive to longwave，mediumwave, and shortwave radiation within the electro-magnetic spectrum between about 400 nm and 700 nm. Becau the cone nsitivities are very roughly in the parts of the spectrum that appear red，green，and blue, color scientists denote the cell types as ρ，γ，and , the Greek letters for r，g, and b。（To d
enote the nsors R, G，and B would wrongly suggest a clor correspondence。）Estimates of the spectral respon of the cone types are graphed in Figure 6 above。
Light in the physical world can be characterized by spectral power distributions (SPDs). Colored objects can be characterized by spectral reflectance curves，such as tho of the ColorChecker. However，vision is innsitive to the exact wavelength of a stimulus: According to the modern theory of color science, all that matters is the integral of optical power underneath each respon curve. That there are exactly three types of cone cells leads to the property of trichromaticity: Three components are necessary and sufficient to characterize color。Some people might u the phra “color as nd by the eye，” but I con—sider that qualifier to be redundant at best, and misleading at worst：Color is defined by vision, so there is no need to u the qualifying phra “as nd by the eye," or to u the adjective visible when referring to color.
Overview of CIE Colorimetry
The spectral respons of the cone cells that I graphed in Figure 6 were unavailable to rearchers in the 1920s. Rearchers at the time ud psychophysical experiments，such as the famous color matching experiment，to tea out the data. The CIE is the international body responsible for color s
tandards.In1931，that organization adopted the color matching functions denoted x (λ），y (λ）, and z (λ），graphed in Figure 7.
Figure 7 CIE 1931, 2° color-matching functions. A camera with 3 nsors must have the spectral respon curves，or linear combinations of them，in order to capture all colors。However, practical considerations make this difficult. The analysis functions are not comparable to spectral power distributions！
Weighting a physical SPD under each of the three curves （that is, forming the wavelength-by—wavelength product)，and summing the results，forms a triple of three numbers, denoted X, Y，and Z。In continuous mathematics，three integrals need to be computed；in discrete math，a matrix product is sufficient。The X, Y, and Z tristim-ulus values characterize color. They are linear—light quantities，propor-tional to optical power，that incorporate the wavelength nsitivity of human vision. The Y value is luminance，which is ordinarily expresd in units of candela per meter squared (cd·m-2）。If you are measuring reflectance，the reflected tristimulus values depend upon the spectral characteristics of the illuminant，and their amplitudes scale with the power of the illumination。Relative luminance is the ratio of reflected luminance to the luminance of the illumination; it is also known as the luminance factor。
Figure 8 SPDs of various illuminants are graphed here. Illuminant A, shown in orange，is reprentative of tungsten light sources；it is deficient in shortwave power, and may cau errors in nsing blue colors. The blue line graphs the SPD of a Nichia white LED。There is a peak in the blue portion of the spectrum：Uncorrected，the nsor would report excessive blue values. The other four lines reprent CIE standard illuminants C，D50，D55，and D65。
In many applications，tristimulus signals (including luminance) scale with the illumination, and are otherwi uninteresting in themlves. What is more interesting is the ratios among them, which characterize color disregarding luminance. The CIE has standardized the projective transformation of Equation 1，in the margin，to transform ［X, Y, Z］values into a pair of ［x, y］chromaticity coordinates that reprent color disregarding luminance。The coordinates are suitable for plotting in two dimensions on a chromaticity diagram。
Eq 1 Chromaticity coordinates
Illumination
A nonemissive object must be illuminated in order to be visible. The SPD reflected from an illuminated object is the wavelength-by—wave—length product of the illumina nt’s SPD and the spec
tral reflectance of the object。Before light reaches the eye，the interaction among light sources and materials takes place in the spectral domain, not in the domain of trichromaticity. To accurately model the interactions requires spectral computations。When applying the TCS230，attention must be paid to the spectral content of the illumination and to poten-tial interaction between the illumination and the samples to be nd。Generally, the less spiky the spectra，the better. Figure 8 graphs veral illuminants.
Your application may involve nsing color，in which ca the preceding description applies. However，some applications of the TCS230 involve not so much estimating color as en by the eye but rather nsing physical parameters associated with optical power in the visible range。In such applications, to approximate the visual respon may not be the best approach: It may be more effective to take a more direct approach to estimating the parameters of the underlying physical process. The Color Checker
Equipped with knowledge of how spectra are related to colors，the plotting of chromaticity coordinates, and the dependence of colors upon illumination，we can
return to the ColorChecker. GretagMac-beth doesn’t publish or guarant ee the spectral composition
of the patches of the ColorChecker. However, nominal CIE ［X, Y，Z］values are published。The patches in the bottom row of the ColorChecker contain neutral colors; the numeric notations in the legends of Figure 5 reflect one tenth of the lightness (L＊）values of tho patches。
Thespectra graphed on pages 2 and 3 reprent the physical wave-length—by-wavelength reflectance of the patches. The spectral reflec-tances have been measured by color measurement instrument called a spectrophotometer。If you had access to a light source having perfectly even distribution of power across the visible spectrum, then the reflectance curves graphed here could simply be scaled to repre—nt the reflectance in your application. Practical light sources do not have perfectly even spectral distributions, so compensation is neces-sary: You must compute the wavelength—by-wavelength product of the illuminant’s SPD with the spectral reflectance of the chart。
We will first calculate the CIE ［X, Y，Z］values from the chart. （The values should agree with the figures provided by Gretag.）Then we will calculate the [R, G, B] values that will be detected by a TCS230。
To calculate CIE [X，Y，Z］, we take the 31×3 matrix reprenting the color matching functions （C
MFs) of the CIE Standard Obrver，and perform a matrix product with 31 spectral respon values as corrected for illumination。This produces the [X, Y，Z］tristimulus values. When chromaticity coordinates [x，y］are computed from ［X，Y，Z] through the projective transform in Equation 1，then plotted, the chromaticity diagram in Figure 9 results. The horshoe-shaped figure, clod at the bottom, contains all colors: Every non—negative spectral distribution produces an ［x，y] pair that plots within this region。The lightly—shaded triangle shows the region containing all colors that can be produced by an additive RGB system using sRGB （Rec。709）primary colors。This region typifies video and desktop computing (sRGB）。The points plotted in Figure 9 are the colors of the ColorChecker。White and gray values are clustered near the center of the chart。
Figure 9 Coordinates of ColorChecker patches are graphed on the CIE [x，y] chromaticity diagram。The horshoe enclos all colors；the triangle enclos the colors that can be
reprented in video （Rec. 709）and in desktop computing （sRGB）。
The TCS230
Figure 10 shows the respons of the four channels of the TCS230. The black
curve shows the respon of the unfiltered nsor elements。The red，green, and blue curves show the respons of the longwave-nsitive, mediumwave—nsitive, and shortwave-nsitive elements respectively。
As I mentioned on page 5，the CIE model of color vision involves inte-grating an SPD under the X(λ), Y（λ）, and Z(λ) color matching func—tions （graphed in Figure 7), producing X, Y, and Z values。To u the TCS230 to estimate color we perform an analogous calculation, but using the TCS230 nsitivity functions instead of the CIE CMFs：We integrate the SPD under the TCS230’s nsitivity curve s, and produce R, G, and B values。The device R，G，and B values will depend upon veral factors: the spectral content of the illuminant，the spectral reflectance of the sample, the spectral attenuation of any intervening optical components (such as the lens), and finally, the spectral respon functions of the TCS230。The various spectral phenomena are modelled by computing wavelength-by-wavelength products.
Figure 10 TCS230 spectral nsitivities are graphed here。The red，green，and blue channels are graphed in the corresponding colors; the gray line reflects the nsitivity of the clear (unfiltered）channel。Becau the respons are different from the CIE standard obrver，the values reported by the TCS230 are not colorimetric。However, suitable signal processing yields color infor
mation that is sufficiently accurate for many industrial applications.
Owing to the fact that the TCS230 is nsitive to infrared light （having wavelengths above 700 nm), and the fact that most light sourcesproduce power in the infrared region，typical applications include an IR cut filter in front of the TCS230. Figure 11 overleaf shows the respon of a typical IR cut filter。
To form a more accurate estimate of color requires processing the raw TCS230 R, G，and B values through a linear 3×3 matrix who coeffi—cients are optimized with respect to the spectrum of the illuminant，the spectral respon of intervening optical components，and therespon curves of the TCS230。The data processing operation can be reprented in matrix form as follows:
x=M•tEq2
The symbol t reprents a three—element vector containing the device values captured from a color patch. M reprents the 3×3 color correction matrix that we will apply to the values through matrix multiplication，denoted by the • symbol。The symbol x reprents the resulting vector of estimated [X，Y，Z] values。
We can u matrix notation to symbolize processing a t of three color patches at once, by arranging the three ts of device values into successive columns of a 3×3
