Java greyscale conversion

Using Java, how do you convert a color image to greyscale? The most straightforward answer I've found was on Code Beach, and is as follows:

BufferedImage image = new BufferedImage(width, height,  
    BufferedImage.TYPE_BYTE_GRAY);  
Graphics g = image.getGraphics();  
g.drawImage(colorImage, 0, 0, null);  
g.dispose();

Create a BufferedImage of type TYPE_BYTE_GRAY, and grab its Graphics object. Draw the color image onto it. Fin.

The described method uses a small amount of client code for a fast, good quality greyscale conversion, but a lot goes on under the hood to make this happen in the Java runtime jar. Tracing through the code, eventually you arrive at the getDataElements method of java.awt.image.IndexColorModel, and the following line is executed over every pixel in the image:

int gray = (int) (red*77 + green*150 + blue*29 + 128)/256;

This takes the red, green, and blue components of a pixel's composite color, and calculates its relative brightness (on a scale here that is, oddly, between 0.5 and 1.5), and later uses this value to mix a grey color of the same brightness for the replacement pixel.

A quick (and admittedly non-expert) course on color theory as it relates to computers:

Your computer screen does not have pixels that can change color. For every perceived pixel, there are three "lights" (for simplicity's sake) very close together that are the primary colors red, green, and blue. When all three lights are off, you see black. When they are all turned on at full brightness, you see white. Mixing the lights at different power levels produces very close to the full range of colors that our eyes can perceive.

The lights have 256 power levels, for a total color space of 16,777,216 colors (256 cubed). Black is seen when red, green, and blue are all 0; white when they are all 255. There are also 254 shades of grey, seen when the red, green, and blue values all match and are between 1 and 254. This brings us back to the brightness equation above: find a color's brightness, scale it between 0 and 255, and you can make a grey with the same brightness by setting the red, green, and blue of the pixel to that value. Repeat for all the pixels in an image and you have a greyscale copy of it.

Whence the 77/150/29 ratio?

In 1982, when I was 11 years old and a holy terror on my 6th grade teachers, the International Telecommunication Union published Recommendation 601 for transmitting video signals digitally. In it the following luminance (brightness) equation is mentioned in section 2.1:

E'Y = 0.299 E'R + 0.587 E'G + 0.114 E'B

These numbers add up to 1, where 77, 150, and 29 add up to 256. Dividing each of those by 256 gives a close approximation to the Rec 601 ratios, with the added benefit of using only integers and dividing by a power of 2, which computer processors are very good at. These specific ratios of red, green, and blue to determine luminance can be traced back to the XYZ color space, created by the International Commission on Illumination (CIE) in 1931, which in turn references work by James Clerk Maxwell (Experiments on colour, as perceived by the eye, with remarks on colour-blindness [1855]) which itself references work done by none other than Isaac Newton in the book Opticks (1704) which talks about breaking light into its component parts using prisms, discussing compound vs. homogeneal light, building lenses and telescopes, etc.

I won't attempt a full analysis of all these works to trace the source of the numbers, but suffice to say in the 19th century, people stared at colored lights and twisted knobs to match colors, and scientists wrote down the final settings. After some math and repeated experiments, it was determined that green light looks pretty damned bright to us, red a little less, and blue less than that.

Are there better ratios to create a crisper greyscale image?

In 1993, the ITU published Rec 709, a standard for modern HDTV systems. In section 4.2, the recommended luminance formula for the 1250/50/2:1 scanning standard used the old Rec 601 values. For 1125/60/2:1 (1080i) scanning, new values are recommended:

E'Y = 0.2126 E'R + 0.7152 E'G + 0.0722 E'B

That section's footnote says "The coefficients for the equations have been calculated following the rules laid down in SMPTE RP177-1993." This refers to a 1993 publication called "Derivation of Basic Television Color Equations" by the Society of Motion Picture and Television Engineers.

...and for the low cost of $50, I can buy the paper and see what their math was. From what I gather from other sources, the new calculations change when Gamma levels are applied, with the result being values better lined up with the 1931 XYZ color space. Finding out more isn't worth $50 to me. Some hoodoo is done with the gamma to produce a better luminance value - I'll take their word for it.

Can I use the new values to greyscale an image in Java.

Yes. Instead of using Graphics.drawImage and letting Java figure out what to do with the pixels, you have to get your hands a little dirtier and work with PixelGrabbers, bitwise operators and integer arrays. But it turns out to not be that difficult.

Colors are defined in Java as 32-bit integers in four octets: alpha(opacity), red, green, and blue. For example, 0xff007f00 would be an opaque hunter-green, with the alpha set to ff(255) and the green set to 7f(127), with red and blue at 0. To take this integer value and pull out only the green part, you can either create a Color object and call its getGreen() method, or not waste the construction overhead and just do the math: right-shift 8 bits, then bitwise "and" the value to 0xff to zero out all but the last octet, with a statement like this:

int g = (color >>  8) & 0xff;

Getting the red value is similar, just bitshift 16 places instead 8, and with blue no bitshift is needed, just the bitwise "and".

int r = (color >> 16) & 0xff;
int g = (color >>  8) & 0xff;
int b = (color      ) & 0xff;

These values can then be used to construct a luminance value by plugging them into the Rec 709 numbers:

int lum = (int) ((double)r * 0.2126 + (double)g * 0.7152 + (double)b * 0.0722);

And finally a grey can be constructed by creating an integer whose red, green, and blue values all match the luminance number. This can be done with little overhead by left-shifting and using the bitwise "or" function:

int greyColor = (lum << 16) | (lum << 8) | lum;

The sample code below is a method that can be called specifying which luminance standard to use. A PixelGrabber grabs all the RGB values and sticks them into an integer array. Then the array is iterated over, converting all the colors to matching greys, and finally updating a BufferedImage with the new pixels:

private Image greyScale(int mode, Image sourceImage) {
  double redLum, greenLum, blueLum;
  switch(mode) {
  case 601:
    redLum   = 0.299;
    greenLum = 0.587;
    blueLum  = 0.114;
    break;
  case 709:
    redLum   = 0.2126;
    greenLum = 0.7152;
    blueLum  = 0.0722;
    break;
  default: return null;
  }

  int w = sourceImage.getWidth(null);
  int h = sourceImage.getHeight(null);
  int[] pixels = new int[w * h];
  
  PixelGrabber pg = new PixelGrabber(sourceImage, 0, 0, w, h, pixels, 0, w);
  try {
    pg.grabPixels();
  } catch (InterruptedException e) {
    e.printStackTrace();
    return null;
  }

  BufferedImage greyImage = new BufferedImage(w, h, BufferedImage.TYPE_3BYTE_BGR);

  int offset = 0;
  for(int y = 0; y < h; y++) {
    for (int x = 0; x < w; x++) {
      int color = pixels[offset++];
      int r = (color >> 16) & 0xff;
      int g = (color >>  8) & 0xff;
      int b = (color      ) & 0xff;
      int lum = (int) ((double)r * redLum + (double)g * greenLum + (double)b * blueLum);
      int greyColor = (lum << 16) | (lum << 8) | lum;
      greyImage.setRGB(x, y, greyColor);
    }
  }
  return greyImage;
}

Here are a couple screengrabs from an app using this method to compare the original and greyscaled images.

First using the 601 standard:

Then the 709:

The difference is subtle, but the print on Scout's dress is a little darker and better defined in the 709 image. And with that anecdotal and completely subjective comparison, I declare the 709 formula the better of the two. On a whim, I shot Oracle an email about the IndexColorModel class using the outdated formula, showing a simple change to use the updated one:

int gray = (int) (red*55 + green*183 + blue*18 + 128)/256;

I doubt that will come to anything, but, you know, it was worth at least mentioning it to them. If I get the added bonus of my line of code in the official rt.jar, so much the better.