Breakthrough in Photography: The Next Photography Revolution

"It's easy to have a complicated idea," Carver Mead used to tell his students at Caltech. "It's very, very hard to have a simple idea."

The genius of Carver Mead is that over the past 40 years, he has had many simple ideas. More than 50 of them have been granted patents, and many involved him in the start-up of at least 20 companies, including Intel. Without the special transistors he invented, cell phones, fiber-optic networks, and satellite communications would not be ubiquitous. Last year, high-tech high priest George Gilder called him "the most important practical scientist of the late 20th century."

Digital cameras have relied on image sensors that can't do what color film does: record all three primary colors of light at each point in the image. Instead, each light-sensitive point in the sensor measures just one color—blue, green, or red—and complicated software in the camera calculates the missing colors. Foveon's breakthrough X3 chip solves the problem with a three-layer design that captures red, blue, and green light at each point. To demonstrate quality differences, the monarch butterfly on this page was photographed with three cameras: an $1,800 Sigma SD9 with an X3 chip; a $300 Nikon Coolpix 2500; and a $2,300 Nikon 35 mm F5 film camera. Insets show magnified detail from each camera's image.

And now one of Mead's simplest ideas—a digital camera should see color the way the human eye does—is poised to change everything about photography. Its first embodiment is a sensor—called the X3—that produces images as good as or better than what can be achieved with film. That would make the X3 the most important advance in photography in nearly 70 years, but the long-term implications are even richer. In a year or two, you will be able to pack a true hybrid camera on vacation. It will take high-resolution stills, or upon the flip of a switch, it will take full-length, full-motion video far exceeding the capabilities of present-day hybrid cameras. In the long run, X3 technology could even make cell-phone video sharp enough to project onto a big-screen TV, which would make dandy travelogues to send back to the folks at home, or enhance collision-avoidance systems in automobiles, or improve robot vision.

X3 is the latest and most innovative product from Foveon Inc., the Silicon Valley digital-imaging company that Mead, 68, founded in 1997. Named for the fovea centralis—the part of the human retina where vision is sharpest and most color perception is located—Foveon took as its mission another radically simple idea Mead loves: "Use all the light."

Don't cameras already use all the light that enters the lens? Film cameras do, but digital cameras, with few exceptions, don't. As Mead puts it, "They throw away two-thirds of the light." That makes sense only if you understand how a typical image sensor works. It's basically a rectangle of silicon on which millions of microscopic light-sensitive pixels (technically they're not pixels, but that's what these light-sensing points have come to be called in the digital-camera business) are arranged in a grid. Pixels can't sense color. So a checkerboard of tiny red, green, or blue filters must be bonded to the surface of the sensor so that each pixel lets in one of the three primary colors of light. In so doing, it blocks out the other two.

By comparing each pixel's single-color reading with that of its neighbors, software can derive the values of the two missing colors at each site. That takes approximately 100 calculations per pixel. In a four-mega-pixel camera, a size commonly available today, that adds up to a lot of number crunching. The process is called interpolation, and Mead has a less kind name for it.

"It's a hack," he says. "They have to do all this guesswork to figure out what they threw away. They end up with a lot of data, but two-thirds of it is made up. We end up with the same amount of data, except ours is real."

That is because X3 does what until now only film has been able to do: in one exposure, on one image plane, measure all three primary colors of light at every point on the picture. By doing so, it does away with the bugaboo of so-called mosaic sensors, which often guess wrong, especially at the edges of complicated patterns, introducing moiré effects and jagged color errors called artifacts.

Sensing all three colors at each pixel sounds simple, but more than one industry analyst has described it as "the holy grail" of digital photography. "Engineers have been trying to solve this since the earliest days of digital imaging," says Alexis Gerard, publisher of The Future Image Report. Phil Askey, whose exacting equipment tests on his Web site, dpreview.com, are must reading in the trade, says, "This could be the first sensor to truly surpass film."

Carver Mead says, "The eye itself has taught us that remarkable things can be accomplished by building intelligence into the image plane. An intelligent image plane gives you higher quality photography with less demanded of the user."

The only camera to contain an X3 sensor now is called the Sigma SD9, a single-lens reflex with a price tag of $1,800 (not including the lens). But about this time next year, point-and-shoot cameras should be available from other manufacturers with X3 technology. They will have chips with slightly less than half as many pixels as the chip in the Sigma and sell for about $500. To be sure, Foveon will not find it easy to elbow its way into a market heavily committed to existing technology. But it has some influential advocates, including Microsoft's Gates.

X3 is based on a well-known property of silicon: It absorbs different wavelengths, or colors, of light at different depths from its surface. A standard wafer of pure crystal silicon—the polished disc, five to eight inches in diameter, on which most microchips are made—is about 1/25 of an inch thick. The absorption of visible light takes place within 1/10,000 of an inch of the surface. If you think of that 1/25 of an inch of silicon as if it were a place in the ocean where the water is 1,000 feet deep, then all the light absorption would be taking place within two or three feet of the surface. At that scale, a human hair would be about 50 feet thick.

What Foveon has done is imbed a sandwich of three light sensors within that first 1/10,000 of an inch. How they do it is a guarded trade secret, but the principle is basic physics. Blue light, which has the shortest visible wavelength, about 1/50,000 of an inch, is absorbed closest to the surface. Green light, which has a longer wavelength, penetrates a little deeper. Red light, with the longest wavelength, about 3/100,000 of an inch, burrows down farther before it is absorbed. As the photons strike the silicon atoms, electrons are released. These create electrical charges that the sensors measure.

It took film almost a century to figure out the best way to do color. But once Kodachrome was perfected in 1935, competing schemes largely faded away. By devising the simplest and most reliable solution to the problem—a three-layer emulsion—Kodak won the color war. Since then, color film has undergone many refinements, and other companies have grabbed significant market share. But thethree-layer emulsion is still gospel.

"Exactly the same thing is going to happen with electronic capture," says Mead. "X3 is going to be the surviving image-capture technology. There's no question about it."

If the rest of the industry isn't quite ready to throw in the towel, it's because the technology that Mead is up against has been king of the hill, despite all its flaws, almost since it was invented.

If you have ever bought a video or digital camera, you have probably seen the letters CCD stamped on the box. They stand for charge-coupled device. Versions of the CCD were invented independently at Bell Labs and Philips Electronics about 33 years ago. They produced much better images than could be obtained with other kinds of solid-state sensors, and they took off. Specialized long-exposure CCDs have long proved invaluable in astronomy, and the CCD put video cameras and digital still cameras under countless Christmas trees.

If the CCD was the hare of the digital-imaging race, something called CMOS, for complementary metal-oxide semiconductor, was the tortoise. CMOS is a sophisticated process developed in the 1960s that produces chips with many transistors. Without it, X3 would not have happened. Although CMOS chips didn't make good images at first, they made terrific microcircuits and became the backbone of the computer revolution. CMOS was the technology Gordon Moore, the first president of Intel, had in mind when he made his famous 1965 prediction—subsequently known as Moore's law—that transistor density on a chip would double every year. Before he refined his prediction, changing it to doubling every 18 months, Moore consulted with an expert in circuit miniaturization—brilliant young Caltech professor Carver Mead.

Most sensors in point-and-shoot digital cameras are smaller than a fingernail, but the X3 chip is closer to the size of a frame of 35 mm film. So are the sensors in other top-of-the-line digitals, but the X3 is complex to manufacture because of its three layers and its transistor density. "If you were to buy a Foveon chip," says Chris Joyce, director of process technology at National Semiconductor, "you could deprocess it and figure out what we've done. But you wouldn't be able to figure out how we've done it."

True, but partial.

People cannot do the same things with film that they could before.

Film has become more immune to variations in exposure and processing - and therefore less possible for the "people" to vary their results.

As for digital cameras - forget it. Unless the subject is motionless, the quality of image is far below that of photography decades ago.

Film has become more immune to variations in exposure and processing

True. If I'm not mistaken (I'm speaking here B/W which is what I love to do) the newer films are thin emulsion. Meaning you can't really make use of all the zone system offers.
As for digital for better or worse it is the wave of the future. What gets me excited about this new chip is they put one on a film holder and made an image. Now I'm a large format guy so the idea of being able to get my wista out again causes my heart to go pitter patter.

Look at T-max. You can't push and pull it nearly like Tri-X for example, much less the older films.

It's meant to be immune from lax development times/temperatures - hard to screw it up, but that means your variations in development have less (and less predictable) effects.

And yes, digital is the future, if for no other reason than environmental restrictions.

But with each step toward a more error-free consumer product, the ease of use increases, but the flexibility, the possibility of craftsmanship decreases.

If digital does improve markedly, and perhaps this is the breakthrough necessary, then I'm all for it. And I enjoy the speed and ease of a digital darkroom - but it comes with a definite decrease in quality of the final product.

And, I'm predicting it will be the same story, quicker, easier, lower final quality. For example, video claimed the same thing, but movies on film still have a very noticably superior quality.

BTW, have you ever looked at the detail on a deguerrotype?

Or a large format contact print of Weston? Or what Adams did with Tri-X?

I'm glad that photography has become automatic for the unskilled; I'm sad that it tends to replace the tools that were available for those who wished to develop a personal craft.

It is important that the photo begin in the eye of the photographer, yes, but that eye saw the possibilities using the flexible, malleable and quality of the materials available. It may not be long before those have gone away - unless you wish to make your own.

(Ever seen the book, "The Light Catchers"?)

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Here comes a digital-camera chip that could change everything