The Cat & The Camera (warning a long read) but GREAT news

As I walked out with Dick Merrill on a cool February day through the Foveon Corporation parking lot in Sunnyvale to drive to the Lion & Compass restaurant, my hands were sweating. Merrill is an analog-chip designer, and analog people are different from digital people, who author microprocessors and software suites and work in teams and earnestly entertain journalists. Creators of devices without the assurance of simple numerical inputs and outputs-none of those tidy ones and zeros—analog people are loners, full of black arts and trade secrets, volatile resentments and spiky prejudices.

The Lion & Compass is no run of the mill Silicon Valley flesh pit. Four years before, industry eminence Carver Mead had brought Merrill here, hoping to enlist this silicon sage in his latest new company, Foveon. Mead, one of Caltech's treasures and godfather to four decades of history-making technologies, was probably the industry's most important intellect. But Merrill was embedded deep in the comfortable substrate at National Semiconductor. It took months to convince him to join Mead's foray into silicon's wild frontiers. When when he did finally come, he brought with him National Semi's money, manufacturing power and, most importantly, his own string of powerful patents.

Mead had been urging me for years to meet with Merrill, whom he compares with the industry's most famously feral analog chip prodigy, the late legendary Robert Widlar. This was less than reassuring. As I researched Microcosm, my 1989 book on the semiconductor industry, Widlar—whom my simple wish was to make a hero—led me on a wild geek chase from Lawrence Station in Sunnyvale to the Hilton in Puerto Vallarta, Mexico, without yielding a single interview. Merrill surely would not be so hard.

On this bright Silicon Valley noon, though, he seemed shifty, intense and quiet, as if he were planning a getaway. Unlike Widlar, he was at least unlikely to be drunk, or to flaunt a gun. And the trip to the Lion & Compass would be short. Even an artist of analog evasions could not escape during what Mapquest estimated as only a seven-minute jaunt—a few hundred yards up San Tomas Expressway from Foveon and then one exit down Route 101. But Merrill knew a short cut.

Out of the parking lot, I squeezed our ungainly gray Lincoln rental car across a small bridge and through a series of adjacent parking lots, snaking past Intel's Santa Clara plant and companies with names such as Swixx, Svee and Luminant. Merrill was muttering directions and assuring us that we'd be there in no time at all, what with the short cut eliminating the jams and terrors of 101. Chiefly engaging him, though, was a running denunciation of the new Nikon D1 digital camera, which he had used on a recent trip down the Mekong River with his Laotian wife. Chiefly engaging me, beyond looking for street signs, was Merrill's lecture. Even as assembled by Japan's best miniaturizing wizards, he railed, the 2002-vintage digital camera and its charge-coupled-device photosensors were horribly flawed. They inflicted ghastly whirls on the pictures where there should have been whorls, forced delays with their epic battery drain and wasted power (no ready plugs in Laos). "You only know you have a real camera when you want to take it mountain climbing," explained the wiry native Vermonter.

After several turns and a shameful capitulation toward Route 101, Merrill announced that we had missed the key turnoff and overshot the mark. Pulling out a Sprint PCS Motorola, my colleague Richard Vigilante began nervously dialing the restaurant. Moving abruptly into the next lane without first signaling or scanning the traffic, I put us in the path of an angry blue Mercedes, then wrenched us back into the other lane with inches to spare. It was no joke, but Merrill continued his discourse, politely ignoring my threat to his life. Turning onto a back street that featured a company sign with three X's in it—whether adult or pre-IPO, I could not descry—I suggested nervously that perhaps we should return to Foveon and eat in the cafeteria. "That will be fine. I don't eat much lunch anyway," I said. But Merrill still thought we would find the Lion & Compass any moment now, somewhere just across 101. We continued to thrash around for a total of some 35 minutes and several upfarts (which I am told is Finnish for U-turn), as the steering wheel slithered through my hands like a wet snake.

By that time, Richard V. had managed to raise the Lion & Compass, but the woman who answered the phone knew none of the streets or company names Richard cited, except 101. She suggested we try that. "Call back later when you get closer," she suggested. Actually, we had been driving in circles within a mile of the restaurant the entire time, and suddenly we found ourselves crossing the fateful freeway and taking another upfart before the Lion & Compass parking lot suddenly appeared, obscured by what Merrill had described as an "unreadable sign," presumably blurred by digital noise and aliasing. (Actually, it is a two-faced sign with "Lion & Compass" on both sides in large letters.) By that point, we knew enough not to buy a Nikon D1, but Merrill still needed to explain how Foveon, with his help, had arrived at its miraculous alternative.

Merrill is a man of medium height, a stubble of beard, blue eyes and a shy, abashed manner, as if he really wished he could return to work and escape these distractingly personal or intimate questions about semiconductor P/N—positive/negative—junctions and charge-coupled devices. But right away at the restaurant, he clarified Foveon's route to launching a revolution in image catchers. Eschewing digital couplings, charged or otherwise, its model was the surpassing king of all light processors: the utterly analog human retina. The road he, Mead and the rest took, however, more perfectly resembled our circuitous path to lunch.

Foveon and an older Mead-launched corporate sibling, Synaptics, had indeed traversed an ambageous route. Narrowly missing a possibly fatal collision with digital camera monster Sony; overshooting key turnoffs toward simple solutions; and pursuing lengthy roundabout paths and kludges to complex camera systems with three chips and exquisite precision optics and color supplied by epoxied prisms, the team frequently debated whether they really wanted to go to the destination anyway. (Synaptics, after all, went off on its own back to the cafeteria, and hit the jackpot with world-beating computer touchpads.) Maybe Foveon should have merely exploited the niche market for professional studio cameras that cost $70,000 apiece. Selling them for half that amount would still be cheap tuition for a trip down the learning curve. And maybe in the end they could lease the things for a few thousand a month, like a T-1 line, and get as rich as Pac Bell. . . .

Down the hall from Merrill's office at Foveon's Santa Clara headquarters is what appears to be a heap of colorful Navajo blankets, with a sheet of glass resting on the top. Only when you try to remove the glass do you realize that it is in fact a photograph, indistinguishable in hue and fiber from the pile of real blankets below. Now you understand Merrill's impatience with his mere Nikon. But the trompe l'oeil trick is just candy for your eye. This company has its vision set on far grander horizons. Foveon expects to capture the mind and enter the wallet of every photographer, still and motion, on the planet.

On the wall as you enter the Foveon building is a more significant photograph, one whose story portends the company's likely success in its imperial quest. Taken by Merrill, it depicts a vividly colorful totem pole in Vancouver, Washington, shot against a perfect blue sky. Sky, Merrill points out, offers the best visible index of the noise level in an imager. In Merrill's totem pole picture, the sky is impeccably blue. In the normal course of events in the silicon business, a new chip based on an original design and a novel manufacturing process goes through scores of iterations before it works the way its designers hoped. Some never do. But this impeccable image was the first product of the very first chip to emerge, under Merrill's direction, from the first run at the National Semiconductor wafer fabrication plant in Portland, Maine.

Thus, Foveon's full revolution. Yes, the totem pole picture is flawless, with exquisitely authentic hues and supreme resolution. But more crucially, it is the first working photograph made with a single-chip, full-color imager. Mead and Merrill's Foveon is no mere "digital camera," full of chips and microprocessors and mirrors and shutters—it is a fully solid-state machine, based around a single chip and virtually no mechanical paraphernalia, capable (like the human retina) of both still and moving photography. Single chips have a singular virtue: They can eventually be manufactured in volume for less than 80 cents a piece—about the price of the packaging. In other words, Foveon's X3, as the marketers have dubbed it, will make possible throwaway cameras with a resolution and accuracy better than today's most costly Hasselblad. Merrill simply plugged his new microchip into a circuit board, installed the board in a nineteenth-century camera chassis, snapped the picture and transfigured an industry.

Foveon's saga is really the life story of Carver Mead, which I first told in my book on the semiconductor industry, Microcosm. (See also "The Spectator Interview: Carver Mead," TAS, September/October 2001.) A pivotal point came in 1986, when the Valley was roaring back from its last great cataclysmic slump, with revenues dropping some 45 percent in a year. In a Caltech classroom in Pasadena, the eminent Gordon E. and Betty Moore Professor of Engineering and Applied Science, like many in his trade, seemed to be flaunting his august connections to the technologic eruption underway up north. Projecting the design of a massively parallel processor on the screen, he proposed it as a model for a revolution in computing and said: "Now I've been up in Silicon Valley, talking to the guy who made this thing and …"

Why is this class laughing? Don't they believe in Mead, the industry's first and most profound prophet of Very Large Scale Integration—VLSI—microchips? An intimate of many of the founders of the digital age, from inventor of the integrated circuit Bob Noyce to microprocessor architect Federico Faggin? Indeed, Mead had taught them much of what they knew about the design of digital devices. He had performed the crucial researches from which Moore's law itself derived, ordaining the doubling of digital computer performance every eighteen months. But the design he was showing on the screen to such friendly hilarity was not a digital device at all. It was analog, not a bit or a byte in sight. It was a schematic of the human brain.

Whether it was God or Gordon Moore, whom Mead had been consulting up there in the Valley, Mead's citation of the brain was not unusual in computer science. What was radical was that rather than treating the digital computer as a possible model for an extended or ultimately superior brain, he was offering the analog brain as a model of an incomparably more powerful computer. After twenty years as the industry's most authoritative proponent of the power of digital electronics, he was reversing direction and declaring the onset of a new era, the analog age.

Prone to dour observations about the perceptual powers of digital computers compared with those of, say, fruit flies, Mead believed even back in the 1980s that digital machines were reaching a dead end. He still does. In particular, he has long maintained that what he termed "neuromorphic analog VLSI" offers the possibility of a radically more effective image processor. Neuromorphic means derived from human models rather than from mechanical logic; analog means continuous representations by currents and voltages, rather than by digital math; VLSI signifies the kind of tiny geometries characteristic of leading-edge digital devices, such as the microprocessors that run your personal computer—tens of millions of transistors on a fingernail-sized chip. At the time Mead presented his brain model, analog devices were orders of magnitude larger than VLSI implied; the closest anyone was coming to neural computational models was basement-IQ artificial intelligence software. No one was building chips that simulated brain functions.

Later in the class, however, Mead presented the first actual example of such a machine, a silicon retina chip, modeled on the human eye, that could follow a rotating fan without aliasing (seeming to reverse direction, as spinning wheels do on movie film). Mead's retinal chip could also adapt to changing intensities of light. It was a significant first step toward creating a real-time imager on monolithic silicon. In the future, he believed, such devices could be the basis of improved forms of machine vision, or of a superior solid-state camera.

To the argument that analog processing would require too much power, he ran his chips at subthreshold voltages, like the micropower systems in digital watches. And to skeptics who argued that complex analog chips could never be manufactured economically, Mead offered the final audacious piece in his plan: he would use plain bulk CMOS—complementary metal oxide semiconductor silicon, the same manufacturing process used to fabricate the chips in your PC.

The key breakthrough involved light-receiving photoreceptors. CMOS designers face a fundamental problem: Between each cell's two transistors—the complementary negative and positive devices—is a potential bipolar transistor, called the "latch-up" or "parasitic" device. Its active area is termed the P/N—positive/negative—junction; minimizing the leakage of electrons across that atom-sized gap is a classic CMOS problem. But instead of trying to neutralize it, Mead—in his own famous phrase-"listened to the technology," to what the silicon "wanted" to do. And instead of neutralizing it, he enlarged and enhanced the bipolar transistor, increasing the P/N junction's sensitivity to light. Thus the latch-up transistor became an effective photoreceptor—so effective that it outperformed ordinary photodiodes, even as it was integrated onto the chip. In a stroke, Mead showed the way to create analog systems that scaled like digital ones, with costs that fall off cliffs in accord with Moore's law.

Within a few months of the Caltech lecture, Mead launched Synaptics, with the brazen ambition of applying his breakthrough to all the human senses, from hearing and imaging to touch. Joining him was Federico Faggin, the builder of Intel's first microprocessors and inventor of the self-aligned silicon gate that made them possible. Eventually, looking for a fab—silicon fabrication plant—that could manufacture Synaptic's revolutionary chips, Mead encountered Merrill at National Semiconductor.

Mead calls Merrill the most creative engineer he has ever met, in the combined disciplines of wafer fabrication, circuit design, device physics and photography. (Bob Widlar runs a close second.) The unforgiving fields of silicon electronics quickly reveal who is real, who truly listens to the technology, who feels the physics behind the device and who can match the intrinsic properties of the material with the essential demands of the function. And by contrast, it exposes who merely slips along on a secondary slurry of hearsay and hype, schussing through the media access channel toward the presentation layer, making PowerPoints and strutting the runways of industry fashion shows. Ask Merrill who the inspiration for his own work is and he does not say Mead, his boss—though he shares in the general awe of the man. ("Up at his house, I asked what the document was on his computer screen and he told me he's writing a book on gravity"—the final frontier of physics.) As his model and inspiration as a chip designer, Merrill names Bob Widlar.

Talking to Merrill snapped me back to my own past, nearly 20 years earlier, when all in electronics seemed new. I first encountered Carver Mead in a noisy restaurant at the Marriott near Newark Airport. He started drawing chip designs for me, on napkins and blank pages of his canonical book and on scraps of paper that piled up on the floor next to the booth. He already believed that some day electronic imagers based on his model of the human retina and on his analog chip wizardry would sweep the world.

Meeting Merrill also summoned from the mazes of my memory the recollection of a frantic rush down Sixth Avenue in New York in 1985, still chasing the elusive Bob Widlar. Widlar's inventions had sustained most of the key companies of Silicon Valley; his experiments with explosives in the National Semiconductor offices and with wild animals on the company lawn inspired many of the conversations down the street at Lawrence Station's bar and grill. His "101" operational amplifier was the first one-chip device with a stabilizing feedback loop, enabling its reliable use in any situation that required accurate boosting of a frail signal from the real world. Used in radios, disk drives, keyboards, microphones and oscilloscopes, Widlar's op-amp chip launched an analog movement that provided much of the early cash flow for the semiconductor industry, from Fairchild and National to AMD.

At Widlar's last International Solid States Circuits Conference, in New York in 1985, he won the best-paper award for the fifth time. I was pursuing the leonine drunk for a random quote on the street, when he abruptly turned back into the Hilton and headed for the bar. There I trapped him at last for several hours while he himself, systematically cut off all routes of his own escape, subsiding into a plush chair in a haze of smoke. Amid a growing flock of empty glasses, he quoted the apothems of Ambrose Bierce (the results are a chapter in Microcosm) until he fell asleep. Was Widlar crazy? Well, he cites Bierce: "Crazy—affected with a high degree of intellectual independence; not conforming to standards of thought and speech and action derived by the conformants from the study of themselves." He added with a sneer at me: "Reporter—a writer who guesses his way to the truth and dispels it with a tempest of words."

Widlar was the source of a key principle of the Foveon process, and one which had been ingrained into National's culture: exploit the potential of the silicon material, wherever it led. Rather than accept the prevalent designs or templates and try to work around them, as digital designers did, Widlar would tweak the material itself to achieve new unexpected functions.

If this were a dot-com story or a Telecosmic tale from the last few years, we might expect an early IPO and swift ascent to success, followed by wild acclaim and a possible crash. Under capitalism, where entrepreneurs depend entirely on the free responses of others, both customers and investors, the possibility of failure is a crucial filter. The company cannot control or even reliably predict the market itself. If outside buyers and suppliers lose faith, the company can come a cropper overnight, regardless of how valuable its goods and services ultimately may prove.

Thus, as it turns out, there would be no quick, giddy ascent for Synaptics, or for Mead's new analog vision. Synaptics struggled for seven years before bringing a significant product to market. Mead eventually broke with Faggin and relinquished his role as chairman, though retaining his shares. Only with agonizing delays would his analog VLSI vision gain adherents outside the circle of his own students.

I learned myself how hard such a path could be when I presented Mead's ideas to an audience at MIT in 1986, a few months after his classroom revelation of the analog retina chip. MIT's gurus at the time were advocating optical computers that used photonic binary transistors, directly translated from the digital silicon model. They laughed out loud at my assertion, borrowed from Mead, that the absence of digital optics in the brain—to this day, the world's only fully successful image processor—cast serious doubt on the prospects for digital image processing.

Only slowly did the analog retina chip eke forward. But in 1991, after some twenty iterations, Mead's "artificial retina" made the cover of Scientific American, with the blurrily captured image of a cat's face. The story, by Mead and his student, the late Mischa Mahowald, was confident: "The behavior of the artificial retina demonstrates the remarkable power of the analog computing paradigm embodied in neural circuits. . . . A neuron is an analog device; its computations are based on smoothly varying ion currents rather than on bits representing discrete ones and zeros. Yet neural systems work with basic physics rather than trying constantly to work against it."

During a long walk over the bare brown hills above Pasadena, Mahowald had explained to me the magic of analog. "Consider the electron. It took me six years of a Caltech education to understand the Schrödinger equation, on the wave behavior of electrons. But the electron already knows the Schrödinger equation. It doesn't think it's hard. It computes Schrödinger equations day and night."

Thus the key insight: For a digital computer to compute electron paths requires millions of individual calculations. A high-resolution digital imager must take billions of steps, consuming scores of watts of power and heat, to calculate a single picture. But the human eyes and mind see and sense by analogies, embodied in the neurochemistry and electrical patterns in the brain. They don't even that what they're doing is hard.

Taking this biological inspiration, Mead and Mahowald stressed the energy efficiencies of analog, which would soon be crucial to its triumph in handheld and other power-constrained devices: "In digital systems, data and computational operations must be converted into binary code, a process that requires about 10,000 digital voltage changes per operation. Analog devices carry out the same operation in one step and so decrease the power consumption of silicon circuits by a factor of about 10,000."

Next the Caltech pair pointed to a little-recognized virtue of analog devices, which originated with Widlar's op-amp and would prove pivotal in future imagers: "They respond to differences in signal amplitude rather than to absolute signal levels, thus largely eliminating the need for precise calibration. . . . Because [in Claude Shannon's theory of information] only changes and differences convey information, constant change is a necessity for neural systems—rather than a source of difficulty, as it is for digital systems. . . . The success of this venture will give rise to an entirely new view of information processing that harnesses the power of analog collective systems to solve problems that are intractable by conventional digital methods." But all this was theory. And regardless of its academic significance—the clear signs of an advance in understanding the nature of vision—the blurred cat on Scientific American's cover was a downer for most observers, belying both the authors' confident assertions inside and the grandiose plans of their company, Synaptics. Captured in only 2,500 pixels, Mead's cat hardly seemed a threat to the Moore's law digital juggernaut, which already propelled a thriving industry of machine vision for manufacturing applications.

Traditional digital imaging is based on the charge-coupled device—a kind of silicon bucket brigade resembling a single stretched transistor with thousands of information-bearing "gates." CCDs convert incoming photons into electronic intensities, then pass them on to digital processors that convert them back into an image. By the early 1990s, camera-company laboratories were already experimenting with digital imagers offering resolutions far higher than Mead's. Few were awed by his claim that he could scale his device to densities a hundredfold greater. A quarter of a million monochrome pixels scarcely endangered Kodak or Sony.

Even today analog has an archaic sound, calling to mind the very first computers, used for aiming anti-aircraft artillery fire during World War II. They transmitted radar signals, received their echoes, then calculated the telltale delays and lead distances for targeting, all without ever converting any of the measurements to digital ones and zeros. Cameras, slide rules, radios, television sets, telephones, ovens, airplanes and automobiles, all operated with nary a digital circuit—except possibly a simple clock—until the mid-1980s. Numbers appeared on panels and consoles, but no bits or bytes coursed through the circuitry inside, where values transpired as levels of charge, current, voltage, frequency and amplitude.

Today, so it appears, most remaining analog systems are giving way to the digital tide. Most television sets now receive digital sound and images, compressed in MPEG2 (Motion Picture Experts Group) codes. Movie theaters will soon download their fare in multi-gigabyte files, using technology from Qualcomm and Williams. Although radios remain dominantly analog, new digital radio systems are emerging, including Sirius and XM Satellite Radio. Video increasingly spurns tape, preferring digital video or versatile disks—DVDs. As the personal computer in all its forms extends its domains across the Internet, the bulk of music takes the form of the MP3 files used by Napster and its descendants. Images flash across the Net as digital GIFs (Graphics Interchange Format) or JPEGs (Joint Photographic Experts Group). Indeed, although the GIF and JPEG standards are too crude to exploit the superior verisimilitude of a Foveon image, they seem sufficient to most users. Good enough for the Net, the CCD-based digital camera continues its advance, dominating newspaper and magazine photography and making its way steadily into the huge amateur market.

In telecommunications, similarly, the march of digital seems inexorable, as every telephone, cell phone and network succumbs to the advantages of neat clean bits and bytes for bringing order to the noisy and nosy analog world we inhabit. Digital's progress has become an axiom, as if the reduction of all information to numbers was the essence of progress itself.

Steering against that tide of ones and zeros, Mead has long seemed a foolhardy surfer, sure to land hard on his face on digital silicon's ever-spreading beach. His crucial book—Analog VLSI and Neural Systems—was completed in 1989, just ten years behind his classic text on digital chip design, Introduction to VLSI Systems (written with Lynn Conway). But the Mead-Conway model for integrating multiple functions on a single chip triumphed so widely and rapidly—if often in crude defiance of Mead's larger vision-that he, himself, played chiefly the role of laureate and guide. Despite his prophetic early experiments and inventions with tunnel diodes and high frequency transistors, Mead's ideas for analog VLSI incurred solid resistance even from leading analog companies, including Analog Devices, Texas Instruments, Linear Technology and Maxim.

Synaptics was supposed to be the answer. At the outset, it targeted the three key human senses: touch, vision and hearing. But through the early 1990s, it made only fitful progress in fulfilling Mead's vision of neuromorphic devices and large-scale analog neural networks. Failure loomed. Then a string of ingenious mixed-signal inventions by Mead student Tim Allen broke through, in the realm of touch, where Mead himself had done little work. So superior were Synaptic's touchpads—used most visibly today as the navigation device for notebook computers—that they quickly took over the industry, and today hold an estimated 80 percent of the global touchpad market.

As readers of the Gilder Technology Report will know, Synaptics is now fully engrossed in the touchpad business, with possible new markets beckoning in other fields of pattern recognition. It burst through the market doldrums this January, with the first major technology IPO since the 2001 crash, the first public vessel of Mead's analog vision.

Over the long run, however, Synaptics' most valuable asset is likely to be its 15 percent stake in Foveon, which was spun out as a separate startup in August 1997. Its explicit mission was to do what Synaptics had done, but on the far bigger playing field of imaging and artificial vision.

The depth of the challenge becomes more apparent considering that before Foveon, no true silicon-based color imager existed. Conventional digital cameras do capture light on silicon, translate that into bits and from there produce color images. But their photoreceptors operate in black and white—more precisely, they measure the intensity of the light striking them, not its wavelength. They are, if you will, indiscriminate photon counters, which capture color only with a costly tradeoff. The photoreceptor for each pixel—roughly speaking the smallest component point in the image: think dots per inch in your ink jet printer—is covered by a filter, which lets in only red, blue or green light (the three colors captured by the rods and cones of the human eye). Together, each group of three photoreceptors does indeed capture a full-color image—but in the process, throws away two-thirds of the light and information that streams in.

The cost is even higher when you move to the next stage—reassembling that information into a viewable image. In conventional digital cameras, the final picture is produced by an elaborate digital guessing game, an algorithmic approximation performed by speedy (but expensive) digital signal processors. Because the algorithms function best by incorporating information from a range of nearby pixels, the guessing game for each can require hundreds of arithmetic operations. So, at mega-pixel levels, the cameras consume inordinate time and power—or both.

Ingenious as those software guessing games are, the original decision to toss away so much information permanently impairs picture quality. Arbitrary color patterns trigger rainbows, checks and whorls where nature intended a blue shirt or a plaid jumper. But as always in the digital realm, the preferred way over the rainbow is to do more with Moore: add more pixels and handle the burgeoning computational load with ever faster digital signal processors. But at gigahertz speeds, digital signal processors often hurtle blithely past crucial signposts from nature. And ultimately pixel size is limited not by Moore's law, but by less tractable limits like the wavelengths of visible light—at roughly half a micron, already close to the smallest features on digital circuits—and the resolution of the human eye.

With Foveon, Mead was determined to avoid throwing away information. As his long-time colleague and collaborator Dick Lyon puts it, they wanted "no guessing at all." Every pixel would register real features of the image—and every color—rather than rely on digital simulations.

For Foveon's first-generation cameras, that meant tossing out the usual red-blue-green filters and substituting a prism—splitting an image's red, blue and green light, and directing each stream of photons to its own single-chip imager. Instead of guessing, the signal processor would then combine the actual red, green and blue values for each pixel to produce a final image. The result: pictures of extraordinary quality that mocked their best digital competitors, and even rivaled the 8" x 10" large-format studio cameras that are the pinnacle for chemical-based photographic film. But the split and polish strategy also meant handcrafted modules of glue and prisms, mirrors and multiple microchips, all aligned with exquisite accuracy. The first commercial Foveon camera's $50,000 price tag marked Hasselblad as its chief competitor and professional studio photographers as its only market.

Fine with Mead. Let the competition scoff that handcrafted prisms would never be the basis for a viable consumer product. Flying under the radar, his team would be free to pursue its real goal—a single-chip, silicon-based color imager that would yield the best, cheapest and easiest-to-use mass market cameras ever made. Gone would be not only film, but virtually all the precision machinery that evolved around it, including, ultimately, the shutter itself. Left would be only lenses, batteries and silicon—a true solid-state camera.

By 1997, most of the ideas that might make Mead's vision possible resided on Dick Lyon's desk. As part of the agreement with National and Synaptics that gave birth to the new startup, Foveon had inherited all the intellectual property on imaging held by both companies. Most of it, Lyon recounts, could be discarded. But Merrill had been a compulsive patenter. ("Patents are a way to do something with an idea, without too much work. You dump it on the patent attorneys.") In particular, while at National, he had tried to create a truly differential analog technology out of CCDs, focusing on differences in energy rather than their mere intensity and thereby minimizing distortions from external changes such as temperature. The problem was that existing CCDs captured electrons (negative energy), but threw away what silicon engineers call the "holes" (positive energy). Merrill's solution was to keep both—the electrons and the holes-and balance them off, registering only the difference. It was the solution Mead and Mahowald had proposed in 1991 in Scientific American.

Merrill's breakthrough exploited a curious fact about silicon: that the frequency of light—its color, in lay terms—determines the depth on to which its photons will penetrate a chip's surface. Following the lead of Mead's bipolar photodetector, Merrill proposed burying P/N junctions at different depths on a chip, to separate and collect different colors at each pixel. Merrill doubted that the device would work cleanly enough to be usable in a high-precision application, but he submitted the idea to be patented, and then essentially forgot it.

Lyon, an expert on vision technology formerly at Apple, had developed a retina chip of his own, independent of Mead's. Spotting Merrill's patent, he was intrigued by the fortuitous color-filtering capability of silicon. As the highest frequency and highest energy color, blue would be captured near the surface, only a half micron down. The less energetic green could sink one-and-one-half microns before it agitated the silicon enough to be absorbed. The lowest energy photons—red—would penetrate some three microns down.

Lyon's knowledge of human vision made him more optimistic than Merrill about the technology's potential. Although a slight overlap of the blue, green and red levels in the silicon persuaded Merrill that the system would be noisy, Lyon observed that the human eye is noisy in almost exactly the same way. What matters in a camera, after all, is perception, not the scientific accuracy of frequency calculation. Lyon recommended tweaking the technology so that the pattern of colors in the silicon correlated closely with the pattern in the eye. Camera and eye would converge if all the colors were collected at every pixel—Mead's mandate from the outset. The result, Lyon predicted, would be an accurate rendition of colors as humans see them.

Recalling that his student Tobi Delbruck, son of the physicist turned Nobel Medicine Laureate, had once proposed a similar idea, Mead was immediately impressed by Lyon's logic. Bipolar photodetectors repeated the original retina design: burying three junctions completed it. If it could be made to work—and manufactured—the single-chip color image plane had the potential to repeat the magic of the digital microprocessor. It would be better, cooler, cheaper and lower power than its rivals. And it would scale, according to Lyon's calculations, to no fewer than 300 million pixels—far more than the human eye could absorb, with its 6 million color cones.

There were other advantages, too. By dispensing with all now-superfluous electro-mechanical paraphernalia of a conventional digital camera, the new chip could process images with virtually no delay. And it could do moving pictures—video—too, at unprecedented resolution.

Contemplating the potential, Mead and Lyon reintroduced Merrill to his own patent and suggested that he reduce it to a silicon device. The first results were far from usable, but Mead was encouraged enough to tell the others, "I think we have to bet the company on this." The first workable chip emerged from the National line a year later; Merrill installed it into an old camera and snapped his totem pole picture.

On the wall outside Mead's corner office at Foveon is a dramatic symbol of how far the new technology has come: a three-foot high image of the face of a cat. It could be the same feline that stared out from the cover of Scientific American a decade earlier. But instead of blurred monochrome, the new image offers a full-color vividness and verisimilitude perhaps never before achieved in photography. The image resolves every hair, whisker, glint and gleam of the feline fur and renders the eyes of the cat with a lifelike glow that gives the viewer the distinct and disturbing feeling that a formidable animal is watching him. Yet the picture is of a kitten.

On the surface, the year 2002 growls into view as a scavenger's feast—an epoch when much of the profit and property created during America's most cornucopian economic boom slips away into the maw of the most reactionary and obtuse parties: quasi-governmental bureaucracies such as AT&T and SBC, marginally competent foreign Internet companies such as Cable and Wireless, para-statal monstrosities such as France Telecom, and class-action racketeers such as Milberg Weiss Bershad Hynes & Lerach, who are taking the lead as plaintiffs in the Enron case.

With my Telecosm list—my buy-and-fold portfolio from the Gilder Technology Report—and the bold propositions of my book all now in a seeming shambles of the monetary deflation and telecom crash, I am inclined to be churlish. But following Mead's call to "listen to the technology," I still find myself rising in joy at its reveille, which is sounding with new urgency in the new year. Like the railroads that bankrupted a previous generation of visionary entrepreneurs—but built the foundations of an industrial nation—fiber-optic webs, storewidth breakthroughs, data centers and wireless systems installed over the last five years will enable and endow the next generation of entrepreneurial wealth. The stocks of companies with capital outlays geared low enough to avoid debt—or sufficiently risky enough not to have attracted it—have remained afloat despite the crash. On the fertile edge of the network, new semiconductor and optical innovations will bring radical new technologies to market, weaving the microcosm and telecosm together in a seamless web.

With Japan's great camera makers all now frantically putting in their bids, Foveon is poised to take a share of the global camera and video components market at least as big as Synaptics' 80 percent share of the worldwide touchpad business. National Semiconductor retains a 49 percent share of Foveon; X3s are rolling off the line at its leading-edge fab in Portland, Maine. Foveon has the single most powerful new commercial technology I have encountered since first meeting Mead at the Newark Marriott, generations ahead of existing digital cameras and video recorders. And thus it will dominate the next era of the Net, as the broadband vessel of images and videos superior in resolution and quality to the finest film images, but as portable as a Web page.

Following Synaptics and Foveon will be at least two other Mead companies. Impinj is focused on radical innovations in self-adaptive semiconductors. Applied Neurosciences has a probable breakthrough venture in speech recognition. A kindred company associated with Mead, Sonic Innovations, emerged in 1999 as the world's fastest-growing hearing aid company, no niche market in a graying world.

This jumble of apparently unrelated ventures embodies the singular new vision unleashed by Mead himself two decades ago in his classes at Caltech and brought to diverse fruition by an amazingly ingenious cohort of his students and associates. It signals the first hot flare of revival from the devastation of the last year in technology. New portents suggest that the Mead revolution is going mainstream.

The ultimate success of Foveon, after all, depends on the continued onrush of digital electronics. Each Foveon picture ultimately translates into a digital file of up to 40 megabytes (millions of characters; a 400-page book is roughly a megabyte of information). In order for Foveon pictures to take over the world of photography and film, these dense images will require broadband communications and trillions of bytes of storage, melded in a combination that I have dubbed "storewidth." Mead's analog technologies will change the world, but the world will have to change, too, to accommodate these new capabilities.

When Bill Gates launched his new XP network-centric operating system at a recent trade show, this king of the digital age did not begin by entering a password or clicking icons in a pop-up window. Instead, he put his finger firmly on a glowing biometric touchpad that recognized him, loaded his personal settings, and opened access to his personal files and digital kingdom. The company that supplied this open sesame is called Digital Persona. But its innovation in pattern matching is analog-based; its technical leader, Vance Bjorn, is a former Mead student, and another knight inerrant in his campaign to transform the interfaces between digital technology and our analog world.

Microsoft itself is developing a new standard for representing photographs on the net. The GIFs and JPEGs of the current World Wide Web are grossly inadequate to capture the vividness and verisimilitude of Foveon pictures. When the announced new standard is perfected, there will be yet another image of a cat—this one will leap off the computer screen toward you. Mead's revolutionary camera will make your screen brim with the intensity of a new era. Like the apparent glass on the top of the pile of carpets, your screen will disappear into a luminous new world of art and color.

Use of separate CCD's for red/green/blue is common in high-end ($10K-plus) video equipment; I would expect digital cameras to be similar in that regard.

Actually, the biggest thing I'd like to see as an improvement for electronic cameras would be the ability to shoot simultaneous or near-simultaneous pictures with different exposure settings and post-combine them. This would be an especially useful technique for video where tripling the bandwidth out of the CCD's would not pose a particular problem.

The way I would set things up would be to have the camera, rather than scan even scan lines every ~64us, then odd scan lines every ~64us, instead scan lines three times as fast in a shuffled arrangement such that three images were produced: the first would let each scan line collect only 4.096us worth of light (64 line times); the second would let it collect only 128us (2 line times); and the last would integrate over the rest of the frame. The first two images would go through a digital delay so that all three images would be perfectly synchronized with each other. They would then be processed, compressing contrast as needed for best effect.

This would allow a video camera to shoot scenes in what would otherwise be totally impossible lighting. For example, in some of the footage of my wedding, most of the church is a completely dark void while my wife shows up as a super-saturated white blob. Opening up the exposure would make the background show up well, but turn by wife into an even more smeared white blob; closng down the exposure would make my wife show up somewhat better, but completely obliterate everything else in the picture. Combining the both effects in real-time, however, should make it possible to get the best of both worlds.

Anyone who works with CCD's have any idea whether such an idea would be feasible? It unfortunately cannot be done at board-level (because the CCD's scan in fixed order) but should not require any exotic new technologies at the chip level--just the ability to read out scan-lines in shuffled sequence. While some post-processing would be needed, it would probably require anything beyond the level of sophistication already found in mid-range video cards.

Anyone find that idea intriguing?

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.