The Actuality Story

Perspecta, a 360° volumetric autostereoscopic display, was the flagship product for the Massachusetts-based Actuality Systems, and an invention that may have been ahead of its time. Inventor Gregg Favalora first began creating 3-D displays as a teenager in the basement of his New Jersey home and went on to found Actuality Systems, which had a 12-year run before being acquired in 2009.

by Jenny Donelan

ACTUALITY SYSTEMS, which was founded in 1997 by inventor Gregg Favalora and others, had basically one product, but it was a standout: a 360° autostereoscopic display that allowed users to view volumetric imagery in real time. Perspecta looked like something out of a science fiction movie – a transparent dome with a three-dimensional image floating inside (Fig. 1). The product was real, however, and useful, too. One application allowed medical professionals to view patients' internal organs in the round, enabling the physicians to better pinpoint locations for procedures such as radiation. The Society for Information Display awarded Perspecta a 2007 Gold Display of the Year Award for Display Application of the Year, for being the first high-resolution volumetric display to be used in preclinical studies for cancer treatment.

Despite the promise of Perspecta, however, Actuality Systems ceased operations in April 2009. The patents for various versions of the display, as well as for Actuality's flat-panel autostereoscopic display designs, were acquired that year by a product-design firm called Optics for Hire, for which Favalora now works. Favalora is optimistic about a new chapter for Perspecta and Actuality's collection of quasi-holographic technologies, especially now that 3-D is emerging as one of the hottest technologies in the market, with many commercial applications. The recent success of stereoscopic (with glasses) 3-D cinema seems to have sparked interest in autostereoscopic (without glasses) displays as well.

Fig. 1: Viewers could see all the way around the Perspecta dome display, which provided a 360° view of the volumetric image within.

However, it is difficult for anyone who has founded a company that lasted a little over a decade to look back without at least some regret. Actuality Systems was based on great scientific invention and high aspiration, and if it did well due to the youthful enthusiasm of its team, it also foundered for part of the same reason. In fact, one aspect of the story of Actuality, says Favalora, is how the best idea in the world may not succeed if the market is not ready. "Perspecta was a great idea," says Chris Chinnock, President of display-market- research and consulting firm Insight Media. "They pushed the technology as far as they could at the time. They were innovators, and they were just a little early."

Inventive Beginnings

Favalora was raised in northern New Jersey, in an entrepreneurial family with interests in both engineering and heavy construction. "I grew up surrounded by lenses and resistors and oscilloscopes," he says. "One day when I was in junior high, my father left his pipe-leveling laser in our basement, and I began playing around with it." As a result, Favalora started developing volumetric displays and continued to do so through high school, and then as an undergraduate at Yale. There, he created a sort of prototype for Perspecta, which he believes was at the time one of the world's highest-resolution volumetric displays. It worked by shining light onto a rotating piece of plexiglass. "The projector used 32 laser diodes, and the only thing I could do on a student budget was repurpose the laser diodes in penlights," he says. His professor designed drive circuitry to make the lasers run "faster than they should have," says Favalora, and the finished student project ended up in a display case at Yale, where it ran (to Favalora's surprise) for over 10 years.

While enrolled in graduate work at Harvard, Favalora won $10,000 in an MIT business-plan writing contest, so he left school and founded Actuality, along with high-tech entrepreneur Rob Ryan and several Harvard computer-science undergraduates. The company began in the basement of a Cambridge apartment Favalora shared with four students. The aim was to produce professional-quality volumetric displays, and the founders shared the assumption that people in mechanical CAD (computer-aided design) or pharmaceutical design would find them useful.

Financial, Technical, and Marketing Challenges

One of the first orders of business was to raise money – difficult to do during the height of the dot.com boom, says Favalora, because the emphasis was then on Web applications. "Everyone was talking about the Internet and we were making display hardware."

The team faced technical challenges as well. Perspecta's basic design featured focused light projected onto a rapidly spinning flat imaging plane. The projector, mounted in the base of the unit, worked with mirrors to relay 198 "slices" of a 3-D volume onto the spinning screen (see Fig. 2).

Fig. 2: To provide volumetric imagery, the Perspecta display projects a series of 198 2-D patterns, called slices, onto an optimized diffuser rotating at or above 900 rpm. The display sweeps the entire volume twice for every complete screen revolution, resulting in a visual refresh rate of 30 Hz.

An early challenge was getting the pixels, which were actually voxels (volumetric pixels), to line up properly in order to draw an image. This was a problem that had not yet been solved in a volumetric display with a spinning screen. The design team therefore had to invent the algorithms to get the voxels to line up in the 3-D space. "We ended up getting quite a few patents in that technology," says Favalora.

Favalora points out that almost all of the firm's early technical challenges would be easier to address now, due to the availability of fast graphics processing units (GPUs) and light modulators. For example, early on, Actuality successfully overcame an electrical engineering challenge, which was that in order to create an image with 100 million pixels that changed 30 times per second, the product required a projector that could change 6000 times per second. The only technology capable of that in 1999 was the TI Digital Light Processing technology. But DLP was not available as a standalone component and had no published specifications. Texas Instruments was not releasing that sort of information back then, says Favalora. So the team figured out how to control the chips themselves, hooking up with a company in New York that had already done the first half of the job. "By 2000, we were one of the only companies in the world that could drive a TI DLP at 6000 frames per second. Six young engineers in Massachusetts were getting contacted by huge companies about it – but we were required to keep the results to ourselves," says Favalora.

The final technical challenge was to design the optics and the mechanics to make it all work together. That was done, and Perspecta went on the market in 2002. There was nothing else like it. "As far as the actual viewer experienced it, it was quite nice, like gazing into a crystal ball," says Brian Schowengerdt, this year's SID Technical Program Vice-Chair for 3-D and a Senior Research Scientist in the Department of Mechanical Engineering at the University of Washington. Two of the great advantages of the display, he notes, was that multiple users could view it simultaneously and, because of the way it created the 3-D effect, it did not cause eye fatigue, as do many stereoscopic displays. A potential drawback, he adds, was that the imagery, due to its diffuse screen, had a "ghostly," translucent appearance, which was a quality that made it very suitable for certain applications (such as some types of medical visualization), but less so for others. Actuality's later solution for this issue – replacing the diffuse screen with a reflector – made the cover of Applied Optics in 2007.

Actuality did fill orders from universities that wanted displays to do volumetric research. But there was not a widespread market for a volumetric display costing $40,000 and upwards in 2002. Earlier on, Favalora (Fig. 3) says he had realized that "I was more technically minded at the time than business minded." He agreed with the board to bring in some seasoned managers as soon as the development work was done.

The first of these was Cameron Lewis, who had come from Netscape. He established a mandate to support continuing development of the product while also figuring out what market it could possibly be used for. Although Actuality's principals had talked about how Perspecta might be applicable to molecular visualization or medical visualization, they had not talked to many customers yet, and Lewis thought they should. So Favalora and some associates packed a Perspecta into a special crate and carted it in a van from San Diego to Seattle, visiting 30 companies in 2 weeks. They learned a great deal from this trip – not least that there was not quite as much interest in the display as they had thought. They learned quickly that molecular visualization was not the right market. They learned that venture capitalists in 2002 were hesitant to invest in display technologies. And they learned that there was potential for using the display in medical imaging. This last discovery was a theme that would persist throughout the life of the company.

Evolution and Funding

Because venture capital was first of all focused on Internet applications at the time, and venture capitalists were leery of display technology, Actuality ended up surviving on the generosity of "angel" investors. "The company had over 80 individual investors, which is astounding," says Favalora. All these people believed in the value of floating 3-D images, he says. At the time, Actuality was also receiving funding from NIST for work on floating imagery outside of a flat screen.

With NIST's backing, Actuality's mission expanded to develop technologies that would permit replacing any display surface with an autostereoscopic display. The team developed a broad collection of these "quasi-holographic" displays, which projected high-resolution imagery visible from a wide field of view. For the most part, flat-screen autostereoscopic displays had previously suffered from banding, low resolution, or small images. Actuality's engineers tackled all three issues, using fast TI DLP projection technology with a variety of beam-steering methods to produce floating 3-D images from a single work-station. These included prototypes and concepts for tabletop, handheld, and desktop 3-D.

Final Years

In 2005 (see Fig. 4), former iFire CEO Michael Goldstein was recruited by Actuality's board to run the company. Actuality began participation in a three-hospital 30-patient cancer study that looked at whether a 3-D volumentric display would help doctors plan cancer treatment. "We finished that study and, resoundingly, the display allowed doctors to derive better plans." (This was the application that earned Perspecta a SID Display of the Year Display Application of the Year award). But as it turned out, explains Favalora, investors were more interested in a company that could improve cancer treatment using just software.

Since the team had gained so much knowledge about medical systems, around 2007, the company completely changed direction and became Actuality Medical, a software developer for prostate-cancer procedures. "During the last 2 years of the company, we would wake up in the morning, put on scrubs, and go into the OR to record prostate-cancer procedures," says Favalora.

Investors were patient through the financial crisis of 2008, he explains, but in 2009, he was asked to return as CEO. Shortly after that, seeing that there was very little money left, he laid off the staff and shut down the company. At its height, there had been 23 engineers and manufacturing personnel at the company's Burlington, Massachusetts, headquarters.

Fig. 4: The team at Actuality Systems poses near a Perspecta display (center) in 2005.

Lessons Learned

"If I had to do it again," says Favalora, "I would not underestimate the importance of a product marketing plan. I would hire additional people who are really brilliant at product marketing so we could understand exactly what the customer wanted and then build that instead of building something and then trying to find customers." From the beginning, Favalora thought it was obvious that a 3-D display was needed and wanted. "And almost universally," he recalls, people's reaction on seeing Perspecta would be to say, " 'wow, that is the coolest thing I've ever seen.' " Or people got excited because the display allowed them to see their data immediately, without having to fiddle around for 15 minutes before they could understand it. But those things did not translate to volume sales.

"There were also comments that we needed to offer more software applications," says Favalora. He thought they were fulfilling that need by offering a software development kit. But people did not want to have to create new software and systems to use the display. "Another lesson we learned was to provide a system rather than a component," he says. It was the equivalent of selling an engine when people wanted a whole car.

The Future

Perspecta itself was a technical success in many ways. The display seldom failed to fascinate, and, at trade shows, Actuality's booth would be swamped. About 25 Perspecta displays are still out in the field, mostly at universities. A graduate student recently earned his Ph.D. with award-winning work involving one.

And then there are the patents – 19 of them, and six pending. Actuality started with the idea for Perspecta and ended up with an extensive portfolio covering prototype-proven 3-D autostereoscopic technologies in volumetric displays, image computing, and quasi-holographic displays. They were acquired by Optics for Hire in 2009 and are now being represented by IP broker Quinn Pacific. Favalora notes a "land-grab" for autostereoscopic 3-D technologies, as consumer electronics and entertainment firms struggle to make sure they do not lose their market positions. Videos explaining several of the patents and their applications are now online atwww.youtube.com/user/opticsforhire.

Is there really a market for a device like Perspecta now? Favalora is confident that autostereoscopic products will become widespread, in step with interest in stereo 3-D. And he notes that it is possible to make Perspecta-type displays available at prices ranging from $1000 to $10,000 apiece, and quasi-holographic systems based on Actuality IP for even less.

Schowengerdt believes that a lower price point could make Perspecta a viable prospect for applications such as gaming, particularly in arcade or other public environments. "It would be an excellent platform, especially for multiplayer games," he says, adding that advertising is another possible area for exploration. So time will tell if new market conditions (and patent buyers) will bring Perspecta and the other Actuality Systems IP-based products back to life again. If so, the story might turn out very differently. "It would be a new game for someone entering this territory now because affordable components are finally meeting up with consumer demand," says Favalora. •