In 2001, I cofounded Micoy with a few friends. Our goal was to create a 3D immersive video technology that had been previously deemed impossible. Within a few months we had created our first functional Experience Capture System (ECS) prototype. The ECS is a spherical multi-camera system that records video of an environment from many different viewpoints simultaneously. That video data is then merged into seamless 3D panoramic videos using proprietary image processing algorithms.
In addition to aiding in the design and construction of the ECS hardware, I developed the video analysis and mosaicing software that harvests the ECS's many 2D video streams and generates a corresponding seamless spherical 3D video. The resulting omnidirectional stereoscopic format is unique in that it maintains proper 3D effects throughout the entire spherical environment (check out this pdf for further explanation). We became the only provider of omnistereo video experiences.
Over the next couple of years, I was the sole developer of Micoy's image processing and video post-production systems and codesigner and developer of Micoy's video playback systems. In the process of refining out initial ECS designs, I expanded our omnistereo product line by adapting the techniques to computer generated imagery. I implemented omnistereo spherical and omnistereo fisheye rendering plugins for Mental Ray, allowing 3D Studio MAX and Maya users to generate and view imagery in our proprietary format.
Micoy's technological developments were impressive enough to win the Gold Award at the 2005 World's Best Technologies Showcase (sponsored by the National Association of Seed and Venture Funds, the Federal Laboratory Consortium for Technology Transfer, and the Southeastern Universities Research Association). I have written and filed five patents for Micoy's core image capture and processing technologies. Two patents have been granted (Pat# 6,947,059, Pat# 7,347,555) and three more are still pending (App# 20060227417, App# 20060256113, unpublished).
ImageFusion is a provider of high-end digital scanning and image processing technologies. They sell proprietary image scanning hardware and software systems designed to produce high volume, high quality image archives at costs far below those of their competition. They also offer a wide range of scanning, image processing, and digital archiving services.
My ImageFusion work began with the start to finish design and construction of a large document scanner for quick, non-destructive imaging of old newspaper archives. This included specifying, purchasing, and integrating all necessary mechanical components (machined aluminum frame, stepper motor, motor controller, linear actuator, proximity sensors, frame grabber, metal halide lamp controllers, power conditioner, encoder, etc). Once initial construction was complete, I programmed the necessary embedded motor controller functions and designed and implemented a GUI based control system for the scanner. The scanning application included a variety of optional real-time image processing functions and used multithreading techniques to ensure that the software system always outperforms the scanner's human operator. The finished scanner is capable of scanning a 30" x 60" area at 300 dpi in approximately 3 seconds. It outperforms commercially available scanners that retail for more than $80,000 with a construction cost close to $15,000.
After my scanner work was complete, I developed a variety of image processing algorithms for ImageFusion. This includes automated image registration and mosaicing applications for use with multicamera scanners, C# wrappers for twain and OpenCV, and a variety of optimized image processing routines for enhancing scanned newspaper archives (auto deskewing, auto border detection, auto cropping, auto color enhancement, OCR, lighting correction, headline detection, etc).
Reality Sports Technologies (RST) is a video based sports training provider. I worked closely with them on the technological development of their primary product, the Quarterback Development System (QDS). QDS uses video footage of live football plays to teach quarterbacks the art of reading defenses. The desired plays are recorded from the quarterback's point of view with large field of view camera systems and are then played back on a variety of computer devices. When the plays are viewed on a head mounted display, embedded in a football helmet, a head tracker allows the player to look around the field and experience the play just like he would in practice. If the video footage is recorded with Micoy's ECS camera system, the play can also be experienced in 3D.
My work for RST focused on their video post-production and playback processes. I created an immersive video playback system that supports both 3D and 2D video streams in a variety of input and output formats and allows for panoramic navigation with a variety of control devices, including head trackers. I also created both a semi-automated post-production system for Micoy ECS camera footage and a completely automated post-production system for footage shot with a fisheye lens system.