Research

Low-Latency Image Processing for Digital Night Vision, Augmented Reality

High-performance, low-power, and low-latency processing is needed to perform image processing algorithms in next-generation Augmented Reality Digital Vision Systems. Existing helmet systems and tactical operator head-mounted displays (HMD) are under continuing pressure to improve resolution and performance, while also under pressure to go digital in order to reap the benefits of information and multi-source fusion. But, digital imaging is lagging far behind analog technology in terms of fundamental operability--imaging latency at high resolutions is simply not acceptable in existing digital technology, and the user community rightfully resists moving to digital imaging until the latency problem can be solved. Perceptive Innovations is currently performing research which is proving to change that situation, creating high-performance digital imaging and future fusion applications, in a scalable small-SWaP product footprint leveraging commercial off-the-shelf (COTS) technology. Our objective is to develop architectures and technology for an embedded processor capable of implementing the image processing algorithms required for a digital head-mounted display for dismounted operators, at low enough latency for them to be usable without inducing operator fatigue and nausea.

  • Head mounted displays using analog technology are extremely important to the tactical operator, but can't take advantage of modern information overlays possible with digital imaging.

  • Digital imaging, on the other hand, suffers from latency which is unacceptable in close-quarters combat situations, and requires much more power, which shortens mission time on batteries.

Flash Pixel Processing

Perceptive Innovations has developed a technique for performing typical image processing functions such as Bayer-to-RGB Demosaicing, Noise Filtering, and most importantly image warping distortion correction. As shown below, a digital head-mounted vision system is likely to have significant non-traditional distortion, due to offset, freeform, and aspherical optical elements in the objective and eyepiece optics. Traditional CPU solutions to this require frame buffers, adding unacceptable photon-to-photon latency. PI2 Flash Pixel Processing restores true linear imagery with almost theoretically minimal processing latency. This technique is scalable to high resolutions and frame rates and multiple video streams, with only microseconds of added latency. To implement this we are leveraging the latest FPGA SoC devices, with power levels that permit the creation of multispectral augmented reality night vision goggles running on battery power. Flash Pixel Processing was devised on the USAF LLEVS STTR Program, and first realized in prototypes on the USAF BAARS Program, and are now the basis of the USAF DMARS and PEDBNVIS Programs.

Small Size, Weight, and Power (SWaP) Synthetic Aperture Radar (SAR) Processing

Perceptive Innovations has been working in a series of DoD SBIR Programs for nearly a decade, initially for the Air Force, and most recently in a series of Navy efforts, to advance the state of the art of SAR Processing.

The early Air Force effort investigated the processing synergies between SAR, which uses the radio spectrum, and LIDAR, which uses similar techniques, to form day, night, all-weather imagery using the light spectrum.

The recent and ongoing Navy efforts which started in 2020 focus only on SAR, with specific attention to applying the latest NVIDIA GPU technologies to create concentrated highly parallel processing power in a portable configuration, using greatly reduced size, weight, and power (SWaP).

These efforts have been very successful, reducing the SWaP by more than an order of magnitude while performing enabling the creation of realtime imagery that can keep up with real time radar collection during SAR flight missions. As a result, the Navy is expanding our research in 2025 going forward, this is very exciting progress and work.

Image Source: Air Force Research Laboratory Sensors Directorate Layered Sensing Exploitation Division

Advanced RF Transceiver

This important SBIR effort aims to develop a dynamically reconfigurable, minimal latency and power, 6U VPX RF Advanced Transceiver Architecture and Design Concept with increased bandwidth (IBW) and spectral coverage, greatly increased Processing Power, and general technology refresh of technology in an important Navy SIGINT hardware/software baseline. This effort will increase RF performance, bandwidth, and spectral coverage, while greatly increasing onboard processing resources, most importantly by leveraging 2024+ technology to avoid looming technology obsolescence. We will accomplish this without degrading performance specifications, and maintaining good compatibility with current software and firmware applications.

This Phase II effort will advance this technology to prototype and demonstrations, along with a design concept and manufacturing and commercialization plan to fabricate production Advanced RF Transceiver modules for future integration for multiple Naval applications.

Image Source: US Navy