Unlocking Real-Time Hyperspectral Data: Video vs. Imagery

Hyperspectral imaging combines spatial and spectral information into a single output to unlock insights invisible to the naked eye and traditional cameras. However, capturing the fine spectral details in reflected light increases the complexity of the technology required. Typical hyperspectral imagers need: 

• Long exposure times
• Specialist equipment 
• Lengthy setups 
• Expert users 
• Time-consuming data analysis 

Simply put, cramming significantly more data into each frame leads to low frame rates. This has limited the development of video rate hyperspectral imaging. Plus, previous systems that have achieved high frame rates have done so at the expense of spectral information, limiting their use. 

The Living Optics camera is the first device to bring high spatial, spectral, and temporal resolution to the mass market, enabling real-time hyperspectral imaging at video frame rates. Designed for real-time analysis, the camera combines snapshot technology with high-power embedded compute, allowing users to monitor time-dependent events and quickly respond to changes as they occur.  

Line-Scanning vs. Snapshot Technology 

While previous hyperspectral imaging systems have mainly relied online-scanning methods, video rate hyperspectral imaging is made possible thanks to snapshot technology. 

Line-Scanning Hyperspectral Cameras 

Line-scanning hyperspectral cameras require multiple exposures to capture both spatial and spectral data from a scene. An input slit limits the incoming photons to a single line; while collimating optics and a dispersive unit spreads the light across the other axis according to its wavelength.  

As a result, a single exposure captures only a 1-D slice of the scene, with one axis of the sensor containing spatial information and the other containing spectral information. Repeating this process as the camera scans across the scene produces a hyperspectral data set with the relative intensity of light measured for different wavelengths at each pixel. 

While this delivers high spectral and spatial resolution, the process is inherently slow, and data cannot be output fast enough for hyperspectral video. Line scanning systems can also introduce imaging artefacts as the device scans across the entire scene, potentially introducing motion blur or missing time-dependent details. 

Snapshot Hyperspectral Cameras 

 Snapshot technology captures all the data simultaneously to enable real-time hyperspectral imaging. Multiple snapshot approaches have found use, but these are often limited by low spatial and spectral resolution. In contrast, the Living Optics camera is a dual-sensor system that uses a coded aperture to select over 4,000 spectral sampling points from the scene. Light from these sampling points is dispersed onto one of the two sensors, and the information contained within the resulting image is ‘decoded’ by the camera software to provide spectra with 96 wavelength channels. Meanwhile, the second sensor captures a high-spatial resolution image of the same scene. Together, the two actively aligned and clock-synchronised sensor outputs provide a spatial-spectral data set that is uniquely suited for video-rate computer vision applications.   

This level of resolution (temporal, spectral, and spatial) is delivered in a robust and portable device to enable real-time hyperspectral imaging use cases regardless of the environment. 

High-Speed Hyperspectral Imaging for Time-Dependent Event Monitoring  

Video rate hyperspectral imaging allows time-dependent event monitoring that is impossible with slow, line-scanned systems. This includes providing data on rapidly changing and dynamic processes or monitoring systems in real-time where swift interventions create significant benefits: 

• Combustion process monitoring for industrial processes and providing real-time data on temperature, chemical reactions, and gas emissions. Monitoring combustion with detailed spectral information offers insights into various potential processes, such as fuel efficiency, combustion efficiency, and pollutant formation. 

• Investigating fluid mixing and providing live analysis of the chemical composition, concentration gradients, and mixing efficiency. Hyperspectral imaging could ensure accurate and consistent fluid mixing in many industries, including beverage production, water treatment, paint mixing, and more. 

• Gait and motion analysis by capturing how a person’s skin and muscles respond to movement. Insights from hyperspectral imaging could be used to improve athlete performance or identify medical issues such as muscle strain or fatigue. 

• Biological monitoring provides a potentially non-invasive method of tracking factors such as blood flow or oxygenation in real- time. Hyperspectral imaging can estimate blood oxygen levels due to changes in biological spectral signatures. 

Live Operator Feedback with Hyperspectral Video  

With video spectral imaging, operators can react in real time based on live feedback. This could include adjusting data capture for enhanced performance or focusing on areas of interest. Live outputs from spectral detection / material quantification algorithms also enable real time process control in manufacturing and quality assurance. For example:

• Security systems could identify and adapt to focus on a potential threat, tracking a specific person or package and reacquiring higher-quality data. 

• An agricultural monitoring system could zoom in on a particular plant or weed to get more specific information even while the camera is in motion. 

• In autonomous fruit picking systems, the live video enables robotic systems to manipulate foliage or fruits whilst analysing sugar content, ripeness or bruising without an operator in the loop.  

• Tracking spectral signatures for crop health monitoring and detecting disease and other stress factors early. With real-time hyperspectral data, farmers can optimise and even automate the delivery of resources such as irrigation, fertiliser, and pesticide. Plus, by immediately identifying disease, they can minimise its spread and financial impact.

It Is Time for Real-Time Hyperspectral Imaging 

Living Optics is on a mission to get video-rate hyperspectral data into the hands of developers. This includes enabling new real-time monitoring use cases and getting hyperspectral cameras out of the lab and into the field through simplifying data capture. 

By processing spectral and RGB video streams, simultaneously, using NVIDIA Edge compute, users can capture real-time data locally and embed the Living Optics camera into existing systems. Paring the high resolution RGB video stream enables computer vision developers to make use of existing vision algorithms / models in addition to a wide range of spectral processing techniques. GPU based edge compute enables live spectral signature analysis with the latest advances in machine learning computer vision techniques. 

With high spatial, spectral, and temporal resolution, we’re confident developers around the world can find new ways to use the Living Optics camera and access real-time hyperspectral data. Get in touch to discuss your use case with an expert in the field and learn more about the future of hyperspectral imaging.