Computer vision (CV) applications, using AI algorithms to understand and analyse visual information, have seen a surge in use in recent years. However, most of these CV systems are limited to analysing data from traditional sensors and, as such, fail to take advantage of most of the information on offer.
Sensing more than meets the eye – remote spectral imagery
Traditional imaging recreates how the human eye views the world, returning only three channels (red, green, and blue) of data to create a colour image. In contrast, spectral imaging systems look at the world in much more detail, sampling image data across a large number of channels (wavelength bands) to reveal spectral information beyond just colour.
Objects interact with light in different ways, absorbing, transmitting, and reflecting light based on their chemical structure. Traditional imaging fails to capture this information by grouping reflected light into only three broad channels.
With 10s or 100s of channels, spectral imaging techniques, such as multispectral and hyperspectral image analysis, allow us to observe spectral signatures in the data, learning new information about objects in the image.
These techniques offer passive access to new insights from a scene. Adding new layers of information, or a hyperspectral intelligence, enables CV algorithms to detect and identify different materials, segment an image based on more detailed criteria (beyond colour and shape), and track objects, even when they are being deliberately hidden.
Spectral imaging has the potential to revolutionise CV use cases, whether it is short-range systems (healthcare, food quality assessments, etc.) or sensing at longer range.
Sensing at range with spectral imagery
Spectral signatures are retained regardless of the distance between the object and the sensor. That means, as long as the device’s optics can spread reflected light from an object across a few pixels, a hyperspectral imaging camera can identify the signatures it contains.
This enables remote spectral imaging systems to detect small and or otherwise indistinguishable signatures at long distances, even from space. Remote spectral imaging applications are extremely numerous, and many have yet to be fully developed. They include:
- Environmental monitoring – tracking a wide range of ecosystems for conservation projects and optimising land use.
- Mineral exploration – targeting mineral ores on a large scale to save time and money as well as help investigate difficult to reach locations.
- Agriculture – enhancing farming techniques using hyperspectral image analysis to determine crop health, soil conditions, and more.
- Archaeology – using airborne and satellite hyperspectral imaging technology to identify and map archaeological sites and reveal hidden features.
However, one of the most interesting fields of research and development for remote spectral imaging systems is for national security applications.
The importance and challenges of spectral imaging for national security applications
The extra layer of data, or perception, provided by spectral imaging systems could be invaluable for national security applications, potentially enabling armed forces and intelligence personnel to better understand their environment and the possible threats it may hold.
Using computer vision spectral imagery for national security applications faces many of the same challenges seen in other fields, including:
- Developing robust devices capable of returning accurate data regardless of the conditions.
- Delivering systems that non-experts can operate in the field.
- Providing sufficiently powerful edge compute capabilities to handle and interpret the considerable amount of data produced.
However, unlike in other fields, national security personnel taking data in the field must remain hidden, and the spectral signatures they are trying to observe are likely to be actively hidden from them. Additionally, the consequences of being spotted or returning inaccurate data can be much more severe; collection assets are frequently priority targets for adversaries. Therefore, passive spectral imaging systems offer advantages over other sensing solutions. Active sensors illuminate the scene using their own light source in order to take an image. In contrast, passive sensors rely on natural illumination, such as sunlight. As active sensors emit light to collect data, they potentially reveal the location of the spectral imaging system.
Furthermore, hyperspectral imaging technology offers advantages over multispectral imaging within the same wavelength range. Multispectral sensors are optimised for detecting signatures within particular wavelength ranges. Hyperspectral cameras however, collect information from across a wider range of wavelengths and, as a result, are more resilient to the emergence of new signatures of interest. A hyperspectral approach therefore reduces the risk of eventual hardware redundancy as a consequence of changes in the sensing challenge experienced by users.
Additionally, spectral sensors derisk the adoption of autonomous systems in the field, potentially allowing human operators to position themselves further away from potential danger. Making this a reality requires extreme confidence in autonomous systems and their ability to correctly understand and respond to their environment. Hyperspectral data enables computer vision solutions to reach exceptionally low false positive and false negative results by comparison to the results that can be achieved using traditional imagery.
The speed of interpretation required, and the physical realities of tactical environments mean that traditional hyperspectral imaging technology is not readily transferable to many national security missions. Spectral imaging systems on the frontline must be able to dynamically respond to changes in circumstances whilst providing continuous real-time data capture and analysis. For example, credibly fielding hyperspectral solutions in tactical environments requires such systems to be capable of tracking objects in a scene when both the object and the system exhibit irregular movements.
Working with the national security community to understand the potential of spectral imaging
Given the importance and challenges of developing technology for national security applications, Living Optics has sought out opportunities to collaborate with the community from an early stage.
This has included working with the United Kingdom’s Defence Science and Technology Laboratory (DSTL) and The British Army, trialing Living Optics’ technology to help all parties learn together.
Trials have included testing in rural, urban, and coastal environments using the Living Optics hyperspectral imaging camera to track moving objects outside controlled laboratory conditions. Proving the viability of this technology, opens the door to a range of benefits, including the detection of signatures not previously seen, greater confidence in identifying signatures of interest, and the potential to deploy trustworthy autonomous systems that reduce the exposure of human users and increase the precision and reliability of decision making.
Video-rate tactical hyperspectral image analysis has only recently emerged as a viable sensing approach to overcome the challenges of national security use cases. Therefore, the field is ready for exploration with significant scope for early adopters to reap the potential benefits of hyperspectral imaging technology.
Get in touch today to learn more
