The History of Hyperspectral Imaging

Hyperspectral imaging has evolved tremendously over the past fifty years, now finding itself indispensable in various industries. The technology has found use in: 

1. Health: instrumental in cancer surgery to distinguish tumour margins by assessing blood flow differences between tumorous and healthy tissue. 

2. Recycling: plays a critical role in identifying and sorting recyclable materials, distinguishing between black and clear plastics and or textiles which are visually identical. 

3. Art, Archaeology, and Conservation: excels in revealing concealed or faded text in historical manuscripts, ensuring that even the slightest irregularities are detected. 

4. Pharmaceuticals: aids in identifying counterfeit drugs by analysing and quantifying their active ingredients. 

5. Agriculture: used for evaluating plant health and determining the maturity or ripeness of crops. 

6. Environment: pivotal in identifying and measuring greenhouse gases and their movement, contributing to environmental monitoring efforts. 

7. Defence: serves a crucial role in differentiating friend from foe, enhancing military capabilities and security measures. 

Despite these diverse applications, we are just beginning to uncover the full potential of hyperspectral imaging. Advances in optical and semiconductor design are paving the way for more robust, portable, and cost-effective systems. These enhancements, coupled with improved data processing capabilities, are bringing us closer to obtaining hyperspectral data in almost real-time. 

By providing both spatial and spectral information, HSI uncovers layers of information unseen to the naked eye. Traditional cameras capture scenes in a 2D format, assigning RGB (red, green, and blue) values to each pixel. Hyperspectral cameras deliver a far richer dataset, capturing up to hundreds of spectral bands for each pixel. These systems often extend beyond the visible spectrum, detecting infrared or ultraviolet light, which is invisible to the human eye. The result is a 3-dimensional “hypercube,” a data-rich representation offering detailed insights into each pixel’s spectral properties. 

This wealth of information is invaluable for identifying materials, differentiating visually identical objects, monitoring chemical composition changes, and accurately tracking objects. Hyperspectral imaging, with its extensive range of applications, has evolved into an industry worth billions of dollars. Of course, this transformation from a nascent field to a multi-billion-dollar sector did not happen overnight. What were the origins of this remarkable journey? 

Through the Prism

The Colourful History of Hyperspectral Imaging 

In the captivating story of hyperspectral imaging, familiar figures such as Maxwell and Newton make cameo appearances – setting the stage. We then explore the intense competition of satellite imaging during the Cold War. This story is spiced with accidental discoveries, illustrating how serendipity, alongside strategic genius, shaped this technology. 

The Beginnings of Spectral Imaging

1. James Clerk Maxwell (1861)
   The journey into colour imaging began with James Clerk Maxwell, a physicist who, in 1861, demonstrated the first colour photograph. Maxwell ingeniously used three separate filters – red, green, and blue – to capture three different black-and-white images. By projecting these images together and superimposing them through corresponding-coloured filters, he created the first colour photograph. This method, known as the three-color method, became the foundation of nearly all subsequent colour photography and imaging techniques. 

2. Julius von Sachs (Late 1800s)
   Transitioning from aesthetic to scientific applications, Julius von Sachs, a prominent botanist, utilised different filters in front of a film camera to study plant physiology. This approach represents one of the earliest instances of spectral imaging being used for scientific purposes. By analysing how plants interact with different light spectra, Sachs contributed significantly to our understanding of plant biology and photosynthesis. 

3. Edward Emerson Barnard (Late 1800s)
   Around the same time, astronomer Edward Emerson Barnard applied a similar technique to astronomical imaging. Using various filters with a telescope and film camera, Barnard captured enhanced images of celestial phenomena such as nebulae and the Milky Way. His work significantly improved our understanding of the structure and composition of these astronomical bodies. 

The Evolution of Spectroscopy

1. Isaac Newton and his Prism (1600s)
   The roots of spectroscopy can be traced back to Isaac Newton’s experiments in the 17th century. Newton, (building upon the work of less famous figures: Kircher, Marci, Boyle, and Grimaldi), documented how a prism could split white light into a spectrum of colours. This phenomenon, famously depicted on the cover of Pink Floyd’s album “The Dark Side of the Moon,” laid the conceptual foundation for spectroscopy. 

2. William Hyde Wollaston (Early 1800s)
   Nearly a century after Newton, Wollaston showed by passing sunlight through a prism – distinct dark lines are formed. These lines, later known as absorption lines, indicated the presence of specific elements within the light source. This discovery was pivotal in understanding the interaction of light with matter and laid the groundwork for the use of spectroscopy in chemical analysis and astronomical studies. 

When Imaging Met Spectroscopy 

As the realms of imaging and spectroscopy started to intertwine, the era of the Cold War provided a dramatic backdrop, catalysing advancements that were as much about geopolitical rivalry as they were about scientific exploration.

1. Angelo Secchi’s Starry Insights (1860s)
   The journey begins with Angelo Secchi, who, in the 1860s, revolutionised astronomical observation by using objective prisms to image multiple stars simultaneously. This ingenious approach led to the classification of stars into spectral types, a foundational concept in astronomy.

2. Vesto Slipher’s Planetary Probes (Early 1900s)
   Early in the 20th century, Vesto Slipher’s spectral scanning of planets provided insights into their motion and composition, pushing the boundaries of our cosmic understanding, and hinting at the universe’s expansion.

3. The Zeiss MKF-6 Era (1969)
   Fast forward to the space race era, and we see the Zeiss MKF-6 camera emerging as a technological marvel. Launched in 1969, amid the height of Cold War tensions, this system of six synchronised film cameras captured the Earth’s surface in different spectral bands.

Each unit of the camera was priced at 82 million East German Marks ($160.6 million USD in 2023 money) and weighed 175 kilograms. Its construction necessitated the invention of specialised metalworking tools.

4. Virginia Norwood: the mother of the Landsat MSS (1972)
   The Americans weren’t far behind the Soviets. Importantly, Virginia Norwood – then at the Hughes Aircraft company was willing to make a bet: the brand-new multispectral scanner was to be digital. Not analogue. Not film.
   
   Norwood was adamant that the data stream should be digital. NASA had serious reservations, doubting that the six-bit (today’s consumers cameras are 16 bit) MSS data could produce high-quality images. But she knew that a continuous analogue signal would be difficult to process accurately. Going digital would make it possible to calibrate the photon levels from each sensor very precisely. “And you do want it to be accurate,” she says: otherwise “you get a striped mess” when the data is reconstructed into images.
   

Going digital for Norwood’s multispectral scanner presented formidable challenges. The scanner was restricted to just 24 pixels, divided among four spectral bands, which was insufficient for capturing detailed images from space. The solution required a dual scanning process. While one scanning direction was straightforward, achieved by the satellite waiting for Earth to rotate beneath it, the other direction posed a significant challenge.

Norwood realised that a motorised scanner could not endure years of continuous back-and-forth movement. Her team innovatively designed a weightless, endlessly pivoting mirror that could take advantage of the low-gravity conditions of space. This mirror, pivoting more than 13 times per second without air resistance, would bounce between two carefully placed bumpers. Each time the pivoting mirror swept – a large swath of land would be imaged, and when the mirror arrived at its terminus, precisely calibrated light would be shone onto the 24 pixels.

NASA officials would later tell Norwood, that the MSS data would be the first data transmitted digitally from space. And it would set the standard for future quantitative remote sensing.

More bands needed

Hyperspectral cameras go on Tour

The achievements of Norwood’s MSS and the MKF-6 led to an overt question: What could be achieved with more than just 4 or 6 spectral bands? This curiosity began on Earth and eventually found its way back to space.

Despite the MSS’s success, geologists analysing its four-band data noticed a significant limitation: key mineralogical indicators were missing from these bands. To address this, Alexander Goetz was tasked by NASA to conduct geological mapping of Arizona’s Coconino Plateau. Goetz’s approach involved performing detailed spectral measurements on surface soil samples to enhance the interpretation of MSS images. In 1974, his team developed the Portable Field Reflectance Spectrometer (PFRS), designed to span the full spectrum of solar-reflected radiation.

Their research uncovered that spectral reflectance was capable of identifying the mineral composition of soils. For instance, they could detect the spectral signature of rocks indicating the presence of oxygen-hydrogen bonds. However, Goetz realized that for accurate mineral identification, a spectrometer needed a resolution finer than 10nm. The spectral bands of the MKF-6 and MSS, with their 100-200nm width, were simply too broad to effectively isolate individual minerals.

Goetz’s results were critical in the development of what is considered the first modern hyperspectral imager for remote sensing, the Airborne Imaging Spectrometer (AIS).

The optical layout of Goetz’s AIS. The grating – the element which separated wavelengths of light to different parts of the sensor could be rotated to direct more bands of light onto the detector.

The Airborne Imaging Spectrometer (AIS) and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)

In the early 1980s, Goetz and his team at NASA’s Jet Propulsion Laboratory (JPL) saw the development of the Airborne Imaging Spectrometer (AIS). The AIS took advantage of the then brand-new infrared array detector – a sensor with 32 × 32 resolution. Not much resolution, but a significant improvement over Norwood’s MSS of 24 individual detectors. This allowed for the AIS to capture 32 channels for 32 locations simultaneously. However, NASA wanted more than 32 channels. The team designed a contraption which finely rotated an optical element between four fixed positions, such that 4 sets of 32 channels were mapped on the infrared detector.

The enhanced resolution and sensitivity to infrared wavelengths of the AIS soon demonstrated its value. In a practical application, the AIS was bundled onto a cargo plane and flown over copper mines in Nevada. The mission was a success: the NASA team was able to differentiate between economically valuable copper minerals like as chalcopyrite or cuprite, and less valuable minerals clay and gypsum minerals.

The results released by Goetz were a turning point, solidifying the belief in the significant value of hyperspectral imaging. This value extended beyond scientific research to include commercial applications, opening new doors in the field of remote sensing.

Commercial Hyperspectral Imagers

In the closing decades of the 20th century, a quiet revolution took place in the realm of remote sensing, marking the transition of hyperspectral imaging from specialized research labs to the broader commercial market. This shift began in the late 1980s, a period that witnessed the technology’s first forays into practical applications.

It was Itres Research Limited of Calgary, Canada, that spearheaded this movement with the development of the Compact Airborne Spectrographic Imager (CASI). CASI may not have boasted high-resolution capabilities by today’s standards, but it was a trailblazer in its time, enabling detailed environmental, agricultural, and forestry analyses through a new lens that extended beyond the visible spectrum.

The momentum continued into the 1990s with HyVista Corporation’s HyMap, an Australian innovation that offered enhanced spectral resolution. This advancement was a boon to fields like mineral exploration and environmental science, offering a more nuanced view of the Earth’s surface and its myriad features.

Hyperspectral Imaging for Everyday Use

The expanding reach of hyperspectral technology globally is not just a story of scientific advancement; it’s a revelation about the world around us. This technology does more than just map the Earth’s landscapes and ecosystems. It offers a new way to perceive the familiar objects we encounter daily.

Picture using a hyperspectral camera while grocery shopping to pick the ripest fruit, or in your kitchen, where it reveals the freshness of herbs and the quality of your morning coffee beans. Your favourite shirt or dress in the closet isn’t just a fashion statement anymore; through this technology, it’s a story of fabric quality and sustainability. Art lovers can peer into the layers of a painting, uncovering the artist’s process or the authenticity of an antique.

In the garden, hyperspectral imaging helps in diagnosing plant health, guiding you to nurture your plants with precision. Even in recycling, this technology plays a role, helping sort materials more accurately. In the future hyperspectral imaging will provide timely insights for personal healthcare, diagnosing skin conditions or monitoring the healing of wounds.

Through this lens, the world becomes a rich mosaic of information. Everyday objects reveal their stories in wavelengths and frequencies, telling us about their composition, condition, and history. This isn’t just about gathering scientific data; it’s about deepening our understanding of the objects that fill our lives. Hyperspectral imaging invites us to see the world not just in colour, but in a spectrum of information, transforming the ordinary into something extraordinary.

Living Optics

At Living Optics, we are dedicated to a future where hyperspectral imaging is not just a tool for the few but an asset for the many. Currently, these powerful systems are often expensive, complex, and limited to industrial use. We’re changing that. We’re developing hyperspectral imagers that are not only more affordable but also easier to use, breaking down the technical barriers that have kept this technology out of reach for many.

Our vision is to seamlessly integrate hyperspectral imaging into the fabric of everyday life, a commitment that is reflected in our name, Living Optics – because we want to be an integral part of how people see and interact with the world around them.