Counting Grapes with Hyperspectral Data for Crop Yield Estimates

With an accurate estimate of how much is growing in their fields, farmers can: 

• Adjust how they apply their resources 
• Better manage their delivery estimates, logistics, and cash flow
• Improve their profitability

In viticulture, proper yield estimates help enhance berry quality and plant growth. Farmers have a better understanding of how many grapes are growing across the vineyard, enabling them to prune, irrigate, and schedule nutrients based on actual data instead of just eyeballing it. 

How Current Methods for Estimating Crop Yields Fall Short 

Many traditional methods of estimating crop yields fall short and provide little more than a ballpark figure for farmers to go by. They are often based solely on the farmer’s experience, or they require time-consuming manual sampling across the vineyard to still only produce limited results. 

An improved method utilises remote sensing with standard RGB or multispectral cameras. This approach mounts the camera on a aircraft, UAV, or ground vehicle to capture images of the crops in order to apply more in-depth analysis techniques and return a closer estimate of the overall crop yield. 

This includes adopting sophisticated computer vision algorithms for analysing images obtained from UAVs. However, this again has limitations due to leaf occlusion, which blocks the computer vision system from reliably seeing the volume of crops. This can be overcome with ground-based instruments that image from the side of the vine rows rather than top-down.  

Counting Grapes with In-field Hyperspectral Imaging 

On top of that, upgrading from RGB or multispectral imaging to hyperspectral cameras can both increase the accuracy of computer vision grape counting systems while also simplifying the task and reducing the level compute needed. 

To demonstrate the feasibility and value of in-field grape counting for crop yields, Living Optics partnered with JoJo’s Vineyard located in Russels Water, Oxfordshire. A series of measurements were taken on the site between May and October 2024, shortly before harvest. 

An adapted version of the Living Optics Development Kit was mounted on a backpack to capture images: 

A GPS receiver was connected to the edge device to track the position of measurements throughout the vineyard. A white reference target (Tyvek) was mounted on an extendable rod to be in the frame of the image to convert the camera’s output from spectral radiance to spectral reflectance.  

This data was taken by a two-person team, with one operating the Living Optics camera while the other monitored exposure conditions (lighting, etc.) and tracked position on the GPS. 

Using Hyperspectral Data to Quantify Grape Bunches 

The Living Optics hyperspectral datasets were processed by extracting a series of reference grape and background spectra. Then, a random forest classifier algorithm was applied to autonomously detect grapes.  

To create a spectral detection mask, the random forest machine learning algorithm distinguished between all spectra captured in the image as either being grape or background. 

Next, each frame was segmented using FastSAM from Ultralytics. The output produced was a set of image segments. Performing a simple logical “AND” between the RGB segmentation mask and the spectral detection mask, we were able to classify the image segments corresponding to grapes.

Moving Forward With Hyperspectral-Based Crop Yield Estimates 

With information-rich hyperspectral frames, only a small dataset was required (13 labelled images for a total of 60 labelled grape bunches as training data) to train the random forest classifier to an accuracy of 94.78%. 

Relying on higher spectral resolution datasets makes identifying grapes easier while improving accuracy compared to previous techniques. The vineyard owner stated that the grape counting capabilities of the Living Optics Development Kit could provide commercial benefits to his operations, helping improve yield estimates and plant management. 

Future work will focus on improving hyperspectral analysis for field measurements. This includes accounting for non-planar, non-uniform illumination when normalising data using the white reference. The translucency of plant matter causes superposition of light reflected or backscattered from its surface or transmitted through. 

Despite the challenges, we believe as long as there is a basic level of consistency in illumination and imaging geometry, analysis algorithms can be developed to account for these effects. 

In the near future, farmers can image grape bunches using a hyperspectral camera and feed the data into a computer vision algorithm to quickly return an accurate crop yield estimate. 

To learn more contact us today.