Strawberry Ripeness Assessment Via Camouflage-Based Data Augmentation for Automated Strawberry Picking Robot

Abstract

Vision-based strawberry picking and placing is one of the main objectives for strawberry harvesting robots to complete visual servoing procedures accurately. Occlusion is the main challenge in strawberry ripeness detection for agriculture robots. In this study, strawberry ripeness detection was proposed using a camouflage-based data augmentation strategy to simulate the natural environment of strawberry harvesting conditions. Yolov4, Yolov4 tiny and Yolov4 scaled, and their traditional data augmentation and camouflage-based data augmentation derivatives were used to find out the effect of camouflage-based augmentation technique in overcoming the occlusion issue. Then the results were mainly evaluated based on mean Intersection over Union (IoU), F-1 score, average precision (AP) for ripe and unripe strawberries and frame per second (fps). Yolov4 tiny with camouflage-based data augmentation technique has demonstrated superior performance in detecting ripe and unripe strawberries with 84% IoU accuracy ~99% AP for ripe and unripe strawberries at an average of 206-fps, satisfying the agriculture strawberry harvesting robot operation need. The performance of the suggested technique was then tested successfully using a dataset termed the challenge dataset in this study to demonstrate its performance in a complex and occluded strawberry harvesting environment. Camouflage-based data augmentation technique helps to increase the detection procedure of ripe and unripe strawberries toward autonomous strawberry harvesting robot.