1.

INTRODUCTION

Agriculture is one of the largest and most important industries in Cyprus and worldwide [1]. Crop yield estimation underpins planning strategies to fulfill projected demands of human population under the constraints of food security. Precision agriculture (PA) is a farming method that uses data and modern technologies to maximize crop yield through decision-making applications. A key goal of PA is to improve yield estimation, which has traditionally been estimated with manual sampling or other indirect methods. Remote Sensing (RS) has been found important in PA and yield estimation tasks during the last decades, due to the daily volume of data it generates [4]. However, validating such RS-based models for yield estimation requires further enhancement with in-situ work, e.g., fruit counting. Those field exercises require many manhours. Estimating yield with higher accuracy helps farmers, stakeholders, and relevant governmental bodies for a better planning to satisfy the needs of consumers. Moreover, yield assessments can enhance the market projections and more precise financial management of agricultural market[9].

The use of data-driven application in farming using Artificial Intelligence (AI) has shown promising performance to efficiently validate RS models [3]. For example, data derived from unmanned aerial vehicles (UAVs), robots and cameras installed in fields are combined with Deep Learning (DL) models that are able to perform object detection tasks and, consequently, perform fruit counting [5], [6], [7]. Two primary categories exist in the subject of object detection AI models, i.e., one-stage and two-stage models. Two-stage models are typically based on the architecture of region-based Convolutional Neural Network (R-CNN) and they are utilizing a region proposal network (RPN) such as the Faster R-CNN [10] to extract potential object regions. The extracted regions are then used to classify the objects of an image and execute the bounding box regression to mark the object. These algorithms are executing tasks with higher accuracy due to their capability to execute detection in two stages by enhancing object localization, but they are slow and not usually suitable for real-time applications. More variations of those methodologies have been developed through the years such as Fast R-CNN and Mask R-CNN [11], [12]. On the other hand, one-stage models are more efficient in the aspect of realtime processing as they are executing object classification and bounding box regression in the same neural network. Popular models are the different versions and variations of the You Only Look Once (YOLO) algorithm [13], [14].

According to a study, YOLOv5s achieved 1.84% higher precision on different varieties of grapes’ real-time detection in complex agricultural environments compared to other YOLO variations [15]. A channel pruning is applied on YOLOv5 to develop a lighter architecture by reducing the floating-point operations and then used on real-time tracking of grapes on field images and reached a mean Average Precision (mAP) of 82.3% while also improved the detection performance on overlapping grapes [16]. An improved YOLOv5s using the method Soft Non-Maximum Suppression (Soft-NMS) is used for tomato detection in a working environment as a part of picking robots [17]. The model succeeds 92% and 82% of precision and recall, respectively. Another study shows the ability of YOLOv5 installed on picking robots for real-time apple detection [18]. A YOLOv5s enhanced with a spatial attention module and an adaptive context information fusion module is developed for pineapple buds’ detection on UAV’s in-situ captured images. The proposed methodology increased the mAP@0.5 (mean Average Precision at Intersection of Union equal to 0.5) by 7.4% and mAP@0.5:0.95 (mean Average Precision at Intersection of Union from 0.5 to 0.95) by 31% [19]. An image processing methodology is proposed for the detection of cotton bolls on images captured by a remote-controlled robot, using more traditional methodologies [20]. Later on, an improvement of this work was proposed by the same author, using a variation of Faster R-CNN, known as FrRCNN5-cls, which achieved an R-squared (R² or the coefficient of determination) of 0.88 and Root Mean Square Error (RMSE) of 0.79 [21]. Another model, which uses YOLOv5s as the base model with ShuffleNet-v2 as the backbone model, after fine tuning achieved an improvement of 3.5% at mAP on litchi fruit detection [22]. The YOLOv4 is evaluated on RGB chestnuts’ images captured by a UAV and achieved an R² of 0.98 and RMSE of 6.3 [24]. Moreover, improved fundamental variations of YOLOv5s adapting attention mechanisms and robustness to fog are considered for real-time object detection tasks in uncertain agricultural landscapes [25], [26].In this study one of the light variations of YOLOv5s is adopted on two different benchmark datasets for counting mangoes and wheat heads. Mango fruit is one of the most demanded fruits around different geographical areas in the world such as Asia, Africa, and Latin America. On the other hand, wheat has a high importance for Cyprus with an annual value of ca. 16.4 million euros, covering an area of ca. 12.000 ha and a production of ca. 25.000 metric tons (t) per year [8]. Furthermore, this study aims to evaluate YOLOv5s on crop yield estimation tasks using in-situ captured images as a preliminary work before comparing it with other lightweight variations and examine the different on-board processing aspects.

The rest of the paper is structured as follows. Section 2 describes the methodology followed for the fine tuning of YOLOv5s, the experimental setup and the evaluation strategy used. Moreover, describes the two benchmark datasets used in the experimental study. Section 3 gives an overview of the experimental results concerning the performance and time consumption of the investigated model. Finally, Section 4 concludes this work.

2.

MATERIALS AND METHODS

2.1

Benchmark datasets

In this study. two benchmark datasets are utilized for the preliminary evaluation of YOLOv5s on yield estimation from field images. Both datasets are acquired using the AgML Python library [27], which is an open-access package offering access to fundamental object detection models and benchmark datasets for agricultural tasks. The first dataset consists of 1730 images of mango trees during night-time. The images are captured using a ground-based RGB sensor from mango orchards in Australia [28]. On the other hand, the dataset for wheat heads is also developed from images acquired with a ground-based RGB sensor. The images are captured from plots cultivated with wheat from around the world and are 6512 in total [29]. Both datasets consist of single classes and are suitable for training fundamental models performing object detection tasks.

Figure 1.

Example of the two benchmark datasets. On the left there is a representative example from the mango dataset whereas on the right a representative example from the wheat head dataset. Red boxes on both images indicate the annotation boxes.

2.2

You Only Look Once version 5

YOLOv5, or You Only Look Once version 5, is a fundamental object detection model with real-time capabilities. The architecture (figure 2) of YOLOv5 consists of a backbone network, neck and head (output). The primary function of the backbone, which is usually an altered CSPDarknet53, is to extract detailed, layered features from the input images. The neck, which consists of PANet (Path Aggregation Network), connects the various backbone stages to improve feature fusion and the model’s capacity to detect objects at different scales. Ultimately, the head is made up of multiple convolutional layers that predict confidence scores, object classes, and bounding boxes. YOLOv5 is enhanced in four different aspects. The first advancement in the model is the operation of mosaic-data-augmentation which improves the training speed and network accuracy. YOLOv5, in order to enhance its ability of detecting targets of various sizes, is using the adaptive anchor computation and the adaptive picture scaling techniques. Furthermore, the model adopts fresh concepts in the backbone network and, to enhance the computation and improve the feature representation, adds the focus structure and the cross-stage partial (CSP) structure. Moreover, the Neck network is using a Feature Pyramid Network and a Pyramid Attention Network and adds the CSP2 architecture [30], [31]. Those newly added strategies enhanced the general model’s performance. Still, YOLOv5 faces limitations in detecting tiny targets. YOLOv5s is the second smallest and fastest architecture in the family of YOLOv5 algorithms [32].

Figure 2.

YOLOv5 architecture consists of the input layer, backbone, neck and head (output) components.

2.3

Evaluation metrics

The performance of YOLOv5 was evaluated using three different well known evaluation metrics for object detection tasks. The first metric given in eq.1 is the Precision, which measures the ability of a classification algorithm to identify only the relevant data points. Recall, which is given in eq.2, is the ability of a model to identify all the relevant cases within a data set.

where true positives is the number of model’s correct predictions of the positive class, false positives is the number of model’s incorrect predictions of the positive class and false negatives is the number of model’s incorrect predictions of the negative class.

The mean Average Precision (mAP) given in eq.3 is widely used for object detection models to evaluate their performance based on a range of Intersection of Union (IoU).

where n is the number of classes and AP_k is the average precision of class k. In cases where mAP is evaluated on certain ranges of IoU, it is the average precision of class k at the specific IoU range.

3.

EXPERIMENTAL RESULTS

3.1

Model’s performance

The accuracy metrics are defined in Equations (1) to (3) and are used to evaluate the YOLOv5s. Table 1 shows the precision, recall, mAP@50 and mAP@50:95 achieved on both datasets. The investigated object detection model performed better on the mango dataset with respect to all the evaluation metrics, i.e., Precision = 0.960, Recall = 0.954, mAP@50 = 0.981, and mAP@50:95 = 0.734. On the wheat head dataset the model achieved relatively high accuracy with respect to three of the evaluation metrics, i.e., Precision = 0.910, Recall = 0.824, and mAP@50 = 0.896, but has low accuracy in mAP@50:95. This is because mango colour, shape, and size (especially when they are ripening) creates a contrast with the background of the image and, thus, it is easier for YOLOv5s to detect and generate a bounding box around the fruits. In contrast, wheat colour together with the density of the stems (during both of their states, green and dry) in an image imposes challenges for YOLOv5s to detect the wheat heads. Furthermore the limitations on detecting tiny objects probably influences the performance of the model, despite the large dataset. The model counted a total of 1151 mango instances on the 127 images of the testing set, and 4399 wheat heads instances on the 100 images of the testing set. Figures 3 and 4 present examples of the detected mangoes and wheat heads, respectively.

Figure 3.

Indicative examples of detected mangoes

Figure 4.

Indicative examples of detected wheat heads

Table 1.

Experimental results of object detection tasks using YOLOv5s on mango and wheat head benchmark datasets.

Dataset	Precision	Recall	mAP@50	mAP@50:95
Mango	0.960	0.954	0.981	0.734
Wheat Head	0.910	0.824	0.896	0.519

4.

CONCLUSION

In this work, YOLOv5s, a light variation of YOLOv5, is investigated for its performance on two benchmark datasets for yield estimation using object detection. Based on the model’s performance on the experiments, the following concluding remarks can be drawn. YOLOv5s performed better on the mango dataset, mainly due to the colour, shape and size of the fruits. However, the shape, colour and density of wheat heads generates multiple objects overlaps in the images resulting in lower accuracy for the YOLOv5s. The fine tuning of the model requires approximately 1800 seconds. The inference time of 0.0142 and 0.0116 enables the further examination of on-board yield estimation using ground-based cameras or UAVs. Indeed, the use of AI on-board requires evaluating specific aspects, such as the size of the model and the use of computational resources, which must be limited compared to a ground-based scenario. Therefore, the use of specific onboard hardware will be considered for the future development.