Simultaneous fruit detection and size estimation using multitask deep neural networks
View/Open
Author
Ferrer-Ferrer, Mar
Ruiz-Hidalgo, Javier
Gregorio, Eduard
Vilaplana, Verónica
Morros, Josep-Ramon
Publication date
2023-08-06ISSN
1537-5110
Abstract
The measurement of fruit size is of great interest to estimate the yield and predict the harvest resources in advance. This work proposes a novel technique for in-field apple detection and measurement based on Deep Neural Networks. The proposed framework was trained with RGB-D data and consists of an end-to-end multitask Deep Neural Network architecture specifically designed to perform the following tasks: 1) detection and segmentation of each fruit from its surroundings; 2) estimation of the diameter of each detected fruit. The methodology was tested with a total of 15,335 annotated apples at different growth stages, with diameters varying from 27 mm to 95 mm. Fruit detection results reported an F1-score for apple detection of 0.88 and a mean absolute error of diameter estimation of 5.64 mm. These are state-of-the-art results with the additional advantages of: a) using an end-to-end multitask trainable network; b) an efficient and fast inference speed; and c) being based on RGB-D data which can be acquired with affordable depth cameras. On the contrary, the main disadvantage is the need of annotating a large amount of data with fruit masks and diameter ground truth to train the model. Finally, a fruit visibility analysis showed an improvement in the prediction when limiting the measurement to apples above 65% of visibility (mean absolute error of 5.09 mm). This suggests that future works should develop a method for automatically identifying the most visible apples and discard the prediction of highly occluded fruits.
Document Type
Article
Document version
Published version
Language
English
Subject (CDU)
631 - Agriculture in general
Pages
13
Publisher
Elsevier
Is part of
Biosystems Engineering
Citation
Ferrer-Ferrer, Mar, Javier Ruiz-Hidalgo, Eduard Gregorio, Verónica Vilaplana, Josep-Ramon Morros, and Jordi Gené-Mola. 2023. "Simultaneous Fruit Detection And Size Estimation Using Multitask Deep Neural Networks". Biosystems Engineering 233: 63-75. doi:10.1016/j.biosystemseng.2023.07.010.
Grant agreement number
MICIU/Programa Estatal de I+D+I orientada a los retos de la sociedad/RTI2018-094222-B-100/ES/Tecnologías de agricultura de precisión para optimizar el manejo de dosel foliar y la protección fitosanitaria sostenible en plantaciones de frutales/PAgFRUIT
MICINN/Programa Estatal para impulsar la investigación científico-técnica y su transferencia/PID2021-126648OB-100/ES/Protección de cultivos de precisión para conseguir objetivos del Pacto Verde Europeo en uso eficiente y reducción de fitosanitarios mediate Agricultura de Precisión/PAgPROTECT
MICINN/Programa Estatal de generación del conocimiento y fortalecimiento científico y tecnológico del sistema I+D+I y Programa Estatal de I+D+I orientada a los retos de la sociedad/PID2020-117142GB-I00/ES/ /DeeLight
FEDER/ / /EU/ /
Program
Ús Eficient de l'Aigua en Agricultura
This item appears in the following Collection(s)
- ARTICLES CIENTÍFICS [2631]
The following license files are associated with this item:
Except where otherwise noted, this item's license is described as http://creativecommons.org/licenses/by/4.0/