ROSE challenge: performance evaluation of autonomous weeding robots

The aim of the challenge is to encourage the development of innovative technological solutions contributing to the objectives of the Ecophyto II plan: to reduce the use of plant protection products by 50%, to better control risks and to reduce the dependence of agriculture on these products.

The four selected projects focus on intra-row weeding in large-scale field crops and in open-field vegetable crops. For four years, the selected teams confront each other in Irstea's AgroTechnopôle (in the French department of Allier) during the evaluation campaigns conducted by LNE and Irstea.

The challenge concerns not only the evaluation of the individual modules of the intervention chain (plant detection, decision and weeding action) but also the global evaluation of the effectiveness of weeding and the economic, sociological and ecological impacts.

The ROSE challenge is the first in the world to implement an assessment in real conditions on agricultural field. The tools developed in the context of the challenge will therefore constitute references necessary for the characterization of future research in this area: the annotated image databases will be particularly useful to the community and the evaluation methodology will make it possible to define standards for evaluation of the performance of agricultural robotic systems, far beyond the ROSE challenge.

We propose to present the methodology for the evaluation of the plant detection task. After a reminder of the general schedule of the challenge, we will present the metrics used to allow a quantitative, rigorous, comparable, repeatable and accepted performance measurement of the systems.

In a third part, we will present the evaluation tools developed by LNE to proceed the evaluation. More precisely, the DIANNE software developed to the clipping, identification and annotation of images, an essential step for the evaluation.

The assembly will finally be invited to visit the ROSE challenge website to soon access to the tools and evaluation methods applied for the challenge.

This project was presented during the Scientific Seminar organised by Robagri regarding the 2019 FIRA.

Authors

Avrin Guillaume, Barbosa Virginie, Delaborde Agnes, Galibert Olivier, Lalère Béatrice , Lardy-Fontan Sophie, Regnier Rémi: Laboratoire national de métrologie et d’essais

Boffety Daniel: Institut national de recherche en sciences et technologies pour l'environnement et l'agriculture, Irstea