Although Luna relies on a broad range of sensors delivering real time feedback for direct control, we are planning to enhance the capabilities and real world Application. There are two distinct ways to improve those Aspect through Machine Learning (ML), on one side we use auxiliary ML models to increase the reliability of our system as a whole through using additional sensors to improve main sensors readings. The other application covers a broader range of application through learning Agent Policies that can control luna on their own and follow given commands such as moving in a certain direction or rotating on the spot. Through the implementation of digital twin we were able to test our software stack and hardware plans before starting the costly process of creating hardware prototypes, therefore increasing the amount of time we needed to get our first working system into a prototype state.

At this year’s Field Robot Event at the University of Maribor in Slovenia, our robot impressively outperformed 14 other competitors in the computer vision-focused Task 3. The task determined the accuracy of the competing robots‘ recognition systems on a series of images of people, deer, and other objects. We also took 2nd place in the Freestyle category and 4th place in the Smart Irrigation task. We would like to thank our sponsors who supported our participation in the competition with financial resources and hardware. For a deep dive into our methodology, we will soon publish our FRE 2023 proceedings on this blog.

Last year, we published our dataset and models for image recognition during the online Field Robot Event 2021. Our blog post ended with a call for cooperation in creating a dataset and AI models for this year’s event, which has since taken place at DLG Feldtage in June. Team Carbonite from the Überlingen Students’ Research Center (SFZ Überlingen) approached and we subsequently worked together on building a realistic image dataset. As a nod to the two teamnames, this dataset shall henceforth be known by the name “Carbonaro”.

From June 13th to 16th we partook in the annual Field Robot Event, the international field robotics showdown. This year the contest was split in a real world field contest and a virtual field contest which again took place in our virtual maize field simulation environment.

Up until now we used the Nvidia Jetson Nano for machine learning acceleration purposes. Sadly, running our object detection models wasn’t viable on the Jetson Nano despite using its CUDA-capable GPU. Our Jetson Nano only managed a poor 5 spf (seconds per frame) and had both its RAM and VRAM running out of space constantly. In order to remedy this, we looked for stronger hardware.

When we upgraded our robot Beteigeuze with entirely new electronic components, we decided on nVidia’s Jetson Nano platform as the robot’s primary computer. The Jetson Nano is an ARM based quad-core System-on-a-Chip (SOC) that features CUDA acceleration for Deep Learning models. As a test, we tried deploying the model that we used during last year’s Virtual Field Robot Event on the new hardware platform. Even though that model was trained for simulation, we hope that models of the same or similar architecture will be of value for us in this year’s event. The model is a ResNet50 from PyTorch’s torchvision.

Following the release of our object detection deep learning code, model and dataset, we are now publishing the code we used for navigating within a crop row at the Field Robot Event. This code has now been in use at Kamaro for several years, and most recently won Task 1 at the competition in 2021. A detailed description of the algorithm is available in the Readme file on Github and in the Proceedings of FRE 2021.

FRE 2021 is over! It’s been a great week seeing all of the other teams again and competing with one another. We are grateful for the opportunity we had in building the competition environment and shaping this year’s Field Robot Event! We were especially pleased with the organizers’ decision to include realistic 3D plant models for the weed detection task. For a long time, simple color-based detectors were sufficient to solve the detection tasks at the Field Robot Event. However, such approaches are not adequate to perform a plant recognition task in a realistic scenario. On the real field, robots will have to distinguish between multiple species of plants — all of which are green — in order to perform weeding, phenotyping and other tasks. Deep learning has been around for a few years now and seems to be the most applicable tool to tackle this challenge.