AI arrives in force

“My father is enthu­si­astic about tech­nology, sensors, and arti­fi­cial intel­li­gence (AI),” says Moritz Hentzschel at the renowned Fruit Research Station in Jork on the Lower Elbe in Germany. “It provides reli­able infor­ma­tion from large amounts of image data and may soon be used on his fruit farm in Röding­hausen, North Rhine-West­phalia.” Moritz himself does not intend to take over his parents’ berry and pome fruit (for example, apples and pears) busi­ness, but as a trained horti­cul­turist, he is involved in the Samson research project. This looks at smart automa­tion systems and services for fruit growing in the Lower Elbe region, and sees him working with various AI tech­nolo­gies that, while not yet imple­mented in prac­tice, could soon become rele­vant for fruit growing.

New research on AI in fruit growing

The 26-year-old’s colleague, David Berschauer, is a research asso­ciate at the Fraun­hofer IFAM, which leads the project leader in Stade, and is concur­rently pursuing a master’s degree in computer science at HAW Hamburg. Given the project’s signif­i­cant progress and the delivery of impor­tant new insights, it is likely to continue receiving federal funding beyond the end of the current round in 2025. “We are confi­dent that our work on AI in fruit growing will continue,” says David, looking opti­misti­cally to the future.

He is happy to explain the tech­nology used: At its core, it is a sensor box that is mounted in front of a tractor’s bonnet, which the researchers use to make their rounds in the crop. The box contains a lidar sensor that precisely measures leaf surfaces as well as two RGB cameras, which each capture five images per second with a ‘stereo effect’ and generate up to 400MB of data per second. “This is too much, and is not yet prac­tical in the long term,” David points out. It is clear that the Samson team can use real-time kine­matics (RTK) to precisely locate every move­ment of the cameras on each indi­vidual tree in the stand to within a few millime­tres.

The mobile Samson sensor box is installed at the front of a tractor.

All data recorded by the sensor box can be viewed imme­di­ately via a smart­phone using the app, as demon­strated here by Moritz Hentzschel.

This is also essen­tial because the imaging process, paired with the AI behind it, requires this degree of accu­racy to ulti­mately deliver usable, insightful data. Data which, it should be empha­sised, could also be deter­mined by the expe­ri­enced, trained eye of a fruit grower — if they actu­ally had time for each indi­vidual tree. This includes the optimal harvest time and a precise fore­cast of the expected harvest quan­tity and qual­i­ties. The latter is crucial infor­ma­tion for timely marketing of the fruit.

So how does the AI work?

“We feed the acquired image data into our soft­ware, which processes it in a neural system with several million para­me­ters,” explains David. “We contin­u­ously train the AI models so that the results deter­mined by the soft­ware are ulti­mately reli­able and rele­vant for the fruit grower.” Since this data is becoming increas­ingly impor­tant in a compet­i­tive envi­ron­ment, AI is now being intro­duced at a rapid pace and with full power to agri­cul­tural oper­a­tions following remark­able advance­ments in automa­tion, GPS, and robotics. The oppor­tu­ni­ties are substan­tial, and the enthu­siasm is some­times even greater. However, there is also signif­i­cant scep­ti­cism towards a devel­op­ment that undoubt­edly possesses revo­lu­tionary poten­tial and will perma­nently trans­form work on farms.

But back to Samson and fruit growing: Data collec­tion via sensors is one side of the coin. Processing the data mean­ing­fully is the other, equally impor­tant aspect. This requires training. Training is currently making rapid progress so that indi­vidual crop manage­ment for each tree will even­tu­ally be possible. What humans cannot achieve despite their greatest efforts — simply due to a lack of time — is accom­plished by the computing power of hard­ware. Ulti­mately, with the help of AI, the spray nozzle opens for wide-ranging crop protec­tion, or frost protec­tion is only applied when it is truly neces­sary. This saves resources and protects the envi­ron­ment. One example is frost protec­tion: Currently, when night frosts are expected, many apple orchards are irri­gated in order to avert poten­tial damage. Often, even when temper­a­tures below zero degrees Celsius are fore­cast, they do not actu­ally reach the dangerous frost level. This incurs signif­i­cant costs (40,000l/ha are used per hour of frost protec­tion), which AI, fed with exten­sive weather data and addi­tional infor­ma­tion on the crops, reduces to the bare minimum.

Nonethe­less, AI also comes at a price. It incurs costs in terms of devel­op­ment, instal­la­tion, and energy consump­tion. It must there­fore be cost-effec­tive if it is to be imple­mented, although young researchers like David and his colleague Fred­erick Blome claim that yield increases of 20-30% can be expected. Providing the fruit grower remains in control and the AI appli­ca­tion is easy to use, David has no concerns: “The risks are clearly calcu­lable”.

Weed detec­tion made easy

Samson in fruit growing is like Sam-Dimen­sion (smart aerial mapping) in arable farming: At 8am in sunlit June, a John Deere tractor pulls a field sprayer through an onion field in Armstedt, Schleswig-Holstein. Tjark Hart­mann-Paulsen from the Hof Hasenkrug farm grows red and white onions on a 15ha plot for the Edeka retail chain. There is nothing partic­u­larly remark­able about the scene itself. But that is decep­tive. This is because an AI-based system from the Stuttgart-based start-up Sam-Dimen­sion GmbH is being used for crop protec­tion by contractor Jan Marten Scheel from Sarl­husen.

Farmer Tjark Hart­mann-Paulsen and drone pilot Jannik Robrahn from the Scheel Sarl­husen contracting company with a SAM Dimen­sion drone.

SAM Dimen­sion provides an appli­ca­tion map based on AI-eval­u­ated drone images, which is used for selec­tive weed control with the tractor GPS and a field sprayer.

The SAM Dimen­sion map with the local­ized weed loca­tions is imported to the John Deere Oper­a­tions Center via a USB stick.

The entire onion field has already been surveyed by a drone, which captured the whole site in detail with a camera. The drone oper­ates at an alti­tude of approx­i­mately 60m, capturing six images per second. It requires about 15 minutes to cover the 15ha. The gener­ated image data package has a volume of several hundred giga­bytes, which is pre-processed in the field and then sent to the Sam central computer. On this computer, AI-based soft­ware processes the flood of images and iden­ti­fies weeds and unwanted grasses at incred­ible speed. Lastly, the AI-based soft­ware gener­ates a field map with a small data volume, which is sent back to the contractor and then imported to the tractor or sprayer.

This field map shows exactly where the weeds are. In conjunc­tion with the John Deere GPS system and a sprayer with PWM, the Sam appli­ca­tion map enables pinpoint, highly accu­rate, and targeted spraying. This saves chem­i­cals. “Where I previ­ously applied the same spray rate across the entire field, with this system it’s only about a fifth, simply because there is no weed pres­sure in many places and the tech­nology informs me of this,” says Tjark.

Contractor Jan is the first in Northern Germany to utilise drones and AI-based data collec­tion. This is all still quite new to the 26-year-old, but he is convinced by Sam and AI systems. He believes it is the future. “I have invested a substan­tial five-figure sum in this,” says the inno­v­a­tive contractor. “That includes the drone as a carrier plat­form for the Sam-Cam AI mapping camera, as well as the complete soft­ware to use the Sam Dimen­sion system and the data processing for map produc­tion.” He charges farmers for his addi­tional service by hectare and crop type — for onions, the cost is €35. Smiling, he adds: “It’s a win-win for everyone.”

Contractor Jan Marten Scheel uses the AI-based SAM Dimen­sion System for weed control in an onion field.

Chamomile in an onion field: This weed is detected by the drone and added to the appli­ca­tion map for selec­tive treat­ment.

And what does the future hold for AI in agri­cul­ture?

It is not yet clear how and where AI will ulti­mately take root. Will it make work on farms easier and better or rather more compli­cated? Lea Fliess, managing director at Forum Land­wirtschaft (FML), an asso­ci­a­tion of over 60 members from the agri­cul­tural sector, has a clearly posi­tive posi­tion on this: “Whether AI is a curse or a blessing for the agri­cul­tural sector depends on how we use it. If it is devel­oped together with farmers — and in dialogue with society — its full poten­tial can be realised,” he says. “Trust can only grow if people under­stand how AI oper­ates in the field and the contri­bu­tions it makes to the envi­ron­ment, animal welfare, and nutri­tion. When used correctly, it saves time, reduces oper­ating resources, and opens up new oppor­tu­ni­ties for sustain­ability. Achieving this will require active plan­ning: Rooted in prac­tice, respon­si­bility and polit­ical backing.”