Improved pizza picking and packing: hackathon at igus uses robots and AI

A hackathon hosted by igus® called “Artificial Intelligence meets Robotics” has developed advanced but cost-effective automation solutions for logistics, involving 17 people divided into three interdisciplinary teams. The aim of the ‘hackathon’ was to create an automated method for packing pizzas using a Large AI-based Language Model (LLM). The Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, the University of Bonn, the Lamarr Institute and the Fraunhofer Institute for Material Flow and Logistics IML took part with igus® in Cologne, Germany.

A supermarket, situated very close to a large frozen pizza manufacturer, needs a constant supply of pizzas to stock their shelves, currently manual work by an understaffed team. 17 participants from two Fraunhofer Institutes, the Lamarr Institute for Machine Learning and Artificial Intelligence and employees from the University of Bonn developed a solution for this during a five-day ‘hackathon’ at igus®.

By combining generative AI and robotics, the aim was to develop a solution that would make this repetitive work more efficient and cost-effective.

Automating industry with AI and low-cost robotics

During the hackathon, three teams worked on this real-world situation where an igus® ReBeL® robotic arm was placed in front of a conveyor belt with various pizza products. Using a webcam and an AI-based segmentation system (Segment Anything Model, or SAM), the system identified the various objects on the conveyor belt and their position. The language model mapped these objects to the right products, based on the language instructions, and placed the products in the correct boxes by following the instructions of the language model.

The project has helped igus® show customers the potential for real manufacturing processes to be automated with low-cost robotics.

Alexander Mühlens, head of Low-Cost Automation at igus®, emphasised the importance of this hackathon: “The cooperation with these institutes gives us the opportunity to show our customers what is possible with the help of AI and low-cost robotics. Our dream is to be able to easily control robots via a voice command, in any application.” The hackathon also showed that the automation of many other processes is possible using LLMs and robotics, such as packing several boxes at the same time, sorting products according to special instructions or checking ingredients for allergens using camera systems.

The successful collaboration between igus®, the three research institutes and the University of Bonn underlines the potential of AI for industrial automation.

About the partners

Fraunhofer IAIS is one of the leading research institutes in the fields of artificial intelligence, machine learning and big data, while the University of Bonn is known as a centre for artificial intelligence and robotics. The Fraunhofer IML contributed its expertise in logistics automation and material flow and the Lamarr Institute is a leader in international AI research and machine learning.