Projectname:
AI-based condition recognition from single images as data source for operator assistance systems and quality assurance
Workgroup: Applied Digitalization
Research Partner and Scientific Guidance:
Financing: IVLV e. V.
Duration: 2022
Processing and packaging operations are characterized by a multitude of micro-disturbances. For several years now, operator assistance systems have come into focus to support operators in finding the best solution. These systems usually work with two parts: In an upstream situation diagnosis, an attempt is made to identify the malfunction as best as possible. With the situation description made, information is presented from a database on the cause of the malfunction and suitable actions to eliminate the malfunction. However, especially in the first part, the situation diagnosis, there is the problem that a part of the situations cannot be detected by the integrated sensors but can only be recognized visually.
If experts with process knowledge are asked about their strategies for fault analysis, it becomes apparent that assessing the visual impression of a product can provide clues to the causes of faults. For example, the cause of a misaligned printed image on a chocolate bar wrapped in foil can pro-vide clues as to whether the cause lies in the foil take-off or in the wrapping process. Experienced employees could recognize from the pouring cone on a dosing nozzle for chocolate mass whether a regularly shaped product would be produced later in the cooling process of the bar. There are many situations like this in the food industry, where supervisors say that personnel must be positioned at different points in the plant to monitor and take action in the event of deviations.
Cameras can be retrofitted relatively easily into existing machinery without interrupting production and interfering with machine control. Existing visual systems for quality control, however, usually only aim to detect rejects and not at eliminating the cause. The aim of the project is to generate machine learning models for the classification of images with typical faults in food production and packaging and to determine the achievable accuracy of the match to the fault causes. This means not only distinguishing “packaging has a crack” from “packaging is OK”, as is already possible with classical image analysis, but also linking the images to causes.
In image classification, computers are no longer inferior to humans in many areas, assuming sufficiently large training data sets. If only smaller data sets are available, as is the case in fault diagnosis for processing and packaging machines, the Deep Learning models can be trained with existing data sets and then transferred to the new problem (“Transfer Learning”). Recent methods from “Few Shot Learning” allow increasingly smaller data sets to be used for learning the models.
The investigation in the project is application-oriented. Images from production are analyzed by members of the project committee and described by process experts. The variety of variants in the project should be as large as possible: For example, images with cracked or compressed packages, images of pouring cones in filling processes or of products in quality assurance are suitable.
If the feasibility of the approach could be demonstrated in principle on real examples in the project, this approach should be pursued further after the project so that the results benefit many users. As a result, processing operations can run more stably, which has a positive financial payoff in terms of reduced scrap and increased production time.
Concrete results of the project will thus be:
