Written by Christian Scholz, Associate Partner, Go Reply
Quality control represents a crucial step in many industrial contexts. While the motivation can be varied such as integrity of building goods, safety of foods, fabric condition or the arrangement in a retail shelf, the goals of a manufacturer or retailer are typically quite similar. Ideally, these control steps can be carried out quickly, are cheap and reproducible and have a vanishing error rate. However, solving all these demands simultaneously can be a demanding task.
Many control steps rely on visual information whether automated or taken manually. Evaluating product photos in a standardised manner can be problematic due to varying quality, lighting and visibility.
Here, rule-based approaches using well-established computer-vision algorithms can be helpful but rapidly reach their limits. These typically employ a fixed set of filters with a finite capacity to grasp the variances across realistic situations. Furthermore, special evaluation processes might need a manual assessment. These are especially time consuming and often need employees with unique expertise. In addition, analysis steps which are carried out manually tend to be less standardized and fluctuate substantially.
In this context machine learning approaches can be beneficial either alone or in hybrid systems. In recent years the application of deep convolutional neural networks, the architecture used when processing image data, showed potential in tackling different tasks. These tasks include detecting anomalies, recognising product types and detecting defects. The advantage being that algorithms based on deep neural networks can more easily generalise. This means that they can capture image variety and product outliers better than rule-based approaches. They are also more robust against environmental changes.
An improved and extended ability to process image data has several consequences. Quality assurance processes can advance further and production and controlling cycles can be sped up. Valuable human resources can be reinvested into other challenges. Level of objectivity is increased with standardised evaluation processes, improving the comparability across the supply chain e.g. between different factories, employees, regions and point-of-sales. The performance of existing procedures can be enhanced with skilful engineering of algorithms. Completely new areas of automation could be explored, which were previously not feasible.
One use case for applying these techniques lies in retail. Manufacturers’ representatives often visit outlets to assess shelf situations,e.g. distribution of products, shelf placement, possible out-of-stock situations or information about competing products. Currently the representative manually collects this data and fills in a matrix to compare it with previous visits.
This kind of data collection has some drawbacks and often leads to visits taking longer than scheduled. The displayed sorting order rarely corresponds to the running direction and entering the information on a laptop without a proper surface can be cumbersome. Besides, direct feedback for the representative is often missing, because the backend evaluation does not run continuously.
One potential solution can be built around a web-application the representative can use from their mobile. The only manual task is to take a photograph of the shelf being assessed. All other processes are automatically carried out in the background. The application employs Google Cloud technology. Taking the computational load to the cloud makes using powerful machine learning models possible. Specifically, services from the AI-platform and the ML-API are implemented. In a first step all objects, like product packages and price tags, are detected. Then, the price-tag text is captured (Optical Character Recognition) and each one is related to the products in its proximity. Afterwards the results are clearly-laid-out depicting an annotated image and all key-performance indicators. Deviation from target values are visible immediately and the representative is able to quickly react.
In conclusion, image evaluation can be greatly enhanced via machine learning methods. These approaches show better performance and react more robustly to environment changes. Wherever quality assurance is based on image data, quality assessment can be significantly accelerated and standardised. Tasks which involve manual data acquisition can be completed faster and consequent reactions can be addressed earlier.