Session: 15-01-01: ASME International Undergraduate Research and Design Exposition

Paper Number: 149873

149873 - An Automated Machine Vision Training Machine for Real-Time Sorting Applications 

Object recognition in real-time using advanced algorithms such as YOLOV8 makes possible a whole class of new and transformative applications. Beyond the need for real-time applications such as quality control, security, and inspection, there are many applications where there are hundreds, thousands, or more of object types that must be recognized. For example, sorting recycling or waste streams that contain large numbers of varied products which each exist in various damaged, dirtied, or altered states has a large number of unique classes that should be recognized. These machine-learning algorithms require hundreds or thousands of images with hand-labeled images for each class. Scaling this labeling process is costly and time-consuming. Whole industries such as Amazon’s Mechanical Turk marketplace have been created to fill this need but they remain costly. While this approach may work for well-funded high-value commercial projects, it is not feasible for high-impact low-revenue projects. 

We have developed a methodology for automatically identifying and labeling object classes for object recognition training and constructed a machine to implement this methodology.  Our low-cost machine uses a camera that captures intermittent images in the UV and visible light spectrum to identify marked objects. Objects are then moved along a conveyor to obtain multiple images from different angles. Objects are shuffled and imaging is repeated. Then training labels along with bounding boxes or object masks are automatically generated for these objects by combining information from the UV spectrum and the visible light spectrum images. Initial testing of the machine is shown to be successful and scaling to higher numbers of object classes is currently underway. The machine uses a single OpenMV RT1602 camera which incorporates a combination of a microprocessor and camera on a single board. This streamlines the coding process and the presence of digital input/output pins on the RT1602 allows for interfacing with a motor driver and input switches. The control code is written in Python and is publicly available via a GitHub repository. Future work discussing improvements, extensions, and additional automation of the machine will be discussed. 

In addition to the cost and time required, there is always the possibility that human error leads to mislabeling during the image hand-labeling process. With access to precisely labeled ground truth images created automatically by the system, we have also undertaken a study of the impact of user error on the accuracy of the trained systems by introducing precise and controlled perturbations of the labeling boxes. The sensitivity of the accuracy measure to these perturbations will be presented. 

In conclusion, our newly designed object labeling system can greatly improve labeling speed, remove user error in labeling, be scaled to larger numbers of classes at low cost, and create opportunities for new applications in the field of object recognition.

Presenting Author: Emma Capaldi Phillips Academy Andover

Presenting Author Biography: The author is interested in using machine learning and robotics to improve human health and benefit society. She has previously worked on medical applications of soft robotics and educational robotics platforms.

Authors:

Emma Capaldi Phillips Academy Andover
Annina Capaldi Phillips Academy Andover