Analyzing the potential of a time-of-flight depth sensor for assembly assistance

Products are becoming more complex and more varied with technological progress. In these circumstances, the assembly process is more prone to errors while training new workers. Adding an assistive system for monitoring the work to give feedback could help to detect errors immediately and to reduce the propagation of errors. Affordable depth sensors that use the time-of-flight technology could be utilized to monitor the assembly process in real-time and to receive immediate feedback. This work evaluates the potential of a depth sensor for assistance in manual assembly. Because these sensors have problems that are difficult to generalize due to external factors such as material properties influencing the measurement, different problems and implications of using these depth sensors for the task were highlighted with a series of experiments with an Azure Kinect, a time-of-flight depth sensor. Rules and restrictions were derived from the results with the goal of improving the use of depth sensors for assistive systems in a manufacturing process. From the experiments and preceding experiences, a couple of rules and restrictions were derived for using depth sensors in that scenario. The results show that although the approach works in general, problems occur in certain conditions, especially due to optical properties of materials for near-infrared wavelengths, such as reflectivity and transmittance, because of the technology of the depth sensor.