Chuntao Guo; Jing Lin; Shunxing Bao; Xin Liu; Yaru Wang; Yunlin Chen (2026).听.听Sensors, 26(4), 1106.听
This study explores a way to automatically monitor how laboratory tasks are performed, focusing on pipetting鈥攁 common technique where small amounts of liquid are transferred using a pipette. Ensuring that such procedures are done correctly is important for safety and for producing reliable results, but it is difficult to track in real time because it involves complex hand movements, tool use, and multiple steps that vary between users. To address this, the researchers developed a vision-based artificial intelligence system that uses video recordings instead of physical sensors. The system first applies a YOLO-based model (a type of object detection algorithm) to identify human body positions and interactions with the pipette. It then uses a bidirectional long short-term memory (LSTM) network, a type of deep learning model designed to analyze sequences over time, to understand how the actions unfold step by step.
The results show that this approach can successfully distinguish between correct (standard) and incorrect (non-standard) pipetting behaviors, including different types of errors, and performs better than methods that analyze images one frame at a time without considering motion over time. Overall, the study demonstrates that AI systems using video analysis can provide a practical, non-contact way to monitor laboratory techniques, potentially improving quality control and extending to other manual procedures in scientific labs.
Figure 1.聽Representative incorrect pipetting behaviors and key challenges for vision-based QA in chemical laboratory environments.