Monitoring labour productivity on a construction site using machine learning algorithms

Introduction. The modern construction industry actively uses end-to-end technologies to assess the productivity of construction workers, reducing the number of defective products. Innovative approaches increase the effectiveness of monitoring workers’ actions on construction sites. The use of modern computer vision technologies, video analytics, and machine learning algorithms can improve the objectivity of assessing productivity on construction sites.

Materials and methods. The developed automated system for analyzing workers’ actions to control labour productivity at construction sites includes a module for recognizing workers’ actions and poses, an action classification unit, and a component for highlighting key points. The MediaPipe platform with the BlazePose model identifies 33 anatomical points on the body to assess the worker’s posture. The computer vision system provides continuous monitoring and accurate recognition of movements. The technology ensures continuous recognition of the worker, even when there is a temporary loss of visual contact, and creates a database with worker characteristics for analytical purposes. The use of an automated system eliminates the need to consider the time factor when analyzing data, which increases the efficiency of the process and allows you to focus on the elements of the scene. By dividing the recording into logical segments, each element of the workflow is examined. LSTM recurrent neural networks optimize the analysis of workers’ actions.

Results. The effectiveness of the automated system was evaluated using a test video of the masonry construction process to determine its accuracy and generalization capabilities. The accuracy of identifying the builder’s actions reached 80.1%. During a total of 46.5 seconds out of the 58-second video, the model correctly recognized the builder’s actions.

Conclusions. The results of the work carried out prove the effectiveness of an advanced system for monitoring labour productivity in construction based on visual recognition technologies. The system provides comprehensive monitoring of work operations, equipment, and external conditions at the facility. The algorithms and information base ensure reliable measurements and the possibility of implementing the methodology in construction production.

1. Xu M., Mei Zh., Luo S., Tan Yi. Optimization Algorithms for Construction Site Layout Planning : A Systematic Literature Review. Engineering Construction & Architectural Management. 2020; 27(8):1913-1938. DOI: 10.1108/ECAM-08-2019-0457. EDN ZQJLJW.

2. Al Hawarneh A., Bendak S., Ghanim F. Dynamic Facilities Planning Model for Large Scale Construction Projects. Automation in Construction. 2019; 98:72-89. DOI: 10.1016/j.autcon.2018.11.021

3. Kaveh A., Vazirinia Y. Construction Site Layout Planning Problem Using Metaheuristic Algorithms : A Comparative Study. Iranian Journal of Science and Technology, Transactions of Civil Engineering. 2019; 43(2):105-115. DOI: 10.1007/s40996-018-0148-6

4. Jaafar K., Elbarkouky R., Kennedy J. Construction Site Layout Optimization Model Considering Cost and Safety in a Dynamic Environment. Asian Journal of Civil Engineering. 2021; 22(2):297-312. DOI: 10.1007/s42107-020-00314-3. EDN ASHFQJ.

5. Prayogo D., Cheng M.Yu., Wu Yu.W., Redi A.A.N.P., Yu V.F., Persada S.F. et al. A Novel Hybrid Metaheuristic Algorithm for Optimization of Construction Management Site Layout Planning. Algorithms. 2020; 13(5):117. DOI: 10.3390/a13050117. EDN VOBMGW.

6. Arshad S., Akinade O., Bello S., Bilal M. Computer Vision and IoT Research Landscape for Health and Safety Management on Construction Sites. Journal of Building Engineering. 2023; 76:107049. DOI: 10.1016/j.jobe.2023.107049. EDN ZQJSLG.

7. Tian Yu., Chen J., Kim Ju.In., Kim J. Lightweight Deep Learning Framework for Recognizing Construction Workers' Activities Based on Simplified Node Combinations. Automation in Construction. 2024; 158:105236. DOI: 10.1016/j.autcon.2023.105236. EDN PITKJM.

8. Baek J., Kim D., Choi B. Deep Learning-Based Automated Productivity Monitoring for on-Site Module Installation in off-Site Construction. Developments in the Built Environment. 2024; 18:100382. DOI: 10.1016/j.dibe.2024.100382. EDN TOTRVH.

9. Chen X., Wang Y., Wang J., Bouferguene A., Al-Hussein M. Vision-Based Real-Time Process Monitoring and Problem Feedback for Productivity-Oriented Analysis in off-Site Construction. Automation in Construction. 2024; 162:105389. DOI: 10.1016/j.autcon.2024.105389. EDN INIOZE.

10. Qi K., Owusu E.K., Francis Siu M.F., Albert Chan P.Ch. A Systematic Review of Construction Labor Productivity Studies: Clustering and Analysis through Hierarchical Latent Dirichlet Allocation. Ain Shams Engineering Journal. 2024; 15(9):102896. DOI: 10.1016/j.asej.2024.102896. EDN NKMMDC.

11. Zhang M., Zhou Ya., Xu X., Ren Z., Zhang Y., Liu Sh. et al. Multi-View Emotional Expressions Dataset Using 2D Pose Estimation. Scientific Data. 2023; 10(1):649. DOI: 10.1038/s41597-023-02551-y. EDN ISMSVS.

12. Bora J., Dehingia S., Boruah A., Chetia A.A., Gogoi D. Real-Time Assamese Sign Language Recognition Using MediaPipe and Deep Learning. Procedia Computer Science. 2023; 218:1384-1393. DOI: 10.1016/j.procs.2023.01.117. EDN EWJOKP.

13. Sundar B., Bagyammal T. American Sign Language Recognition for Alphabets Using MediaPipe and LSTM. Procedia Computer Science. 2022; 215:642-651. DOI: 10.1016/j.procs.2022.12.066. EDN SCJTAC.

14. Kikuta T., Chun P. Development of an Action Classification Method for Construction Sites Combining Pose Assessment and Object Proximity Evaluation. Journal of Ambient Intelligence and Humanized Computing. 2024; 15(4):2255-2267. DOI: 10.1007/s12652-024-04753-7. EDN KRRKHU.

15. Jang Y., Jeong I., Younesi Heravi M., Sarkar S., Shin H., Ahn Y. Multi-Camera-Based Human Activity Recognition for Human–Robot Collaboration in Construction. Sensors. 2023; 23(15):6997. DOI: 10.3390/s23156997. EDN ROYSYQ.

16. Xiao B., Zhang Y., Chen Y., Yin X. A Semi-Supervised Learning Detection Method for Vision-Based Monitoring of Construction Sites by Integrating Teacher-Student Networks and Data Augmentation. Advanced Engineering Informatics. 2021; 50:101372. DOI: 10.1016/j.aei.2021.101372. EDN HHONZJ.

17. Chen C., Zhu Z., Hammad A. Automated Excavators Activity Recognition and Productivity Analysis from Construction Site Surveillance Videos. Automation in Construction. 2020; 110:103045. DOI: 10.1016/j.autcon.2019.103045. EDN BMUNVW.

18. Alashhab S., Gallego A.J., Lozano M.Á. Efficient Gesture Recognition for the Assistance of Visually Impaired People Using Multi-Head Neural Networks. Engineering Applications of Artificial Intelligence. 2022; 114:105188. DOI: 10.1016/j.engappai.2022.105188. EDN RLLYMM.

19. Gladyshev M.D., Rybakov A.V. Integration of sensor data and mathematical modeling of underwater robot behavior using a digital twin. Advanced Engineering Research (Rostov-on-Don). 2025; 25(2):142-151. DOI: 10.23947/2687-1653-2025-25-2-142-151. EDN LDXARH. (rus.).

20. Alpatova M.V., Rudyak Yu.V. Optimal 2D placement of virtual objects in physical space for augmented reality applications. Advanced Engineering Research (Rostov-on-Don). 2023; 23(4):410-421. DOI: 10.23947/2687-1653-2023-23-4-410-421. EDN QPFEMG. (rus.).

21. Karakozova I.V., Lisitsyn I.M., Boldyshev K.V. Methods of designing work operations in the conditions of digital technologies implementation in the construction industry. Vestnik MGSU [Monthly Journal on Construction and Architecture]. 2024; 19(8):1373-1389. DOI: 10.22227/1997-0935.2024.8.1373-1389. EDN GXBZQY. (rus.).