Validation of a MediaPipe system for markerless motion analysis during virtual reality rehabilitation

Markerless systems and inertial sensors, such as inertial measurement units, offer significant advantages over traditional marker-based methods by eliminating the need for laboratory conditions or markers placement. Virtual reality is an innovative technology used in rehabilitation, recently investigated for obtaining kinematic motion data. This research aims to validate a markerless system using a MediaPipe-based algorithm to reconstruct joint kinematics during exercises with the Nirvana virtual reality rehabilitation system. In the first phase, the markerless system was evaluated against the Xsens inertial measurement unit system, analyzing knee, hip, elbow and shoulder movements of 7 healthy participants at the Politecnico di Milano. The MediaPipe algorithm showed high performance in tracking lower limb movements but was less accurate for upper limbs, as indicated by RMSE and %RMSE values. However, it accurately replicated the time course of all angles based on ICC and Spearman’s coefficient, suggesting virtual reality’s feasibility in rehabilitation and movement analysis. In the second phase, a pilot study was carried out at the Fondazione TOG on 2 participants with left hemiparesis to assess the efficacy of the system in a pathological context. The results provide a basis for evaluating the potential clinical applicability of the system.