Trajectory-Preserving Skeletal Representation for Real-Time Gesture Recognition Using Machine Learning: An Evaluation on SHREC’17

Per-frame wrist anchoring, a common step in landmark-based gesture recognition, erases the hand trajectory and with it most of the signal that separates dynamic gestures. On SHREC’17 this cost a compact BiLSTM recogniser about twenty accuracy points and twenty-five points of recall on trajectory-defined gestures. Restoring the trajectory relative to the first frame and adding joint velocities, chosen on validation data, brought a 0.38 M-parameter model to 92.8% and 88.6% on the 14- and 28-gesture tasks, within the range of specialised published methods. Ablations showed that a short window suffices and that self-attention did not help on this benchmark.

[1]     C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, B.  Zhang, C.-L. Chang, M. G. Yong, J. Lee, W.-T. Chang, W. Hua, M. Georg, and M. Grundmann, “MediaPipe: A framework for building perception pipelines,” arXiv preprint arXiv:1906.08172, 2019. [2]     B. Sundar and T. Bagyammal, “American Sign Language recognition for alphabets using MediaPipe and LSTM,” in Procedia Computer Science, vol. 215, 2022, pp. 642–651. [3]     S. Kamble, “SLRNet: A real-time LSTM-based sign language recogni-tion system,” arXiv preprint arXiv:2506.11154, 2025. [4]     M. Garg et al., “MP-GestLSTM: Real-time gesture detection using MediaPipe and LSTM,” Systems Science & Control Engineering, 2025. [5]     Q. De Smedt, H. Wannous, J.-P. Vandeborre, J. Guerry, B. Le Saux, and D. Filliat, “SHREC’17 track: 3D hand gesture recognition using a depth and skeletal dataset,” in Eurographics Workshop on 3D Object Retrieval, 2017. [6]     J. Liu, A. Shahroudy, D. Xu, and G. Wang, “Spatio-temporal LSTM with trust gates for 3D human action recognition,” in European Conference on Computer Vision (ECCV), 2016, pp. 816–833. [7]     S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in AAAI Conference on Artificial Intelligence, vol. 32, 2018, pp. 7444–7451. [8]     L. Shi, Y. Zhang, J. Cheng, and H. Lu, “Two-stream adaptive graph con-volutional networks for skeleton-based action recognition,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 12 026–12 035. [9]     Y. Chen, Z. Zhang, C. Yuan, B. Li, Y. Deng, and W. Hu, “Channel-wise topology refinement graph convolution for skeleton-based action recognition,” in IEEE/CVF International Conference on Computer Vi-sion (ICCV), 2021, pp. 13 359–13 368. [10]    A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang, “NTU RGB+D: A large scale dataset for 3D human activity analysis,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1010–1019.