Occlusion, image noise, and insufficient local joint information remain challenges in 3D hand pose estimation from monocular RGB image sequences. To address them, we propose BFHand, which extracts hand joint heatmaps from consecutive frames and estimates 3D pose from multi-frame 2D joint representations. For global feature modeling, BFHand combines ModernTCN and multi-head self-attention to capture temporal motion, while a multi-frame GraphMLP models joint relationships across frames. For the target frame, a context-aware mixture-of-experts module (CAMoE) uses context-driven routing and feature modulation to combine the outputs of multiple GCN experts. EnhGraphMLP then selectively strengthens local joint features. On the HanCo dataset, BFHand achieves an MPJPE of 12.447 mm and a PA-MPJPE of 5.504 mm. The results indicate improved estimation accuracy, with more plausible hand structures and greater temporal stability.

Ruihang Li, Dongming Yan. 3D Hand Pose Estimation Based on Monocular RGB Image Sequences. Academic Journal of Computing & Information Science (2026), Vol. 9, Issue 8: 69-78. https://doi.org/10.25236/AJCIS.2026.090809.

[1] Liu, X., Ren, P., Gao, Y., Wang, J., Sun, H., Qi, Q., ... & Liao, J. (2024, March). Keypoint fusion for rgb-d based 3d hand pose estimation. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 4, pp. 3756-3764). https://doi.org/10.1609/aaai.v38i4.28166 

[2] Zheng, X., Wen, C., Xue, Z., Ren, P., & Wang, J. (2023, October). Hamuco: Hand pose estimation via multiview collaborative self-supervised learning. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 20706-20716). IEEE.https://doi.org/10.48550/arXiv.2302.00988 

[3] Liu, R., Ohkawa, T., Zhang, M., & Sato, Y. (2024, June). Single-to-dual-view adaptation for egocentric 3d hand pose estimation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 677-686). IEEE.https://doi.org/10.48550/arXiv.2403.04381 

[4] Liao, Z., Zhu, J., Wang, C., Hu, H., & Waslander, S. L. (2024, June). Multiple view geometry transformers for 3d human pose estimation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 708-717). IEEE. https://doi.org/10.48550/arXiv.2311.10983 

[5] Wu, S., Jin, S., Liu, W., Bai, L., Qian, C., Liu, D., & Ouyang, W. (2021). Graph-based 3d multi-person pose estimation using multi-view images. In 2021 IEEE. In CVF International Conference on Computer Vision (ICCV) (pp. 11128-11137). https://doi.org/10.48550/arXiv.2109.05885 

[6] Khaleghi, L., Sepas-Moghaddam, A., Marshall, J., & Etemad, A. (2022). Multiview video-based 3-d hand pose estimation. IEEE Transactions on Artificial Intelligence, 4(4), 896-909. https://doi.org/10.1109/TAI.2022.3195968 

[7] Li, S., Su, W., Luo, G., Tian, J., Han, Y., & Liu, L., et al. (2025). Monocular 3d hand pose estimation based on high-resolution network. Advances in Continuous and Discrete Models, 2025(1), 1-24. https://link.springer.com/article/10.1186/s13662-025-03948-2 

[8] Chen, X., Wang, G., Guo, H., & Zhang, C.. (2017). Pose guided structured region ensemble network for cascaded hand pose estimation. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.06.097 

[9] Zhang, F., Zhao, L., Li, S., Su, W., Liu, L., & Tao, W. (2024). 3d hand pose and shape estimation from monocular rgb via efficient 2d cues. https://doi.org/10.1007/s41095-023-0346-4 

[10] Zhou, F., Shen, P., Dai, J., Jiang, N., Hu, Y., & Lai, Y. K., et al. (2024). Ahrnet: attention and heatmap-based regressor for hand pose estimation and mesh recovery. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3000-3004. https://doi.org/10.1109/ICASSP48485.2024.10446600 

[11] Chen, X., Liu, Y., Dong, Y., Zhang, X., Ma, C., & Xiong, Y., et al. (2021). Mobrecon: mobile-friendly hand mesh reconstruction from monocular image. https://doi.org/10.48550/arXiv.2112.02753

[12] Sharma, S., & Huang, S. (2021). An end-to-end framework for unconstrained monocular 3D hand pose estimation. Pattern Recognition, 115, 107892. https://doi.org/10.1016/j.patcog.2021.107892 

[13] Ren, P., Wang, J., Sun, H., Qi, Q., Liu, X., & Zhang, M., et al. Prior-Aware Dynamic Temporal Modeling Framework for Sequential 3D Hand Pose Estimation. 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE. https://doi.org/10.1109/ICCV51701.2025.00611 

[14] Duran, E., Kocabas, M., Choutas, V., Fan, Z., & Black, M. J. (2023). Hmp: hand motion priors for pose and shape estimation from video. 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.1109/WACV57701.2024.00623 

[15] Wen, Y., Pan, H., Yang, L., Pan, J., Komura, T., & Wang, W. (2023). Hierarchical temporal transformer for 3d hand pose estimation and action recognition from egocentric rgb videos. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 21243-21253. https://doi.org/10.48550/arXiv.2209.09484

[16] Ye, Y., Feng, Y., Taheri, O., Feng, H., Tulsiani, S., & Black, M. J. (2025). Predicting 4D Hand Trajectory from Monocular Videos. https://doi.org/10.48550/arXiv.2501.08329

[17] Li, W., Liu, M., Liu, H., Guo, T., Wang, T., & Tang, H., et al. (2025). Graphmlp: a graph mlp-like architecture for 3d human pose estimation. Pattern Recognition, 158(000). https://doi.org/10.1016/j.patcog.2024.110925

[18] Luo, D., & Wang, X. (2024, May). ModernTCN: A Modern Pure Convolution Structure for General Time Series Analysis. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=vpJMJerXHU

[19] Park, J., Jung, D. S., Moon, G., & Lee, K. M. (2023, October). Extract-and-adaptation network for 3D interacting hand mesh recovery. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (pp. 4202-4211). IEEE. https://doi.org/10.48550/arXiv.2309.01943

[20] Pavlakos, G., Shan, D., Radosavovic, I., Kanazawa, A., Fouhey, D., & Malik, J. (2024, June). Reconstructing hands in 3d with transformers. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 9826-9836). IEEE. https://doi.org/10.1109/CVPR52733.2024.00938 

[21] Liu, S., Tripathi, S., Majumdar, S., & Wang, X. (2022, June). Joint hand motion and interaction hotspots prediction from egocentric videos. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3272-3282). IEEE. https://doi.org/10.48550/arXiv.2204.01696

[22] Gao, Q., Ju, Z., Chen, Y., Wang, Q., & Chi, C. (2022). An efficient RGB-D hand gesture detection framework for dexterous robot hand-arm teleoperation system. IEEE Transactions on Human-Machine Systems, 53(1), 13-23. https://doi.org/10.1109/THMS.2022.3206663

[23] Rudnev, V., Golyanik, V., Wang, J., Seidel, H. P., Mueller, F., Elgharib, M., & Theobalt, C. (2021, October). Eventhands: Real-time neural 3d hand pose estimation from an event stream. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 12365-12375). IEEE. https://doi.org/10.48550/arXiv.2012.06475

[24] Jiang, J., Li, J., Zhang, B., Deng, X., & Shi, B. (2023). Evhandpose: event-based 3d hand pose estimation with sparse supervision. ArXiv, abs/2303.02862. https://doi.org/10.1109/TPAMI.2024.3380648

[25] Chamorro, S., Collier, J., & Grondin, F. (2021). Neural network based lidar gesture recognition for realtime robot teleoperation.https://doi.org/10.1109/SSRR53300.2021.9597855

[26] Meng, L., & Yizhou, Y. (2025). Overlock: An overview-first-look-closely-next convnet with context-mixing dynamic kernels. In 2025 IEEE. In CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 128-138). https://doi.org/10.48550/arXiv.2502.20087

[27] Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2025). LSNet: See Large, Focus Small.https://doi.org/10.1109/CVPR52734.2025.00908 

[28] Cai, W., Jiang, J., Wang, F., Tang, J., Kim, S. H., & Huang, J. (2025). A survey on mixture of experts in large language models. IEEE Transactions on Knowledge and Data Engineering, 1-20. https://doi.org/10.1109/tkde.2025.3554028 

[29] Puigcerver, J., Riquelme, C., Mustafa, B., & Houlsby, N. (2023). From sparse to soft mixtures of experts. https://doi.org/10.48550/arXiv.2308.00951 

[30] Potamias, R. A., Zhang, J., Deng, J., & Zafeiriou, S. Wilor: End-to-end 3d hand localization and reconstruction in-the-wild. In 2025 IEEE. In CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 12242-12254). https://doi.org/10.48550/arXiv.2409.12259

[31] Lin, W., Hu, Y., Dai, L., Tang, X., & Chen, X. (2026). Geohand: unlocking prior geometry knowledge for monocular 3d hand reconstruction. https://doi.org/10.48550/arXiv.2605.17354

[32] Zimmermann, C., Argus, M., & Brox, T. (2021). Contrastive representation learning for hand shape estimation. https://doi.org/10.48550/arXiv.2106.04324

[33] Chen, X., Liu, Y., Ma, C., Chang, J., Wang, H., & Chen, T., et al. (2021). Camera-space hand mesh recovery via semantic aggregation and adaptive 2d-1d registration. https://doi.org/10.48550/arXiv.2103.02845 

[34] Dong, H., Chharia, A., Gou, W., Carrasco, F. V., & De la Torre, F. (2024). Hamba: Single-view 3d hand reconstruction with graph-guided bi-scanning mamba. Advances in Neural Information Processing Systems, 37, https://doi.org/2127-2160. 10.52202/079017-0069