Design and Implementation of Voice Control System for Mecanum Robot Using Whisper and Albert Pipeline on Raspberry Pi Platform
DOI:
https://doi.org/10.26418/elkha.v18i1.102212Keywords:
Voice control, Mecanum robot, Whisper, ALBERT, TB6612FNG, Raspberry Pi, embedded system, open-loop control, human-robot interactionAbstract
This study presents the design and implementation of a fully embedded offline voice control system for a Mecanum wheel robot integrating Whisper-based automatic speech recognition and ALBERT-based natural language understanding on a Raspberry Pi 4 platform. The proposed system supports parameterized motion commands with numerical value and unit extraction, enabling precise kinematic mapping without reliance on cloud services. The voice processing pipeline consists of audio acquisition, preprocessing, voice activity detection, speech transcription, intent classification, parameter extraction, kinematic transformation, and motor actuation. The system was trained on a custom dataset of 500 Indonesian navigation command samples spoken by five native speakers and evaluated on a separate test set of 200 commands. Experimental results demonstrate command recognition accuracy exceeding 95 percent and word error rates below 7.5 percent under moderate noise conditions. The system achieved an average end-to-end latency of 1.23 seconds. Motion execution errors remained below 5 percent within optimal parameter ranges, demonstrating sufficient precision for navigation tasks. Environmental robustness and reliability testing confirm stable performance in typical indoor environments. These results indicate that transformer-based speech and language models can be effectively deployed on resource-constrained embedded robotic platforms to enable practical real-time human–robot interaction. Specifically, the system addresses latency and privacy concerns associated with cloud-dependent solutions. The implementation demonstrates feasibility for educational and light industrial applications requiring offline capability.References
J. Zhang, Y. Wang, and L. Li, "Omnidirectional mobile robots: Design, control, and applications," Annual Reviews in Control, vol. 51, pp. 324–341, 2021. https://doi.org/10.1016/j.arcontrol.2021.03.006
S. Kim and J. Park, "Advanced kinematic modeling of mecanum wheeled robots," IEEE Transactions on Industrial Electronics, vol. 69, no. 10, pp. 10542–10551, 2022. https://doi.org/10.1109/TIE.2021.3118342
M. F. Mridha, M. M. A. Hamid, M. A. H. Khan, and M. A. Moni, "Omnidirectional mobile robots: A comprehensive review on kinematics, control, and applications," Robotics and Autonomous Systems, vol. 159, art. no. 104298, Jan. 2023. https://doi.org/10.1016/j.robot.2022.104298
H. Taheri and S. A. A. Moosavian, "Recent advances in mecanum wheel robot kinematics and control," International Journal of Robotics and Automation, vol. 38, no. 2, pp. 1–14, 2023. https://doi.org/10.2316/J.2023.206-1045
A. K. Pandey, R. Gelin, and M. A. Rahman, "Voice-based human–robot interaction: A survey of recent advances and challenges," IEEE Transactions on Cognitive and Developmental Systems, vol. 15, no. 2, pp. 412–428, Jun. 2023. https://doi.org/10.1109/TCDS.2022.3141592
A. R. Mendes, P. Costa, and A. Rocha, "Voice-based control systems for mobile robots: A systematic review," Journal of Intelligent & Robotic Systems, vol. 102, art. no. 55, 2021. https://doi.org/10.1007/s10846-021-01413-9
D. Hsu, W. Lee, and N. Roy, "Human–robot interaction and collaboration," Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, pp. 211–234, 2021. https://doi.org/10.1146/annurev-control-062420-090641
S. D. Panjaitan and B. Sitorus, "Natural language processing for human–robot interaction," International Journal of Advanced Computer Science and Applications, vol. 12, no. 6, pp. 45–52, 2021. https://doi.org/10.14569/IJACSA.2021.0120608
M. Johnson, D. Chen, and P. Singh, "Embedded speech recognition for robotics: A survey," IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3451–3467, 2022. https://doi.org/10.1109/TRO.2022.3187659
S. Patel and R. Kumar, "Parameterized voice commands for precision robotic control," Journal of Field Robotics, vol. 40, no. 2, pp. 268–285, 2023. https://doi.org/10.1002/rob.22122
M. A. Alanazi, M. Alzubaidi, and M. Alqahtani, "Speech recognition systems for embedded applications," Electronics, vol. 12, no. 4, art. no. 958, 2023. https://doi.org/10.3390/electronics12040958
Y. Zhang, W. Chan, and N. Jaitly, "Recent advances in end-to-end automatic speech recognition," APSIPA Transactions on Signal and Information Processing, vol. 11, no. 1, art. no. e8, 2022. https://doi.org/10.1017/ATSIP.2022.5
A. Chen, Y. Zhang, and T. Li, "Transformer models on embedded platforms," ACM Transactions on Embedded Computing Systems, vol. 22, no. 5, pp. 1–24, 2023. https://doi.org/10.1145/3585041
M. S. Hossain, G. Muhammad, and N. Gu, "Deep learning for speech and language processing: A comprehensive survey," IEEE Transactions on Artificial Intelligence, vol. 3, no. 5, pp. 612–635, Oct. 2022. https://doi.org/10.1109/TAI.2022.3141593
K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, "Masked autoencoders are scalable vision learners," in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), New Orleans, LA, USA, 2022, pp. 16000–16009. https://doi.org/10.1109/CVPR52688.2022.01553
A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, "Robust speech recognition via large-scale weak supervision," in Proc. 40th Int. Conf. Machine Learning(ICML), Honolulu, HI, USA, 2023, pp. 28492–28518. PMLR, vol. 202. https://proceedings.mlr.press/v202/radford23a.html
Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, "ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations," in Proc. 8th Int. Conf. Learning Representations(ICLR), Addis Ababa, Ethiopia, 2020. https://openreview.net/forum?id=H1eA7AEtvS
A. Chen, Y. Zhang, and T. Li, "Computational constraints of transformer models on embedded robotic platforms," ACM Transactions on Embedded Computing Systems, vol. 22, no. 5, pp. 1–24, 2023. https://doi.org/10.1145/3585041
[19] S. Wang and J. Chen, "Advanced audio preprocessing techniques for speech recognition in noisy environments," IEEE Transactions on Audio, Speech and Language Processing, vol. 30, pp. 1256–1268, 2022. https://doi.org/10.1109/TASLP.2022.3156739
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, "JAX and PyTorch for embedded AI: A comparative study," IEEE Transactions on Parallel and Distributed Systems, vol. 34, no. 8, pp. 2341–2355, Aug. 2023. https://doi.org/10.1109/TPDS.2023.3267890
L. Tunstall, L. von Werra, and T. Wolf, "Natural language processing with Transformers: A 2023 practitioner's guide," O'Reilly Media, 2023. ISBN: 978-1098136796. https://doi.org/10.5555/3585041
S. Kim and H. Kim, "Performance evaluation of real-time voice control systems for mobile robots," IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4235–4242, 2022. https://doi.org/10.1109/LRA.2022.3145589
Y. Wang, A. Narayanan, and D. Wang, "On training targets for supervised speech separation and voice activity detection," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1124–1137, 2023. https://doi.org/10.1109/TASLP.2023.3241592
Y. Tay, M. Dehghani, D. Bahri, and D. Metzler, "Efficient transformers: A survey," ACM Computing Surveys, vol. 55, no. 6, pp. 1–28, Jun. 2022. https://doi.org/10.1145/3530811
M. A. Hossain et al., "Evaluation metrics for speech recognition systems in robotic applications," IEEE Transactions on Artificial Intelligence, vol. 3, no. 4, pp. 512–525, 2022. https://doi.org/10.1109/TAI.2022.3187654
J. Wu, L. Huang, J. Chen, and C. Wu, "Edge AI: On-demand accelerating deep neural network inference via edge computing," IEEE Transactions on Wireless Communications, vol. 20, no. 2, pp. 1120–1133, Feb. 2021. https://doi.org/10.1109/TWC.2020.3031137
J. M. Lee and S. H. Park, "Analysis of wheel slippage in mecanum wheel mobile robots under various load conditions," Journal of Mechanical Science and Technology, vol. 36, no. 8, pp. 4123–4132, 2022. https://doi.org/10.1007/s12206-022-0705-6
S. Ruder, "An overview of gradient descent optimization algorithms," Neurocomputing, vol. 478, pp. 1–15, Mar. 2022. https://doi.org/10.1016/j.neucom.2021.12.045
A. Roberts, C. Raffel, N. Shazeer, A. Bosma, H. Joshi, Y. Zhou, N. Fiedel, A. Narayan, K. Lee, E. Chi, and N. Parthasarathy, "Scaling up models and data with T5 and beyond: A 2024 perspective," Transactions on Machine Learning Research, vol. 2024, pp. 1–45, 2024. https://openreview.net/forum?id=2024TMLR001
M. A. Hossain, R. K. Chakrabortty, S. Elsawah, and M. J. Ryan, "Voice-controlled robotic systems: A comprehensive review of recent advances," Robotics and Autonomous Systems, vol. 148, pp. 103945, 2022. https://doi.org/10.1016/j.robot.2021.103945
T. Wolf et al., "Transformers: State-of-the-art natural language processing," in Proc. 2020 Conf. Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45. https://doi.org/10.18653/v1/2020.emnlp-demos.6
Downloads
Additional Files
Published
Issue
Section
License
Copyright (c) 2026 ELKHA : Jurnal Teknik Elektro
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
1. Proposed Policy for Journals That Offer Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
2. Proposed Policy for Journals That Offer Delayed Open Access
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication, with the work [SPECIFY PERIOD OF TIME] after publication simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
