A Study on the LLM-Based Software Architecture for Multi Robot Control
Abstract
This paper examines the latest developments in the convergence of robotics and AI, with a particular focus on analyzing key cases of robot control research using large language models (LLMs). Recent AI technologies such as computer vision, neural networks, reinforcement learning, and LLMs have significantly improved robots' visual recognition, motion control, perception, and decision-making abilities. This study presents an LLM-based robot control architecture that can be applied to small, medium-sized, and home/educational robots without requiring large-scale robot data.
This architecture proposes a 'self-describing' method that automatically generates system prompts from robot control programs, allowing non-experts to easily control robots. Additionally, it offers various LLM model selection options to reduce API usage costs and presents a multi-agent architecture for future robot functionality expansion and control of multiple robots. The feasibility of the proposed method is verified through robots tests using a robot simulator. As robot popularization expands beyond manufacturing sites into everyday life, this research is expected to lower the barriers to applying AI technology to small and medium-sized robots.
References
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I. and Amodei, D. (2020), Language models are few-shot learners, Advances in Neural Information Processing Systems, 33, 1877-1901.
- Huang, W., Abbeel, P., Pathak, D., & Mordatch, I. (2022), Language models as zero-shot planners: extracting actionable knowledge for embodied agents. Proceedings of the 39th International Conference on Machine Learning, 9383-9407.
- Javier Ruiz-del-Solar, et al. (2018), A Survey on Deep Learning Methods for Robot Vision, arXiv. https://arxiv.org/abs/1803.10862.
- Liu, R., et al. (2021), Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focused Mini-Review, arXiv:2102.04148.
- Rawas, S. (2024), AI: the Future of Humanity. Discover Artificial Intelligence. 4(25), Springer
- Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018), Improving language understanding by generative pre-training. OpenAI Technical Report.
- Devlin, J., et al. (2019), BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805.
- Zhao, W. X. and Zhou, K., et al. (2023), A survey of Large Language Models. arXiv:2303.18223
- Yi, L., Guo, L., Zhou, A., Wang, S., Xu, M. (2023), EdgeMoE: Fast on-device inference of MoE-based large language models, arXiv:2308.14352.
- Signh,I., Blukis, V., et al. (2023), ProgPrompt: Generating Situated Robot Task Plans using Large Language Models, In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), 11686-11693.
- Ahn, M., Brohan, A., Brown, N., et al. (2022), Do as I can, not as I say: grounding language in robotic affordances, arXiv:2204.01691.
- Liang, Y., Cabi, S., et al. (2023), Code as Policies: Language Model Programs for Embodied Control, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6045-6056.
- Vemprala, S., Bonatti, R., Bucker, A., Kapoor, A. (2023), ChatGPT for Robotics: Design Principles and Model Abilities, arXiv:2306.17582.
Keywords
Details
| Section | Articles |
| Issue | Vol. 1 No. 1 (2024): Volume 1 Issue 1 (Inaugural Issue) |
| Published | 2024-11-15 |
| Pages | 60-70 |
- Enrolled in the Master’s in AI•Big Data program at the Graduate School of AI, Seoul School of Integrated Sciences & Technologies (aSSIST)
- Areas of interest: AI, robot, LLM, Computer Vision, etc.
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.