Research
(*: equal contribution)
Please feel free to contact me if you are interested in any of the following works.
Journal Publications
Optimal gait design for a soft quadruped robot via multi-fidelity Bayesian optimization
Authors: Kaige Tan, Xuezhi Niu, Qinglei Ji, Lei Feng & Martin Törngren
Applied Soft Computing, 2025
[Paper] [Code] [Abstract] [BibTeX]
This study focuses on the locomotion capability improvement in a tendon-driven soft quadruped robot through an online adaptive learning approach. Leveraging the inverse kinematics model of the soft quadruped robot, we employ a central pattern generator to design a parametric gait pattern, and use Bayesian optimization (BO) to find the optimal parameters. Further, to address the challenges of modeling discrepancies, we implement a multi-fidelity BO approach, combining data from both simulation and physical experiments throughout training and optimization. This strategy enables the adaptive refinement of the gait pattern and ensures a smooth transition from simulation to real-world deployment for the controller. Compared to previous result using a fixed gait pattern, the multi-fidelity BO approach improves the robot’s average walking speed from 0.14 m/s to 0.214 m/s, an increase of 52.7%. Moreover, we integrate a computational task off-loading architecture by edge computing, which reduces the onboard computational and memory overhead, to improve real-time control performance and facilitate an effective online learning process. The proposed approach successfully achieves optimal walking gait design for physical deployment with high efficiency, effectively addressing challenges related to the reality gap in soft robotics.
@article{TAN2025112568,
title = {Optimal gait design for a soft quadruped robot via multi-fidelity Bayesian optimization},
journal = {Applied Soft Computing},
volume = {169},
pages = {112568},
year = {2025},
issn = {1568-4946},
doi = {https://doi.org/10.1016/j.asoc.2024.112568},
url = {https://www.sciencedirect.com/science/article/pii/S1568494624013425},
author = {Kaige Tan and Xuezhi Niu and Qinglei Ji and Lei Feng and Martin Törngren},
keywords = {Soft quadruped robot, Reality gap, Multi-fidelity Bayesian optimization, Edge computing},
}Conference Publications
Investigating Symbiosis in Robotic Ecosystems: A Case Study for Multi-Robot Reinforcement Learning Reward Shaping
Authors: Xuezhi Niu & Didem Gürdür Broo
2025 9th International Conference on Robotics and Automation Sciences (ICRAS)
[PDF pending] [Code] [Abstract] [BibTeX]
This paper presents a bio-inspired reward shaping approach for multi-agent reinforcement learning (MARL) in heterogeneous multi-robot systems, leveraging a formal symbiosis model to enhance cooperation. We categorize interactions based on mutualism, commensalism, and parasitism, introducing constructs such as graph models, state transition systems, and resource flow models to characterize inter-agent dependencies. By incorporating a taxonomy of symbiotic relationships into MARL, we define reward structures that reinforce cooperative behavior in complex tasks. Our experimental results demonstrate that while traditional rewards suffice for simple tasks like CartPendulum, mutualistic rewards provide qualitative benefits in high-dimensional tasks such as ShadowHand Object Passing and Mobile Manipulation, including increased learning stability, smoother convergence, and reduced performance variance. These findings suggest that symbiotic reward shaping provides a structured mechanism for enhancing multi-robot cooperation, with benefits that extend beyond numerical performance metrics. Future work should explore adaptive interaction mechanisms and generalization across diverse robotic applications.
@inproceedings{niu2025symbiosis,
title={Investigating Symbiosis in Robotic Ecosystems: A Case Study for Multi-Robot Reinforcement Learning Reward Shaping},
author = {Xuezhi Niu and Didem Gürdür Broo},
booktitle = {the 2025 9th International Conference on Robotics and Automation Sciences (ICRAS)},
year = {2025},
publisher = {IEEE}
}
Enabling Symbiosis in Multi-Robot Systems through Multi-Agent Reinforcement Learning
Authors: Xuezhi Niu, Natalia Calvo Barajas & Didem Gürdür Broo
2025 IEEE 8th International Conference on Industrial Cyber-Physical Systems (ICPS)
[Paper] [Slides] [Abstract] [BibTeX]
Current cyber-physical systems, including multi-robot systems, often fail to interoperate effectively, resulting in suboptimal performance, inefficient resource utilization, and poor resilience. Inspired by natural symbiotic relationships, such as tree-fungi networks, we propose an architecture that integrates ecological symbiosis principles into multi-robot system specifications. Specifically, we incorporate symbiotic principles into multiagent reinforcement learning (MARL) within a centralized training, decentralized execution framework. Comprehensive scenario-based evaluations in a simulated warehouse environment show that our symbiotic MARL framework improves system performance (10.7%) and resource utilization (13.81%) compared to non-symbiotic baselines. Agents dynamically adjust their behavior in response to environmental changes, ensuring continuous task execution, efficient navigation, and balanced energy use. These findings demonstrate that integrating ecological principles into MARL enhances the system's efficiency and performance. The framework's success in promoting sustainable resource usage while maintaining high task performance suggests broader applications across various cyber-physical domains where adaptive coordination is crucial.
@INPROCEEDINGS{niu2025enabling,
author={Niu, Xuezhi and Barajas, Natalia Calvo and Broo, Didem Gürdür},
booktitle={2025 IEEE 8th International Conference on Industrial Cyber-Physical Systems (ICPS)},
title={Enabling Symbiosis in Multi-Robot Systems Through Multi-Agent Reinforcement Learning},
year={2025},
volume={},
number={},
pages={1-7},
keywords={Symbiosis;Training;Navigation;System performance;Reinforcement learning;Cyber-physical systems;Multi-robot systems;Resource management;Sustainable development;Resilience;Multi-robot systems;Multi-agent reinforcement learning;Symbiotic design;Energy management;Sustainability;Biomimicry},
doi={10.1109/ICPS65515.2025.11087893}
}
Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning
Authors: Xuezhi Niu*, Kaige Tan*, Didem Gürdür Broo & Lei Feng
2025 IEEE International Conference on Robotics and Automation (ICRA)
[PDF] [Code] [Abstract] [BibTeX]
We present a model-based reinforcement learning framework for gait optimization on a tendon-driven soft quadruped (SoftQ). The method combines a surrogate dynamics model with policy improvement and post-training fine-tuning to achieve stable, energy-aware locomotion. Sim-to-real considerations and ablations indicate improved sample efficiency and robustness compared to standard model-free baselines.
@inproceedings{niu2025optimal,
title = {Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning},
author = {Niu, Xuezhi and Tan, Kaige and G{\"u}rd{\"u}r Broo, Didem and Feng, Lei},
booktitle = {2025 IEEE International Conference on Robotics and Automation (ICRA)},
year = {2025},
doi = {10.48550/arXiv.2406.07069},
url = {https://arxiv.org/abs/2406.07069}
}Thesis
- Optimal Gait Control of Soft Quadruped Robot by Model-based Reinforcement Learning
Xuezhi Niu
M.Sc. Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2023
Technical Report
- Electronically Vacuum Regulated Shut-off Valve for Milking System
Carl Egenäs*, Felix Ekman*, Chenqi Ma*, Tim Naser*, Xuezhi Niu*, Axel Sernelin*, Samuel Stenow*, Benjamin StrÖm*
H.K. Project, KTH Royal Institute of Technology, Stockholm, Sweden, 2023
Reviewer
IEEE International Conference on Robotics and Automation (ICRA), IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), IEEE International Conference on Industrial Cyber-Physical Systems (ICPS), IEEE-RAS International Conference on Humanoid Robots (Humanoids), IEEE International Conference on Robot and Human Interactive Communication (ROMAN).