All papers

Research/ IN-SPACE SERVICING

PPO vs. SAC: A Comparative Study for Cooperative Motion Control of Dual-Arm Space Robots

Nasim Fallahiarezoodar, Zheng H. Zhu

AAS/AIAA Astrodynamics 2026/
Space × Robotics × AIIN-SPACE SERVICINGORBITAL ROBOTICS
Technique

Learning-based MethodsOpenAlex topic: Reinforcement Learning in Robotics · Manipulation & GraspingOpenAlex topic: Robot Manipulation and Learning · Motion Planning & NavigationOpenAlex keyword: cooperative motion planning

The growing population of orbital debris has intensified interest in active debris removal and on-orbit servicing, for which dual-arm space manipulators offer improved grasping stability and load distribution relative to single-arm systems. However, coordinating two manipulators under inter-arm collision avoidance, orientation alignment, and sensing uncertainty remains challenging. This paper develops a coordinated reinforcement learning framework for cooperative motion planning of a dual-arm space manipulator in a target-reaching task with inter-arm collision-avoidance constraints. A centralized state representation combines joint-space, end-effector, and target-relative information, while a multi-objective reward function promotes target-reaching accuracy, orientation alignment, motion smoothness, synchronized task completion, and collision avoidance. Within this framework, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) are compared as representative on-policy and off-policy algorithms, and the learned policies are further evaluated for generalization to previously unseen target configurations. Across five random seeds, SAC substantially outperforms PPO within the considered fixed-base simulation scenarios: it reaches an 80% success threshold in every seed (median 1.6M environment steps) whereas PPO does not reach it within a 4M-step budget, and it attains a higher mean success rate (78.6% versus 7.4%) with lower position and orientation errors and tighter bimanual synchronization. Both algorithms degrade on unseen boundary targets, but SAC retains most of its performance while PPO's collapses. The results also expose a failure mode of PPO: under the two-phase shaped reward, it tends to converge to a reward-loitering local optimum rather than committing to the precision task. This behavior highlights the sensitivity of on-policy learning to reward shaping.

Comments: Presented at the 2026 AAS/AIAA Astrodynamics Specialist Conference, Whistler, British Columbia, Canada, July 26-30, 2026. Paper AAS 26-740

Related papers

The 10 papers closest in meaning to this one, from embeddings of each title and abstract, with a small boost for shared missions and techniques.

More from AAS/AIAA Astrodynamics

See all →

Latest in IN-SPACE SERVICING

See all →

Shortcuts

j / k
Next / previous paper
Enter
Open paper
a
Toggle abstract
s
Save to reading list
c
Copy BibTeX
?
Show shortcuts