Dual-Arm Task and Motion Planning Pipeline
TAMP connects symbolic task choices to geometric motion checks so a robot plan is both logical and physically executable.
Motion, sensing, control loops, and plans made visible as a field of forces and trajectories.
Interactive model
From skeleton to verified trajectory
Step through the pipeline from symbolic action skeleton to trace log.
Live HTML simulation · adjust the controls and watch the computed output respond.
Interactive
Learning can rank plans, but geometry still verifies them
This is a simplified teaching model. Its displayed values are computed from the controls; the article explains where the model stops.
Site connection
The TAMP research direction proposes using planning traces to train feasibility, scheduling, and duration guidance while preserving final motion verification.
Research direction: The source presents trace-based learned guidance as a goal and roadmap, not as a completed measured result.
Definition: Linking Discrete Intent to Continuous Motion
Task and Motion Planning (TAMP) couples symbolic decisions—pick an object, hand it off, place it in a container—with continuous decisions such as grasp pose, inverse-kinematics solution, collision-free path, and trajectory timing. A symbolic action skeleton records action order and parameters without yet proving every geometric detail.
Dual-arm manipulation adds arm assignment, synchronization, shared-workspace conflicts, and handoff configurations. A logically valid skeleton can therefore fail late when an arm cannot reach, a path collides, or both arms require incompatible space or timing.
A symbolic plan proposes what should happen; motion checking establishes whether a concrete version can happen in the modeled scene.
Why Trace Supervision Matters
Motion calls can be expensive, but their outcomes carry supervision. A trace can retain the scene, partial skeleton, arm assignment, proposed continuous parameters, check result, duration, and failure stage. Successful and failed attempts together describe where symbolic plausibility diverges from geometric reality.
The project source identifies an early feasibility predictor as the first research goal and lists skeleton ranking, overlap prediction, duration prediction, learned grasp and placement proposals, and failure-aware replanning as a broader roadmap. These are intended guidance mechanisms, not reported performance results.
| Trace stage | Possible label | Guidance use |
|---|---|---|
| IK | reachable / unreachable | Rank action parameters |
| Collision | clear / colliding | Avoid poor scene-action pairs |
| Motion | trajectory found / failed | Prioritize candidates |
| Scheduling | overlap accepted / rejected | Guide arm assignment and timing |
| Execution | completed / recovery needed | Support failure-aware replanning |
Mechanics: A Nine-Stage Research Loop
The proposed pipeline begins by randomizing dual-arm tabletop, shelf, handoff, block-tower, and container-insertion tasks. It produces symbolic skeletons, assigns actions using greedy or OR-Tools CP-SAT scheduling, and sends candidates through analytic and cuRobo-backed IK, collision, and motion adapters.
The system would then log attempts as JSONL traces, build Parquet datasets, train feasibility, duration, and scheduling guidance, insert that guidance back into search, and compare it against planner-only baselines. Isaac Sim 5.0 dual-Franka execution, ROS 2 Jazzy, BehaviorTree.CPP, MLflow, benchmark dashboards, and report/export hooks form the stated integration environment.
The feedback loop is the essential mechanism: planning generates labeled experience, learning changes candidate order or pruning, and unchanged geometric verification tests whether the guided candidate is real.
Worked Example: A Dual-Arm Handoff
Consider moving a block from a left shelf to a right-side container. A skeleton proposes: left-arm pick, move to handoff, right-arm receive, right-arm place. The scheduler assigns intervals and a synchronization point for the handoff.
Suppose the left pick and approach pass IK and collision checks, but the chosen handoff pose leaves the right arm without an IK solution. The pipeline logs a failure at the right-arm receive step rather than labeling the entire task vaguely unsuccessful. It can try another handoff pose or another skeleton; that trace later becomes a negative example for feasibility guidance.
If a learned model later gives that configuration a probability of 0.08, the planner may rank it behind stronger candidates. It must not call 0.08 a proof of impossibility: distribution shift or model error could hide a valid motion, so the system needs a pruning policy with fallbacks and motion checking remains authoritative.
Verification, Evaluation, and Limits
The stated success criteria are fewer expensive motion calls, fewer late geometric failures, lower median and tail planning time, and preserved or improved task success on held-out scene seeds. A fair comparison must use the same task distribution, time budget, verifier, and success definition for learned guidance and planner-only baselines.
Trace data is policy-dependent: the current planner decides which candidates receive checks, so the dataset may omit unexplored regions. Randomized scenes do not automatically match deployment, duration predictions are uncertain, and simulator success does not prove real-world safety. IK, collision checking, trajectory optimization, and execution validation remain the final gate.
Research status: the public source specifies architecture, integrations, goals, and evaluation targets; it does not claim completed learned-guidance benchmarks.
Common Pitfalls
- Letting a learned model replace geometric verification.
- Training only on easy scenes and expecting robust planning.
- Ignoring failed traces even though they provide valuable supervision.
- Optimizing average planning time while missing tail failures.
- Comparing methods under different motion-check or time budgets.