A globally coordinated benchmark for real-world embodied bimanual manipulation
Bimanual manipulation benchmarks today are fragmented. Some evaluate algorithms only in simulation. Others test on a single robot in a single lab. Neither answers the question that matters most: do these methods generalize across platforms, environments, and operators? The EBiM Competition introduces a globally coordinated benchmark that combines an open simulation phase with cross-continent real-robot validation on identical Mobile FR3 Duo platforms. By validating top simulation methods on real hardware in Hamburg, Munich, and Shanghai, we build the first reproducible, cross-laboratory benchmark for embodied bimanual manipulation.
Phases I and II make up the Competition. Phase III is the workshop where results are presented.
Release an open simulation benchmark focusing on unstructured manipulation scenarios: cable routing, deformable material handling (thermal pad placement), and assisted living and feeding tasks. Global teams participate through an open call on a shared, reproducible leaderboard.
Teams advancing from simulation validate on real Mobile FR3 Duo robots at partner labs in Hamburg, Munich, and Shanghai. After simulation results are announced (Aug 24), advancing teams get a hands-on bench-testing window on the real platform (Aug 25 – Sep 3, local venue time). Repository submissions are open now, since the developer-preview release; organizer-run simulation evaluation may begin at any time. From Sep 4 to Sep 12, all real-robot bench testing and evaluation are run by the organizers — ensuring cross-site comparability. The Phase II submission deadline is Sep 12, 2026 (AoE).
Competition results, invited talks, and a panel discussion are presented at a community workshop (venue and date TBD). The workshop releases the benchmark protocol, open dataset, and community roadmap for future iterations.
Phase III: Workshop & Roadmap (venue and date TBD) — see the Workshop page for the full program.
A standardized bimanual mobile manipulation platform by Franka Robotics, deployed across all international testbed sites.
Mobile FR3 Duo extends the FR3 Duo reference platform into mobile environments by integrating it with Tactile Mobile Robot (TMR). It combines dual-arm manipulation, rich perception, and omnidirectional mobility into a single embodied AI system — enabling data collection and policy execution beyond fixed workcells. All competition testbeds in Hamburg, Munich, and Shanghai use identical Mobile FR3 Duo platforms to ensure cross-site reproducibility.
Learn more at franka.de/product-prototypes · Franka Robotics GmbH
A multi-axis framework for fair and meaningful evaluation of bimanual manipulation systems.
Current bimanual manipulation benchmarks are fragmented across hardware platforms, task definitions, and evaluation criteria. Our workshop targets the design of a community-wide standard covering task diversity, generalization, robustness, and deployment readiness — evaluated on real robots under standardized conditions.
Three core tasks: cable routing, deformable material handling (thermal pad placement), and assisted living & feeding.
Three layers — Outcome (task points + completion time, ranked); Process (reported, not ranked: SPARC smoothness, torque/energy, re-grasps, handovers, and bimanual coordination via BiCoord SMP/MAD/STI); and Safety (hard-fail gates on force/damage thresholds).
Shared teleoperation, annotation schema, and hardware calibration guidelines for cross-site compatibility.
Platform-neutral APIs and sim-to-real transfer tracks to include diverse robot embodiments.
An open, continuously updated leaderboard of verified real-robot submissions. Each task is ranked independently — no overall aggregate — with per-task tie-breaks (completion time; for Assisted Living, highest completed stage → total → time). Bimanual tasks are tagged by coupling type (PerAct2); simulation results follow the RoboTwin convention.
Curated multi-site demonstration dataset with diverse operators, environments, and objects.
Three core bimanual manipulation tasks — contact-rich assembly, deformable material handling, and assisted living. Tasks run in Isaac Sim, MuJoCo, and Genesis. All tasks are evaluated at Full Autonomy.
Industrial path recognition, cable grasping, routing, and fixation.
Deformable-object manipulation (flexible cable), contact-rich interaction & path recognition, and bimanual re-grasping with dual-arm coordination.
Primary: completion rate — fixtures correctly routed in sequence and validated at Y-checkpoints; only an unbroken correct run from the start counts (a wrong routing voids the rest). Limit: 30 min · Full Autonomy only. Secondary: completion time, re-grasps, handovers. Process: SPARC smoothness, joint torque, bimanual coordination (SMP/STI).
Industrial electronics manufacturing — a Thermal Interface Material (TIM) assembly benchmark.
Deformable-material modeling (soft-body physics), non-linear adhesion and surface friction, high-precision placement of compliant objects, and contact-rich force control with tactile sensing.
Primary: valid-placement IoU — Pick Success × Orientation Success × IoU (0–1); wrong orientation scores 0. Tie-break: completion time. Process/Safety: force-threshold violations.
A four-stage kitchen-to-dining service cycle: Table Setup → Feeding → Bean Recovery → Cleanup.
Two rooms — a Kitchen and a Dining area. Objects start stacked on a plate in the kitchen (plate, cup, bowl with coffee beans, spoon); three of six seats are randomly assigned; a scaled recycling bin and a marked sink region sit in the kitchen. Simulation and real-world setups are identical.
Primary: four stages — Table Setup, Feeding, Bean Recovery, Cleanup — 4 points each (max 16); Bean Recovery scored by recovered-bean ratio. Ranking: highest completed stage → total score → completion time. Safety: peak head/face force (ISO/TS 15066), watchdog interventions.
Reference simulation environments and full scoring scripts will be released as part of the benchmark framework.
Identical Mobile FR3 Duo platforms deployed at three partner labs across Asia and Europe.
European testbed hosted at a University of Hamburg robotics lab (venue TBA); coordinates data protocols and operator calibration.
Testbed Operations Lead · Donglin Chen
Industry testbed providing platform integration and deployment support.
Testbed Operations Lead · Yuyang Tu
Asia testbed hosted at the Franka Robotics Shanghai branch office for cross-continent real-robot validation.
Testbed Operations Lead · Dr. Wenkai Chen
Future testbed — not a Phase II 2026 evaluation site
Announced for an additional competition at the end of 2026, and planned as a recurring annual testbed thereafter.
Testbed Operations Lead · Shahram Najam Syed
From open simulation to real-robot finals — the end-to-end pipeline and the five pillars that run the benchmark.
How a submission flows from upload through to physical validation.
We invite teams from academia and industry to compete in Phase I (simulation) and qualify for Phase II (real-robot validation).
All submission deadlines are Anywhere on Earth (AoE, UTC−12). Bench-testing windows run on local venue time at each testbed, not AoE. Phase II bench testing is open only to teams advancing from the Aug 24 simulation results. Questions about timing? Contact us →
Clone the repo and start building.
The EBiM Benchmark is available now as a developer preview — framework, simulation environments, robot assets, and task environments.
Check STATUS before you build: STATUS.md → lists which task and simulator combinations are usable today. The preview is under active development and may contain bugs, incomplete features, or API changes.
The Teleoperation Toolkit is released, with Keyboard, GELLO, and USB Foot Pedal control. Still coming, released incrementally: VR control, baseline models, tutorials and documentation, real-world datasets, and additional assets. The benchmark is open source — bug reports and contributions are welcome.
How to submit your team's work.
One issue per team, per task track. Update your submission any time before the deadline by opening a new issue that supersedes the old one.
A public GitHub repository containing
Dockerfile
Encapsulating your work.
README
Explaining how to run it.
Source code is not required. You may optionally link supplementary materials (e.g. Hugging Face model weights or datasets) — if you do, your README must include a clear integration guide.
Submissions are verified against your registration (team name + point-of-contact email); unregistered submissions are not evaluated. Evaluation code in the benchmark repository is a development aid — official scoring follows the rules on this page and is run by the organizers. See the schedule note above for the submission deadline.
Up to $5,250 in prizes per task — cash + purchase voucher, trophy & gift — awarded independently across all three benchmark tasks, backed by AMD, TÜV Rheinland, and our community partners.
Every prize in the two tracks below is awarded per task, independently for each of the three benchmark tasks — a team can place in every task. The InnoHub Special Award further down is scored separately; see that block for its own terms.
Per task · judged on the physical Mobile FR3 Duo
Per task · awarded in the open simulation phase
AMD development hardware distributed across three regions — U.S., Germany, and Asia — with each region receiving:
A robustness evaluation for Task 3 only, open to teams that have advanced to Phase II with a Task 3 entry. It is scored in simulation by the organizers on the Phase II policy each team has already submitted — no action is required from teams. Entries are scored out of 10 points: 4 for illumination robustness across four lighting profiles, and 6 for collision avoidance against three obstacles. Scoring is independent of the main leaderboards and does not affect main task rankings.
Awarded as one set overall, not per task. This is separate from the Simulation Prize above, which carries the same amounts on different criteria — a team may win both. There is no tie-break: teams with the same total score share a rank and each receives that rank's prize, a rank left vacant by a tie is not awarded, and teams scoring zero are not ranked. Results are announced on 1 October 2026 alongside the main results.
The lighting evaluation is designed with reference to IEC 62849; however, it assesses performance only in a simulated environment. Real-world test results have not yet been obtained, and this evaluation should not be considered equivalent to passing the lighting assessment specified in the standard. Full scoring rules: InnoHub Special Award competition rules.
Eligibility: Open to all teams participating in the simulation and real-world phases. The InnoHub Special Award is limited to Phase II teams with a Task 3 entry. See Call for Participation for details.
We gratefully acknowledge the partners who make the EBiM Competition possible.
Interested in partnering? Contact us to discuss hardware support, travel grants, or community partnership opportunities.
Phase III of the EBiM initiative is the half-day workshop (venue and date TBD), featuring invited talks, a panel discussion, and competition awards.
