Diagnosing Gazebo Physics Engine and Plugin Mismatches in Gazebo 11+

When the simulation runs but the behavior is wrong

The recognizable condition is a Gazebo world that loads without crashing but produces inconsistent motion, growing joint drift, or unpredictable step times after an upgrade or when moving between machines. The useful takeaway is that in Gazebo Classic (10 and earlier) versus Gazebo Fortress and newer, the default physics engine and the plugin ABI change. A model tuned for ODE will not behave identically under Simbody or Dart, and a plugin compiled for one major version will not load safely in another.

Plugin-based extension is Gazebo's supported mechanism for custom sensors, actuators and world logic. Physics engine selection is an engineering decision that trades step time for constraint fidelity. Both are version-sensitive.

Cause / diagnostic table

Ordered checks

1. Confirm Gazebo version and default engine. Run on the host running the simulation:

   gz sim --version

   Note the major release. Gazebo Classic 10 and earlier default to ODE. Gazebo Fortress and newer default to Dart. Record the version before changing any configuration.

2. Inspect the world SDF physics block. Open the world file used by the launch and locate the physics element, typically under <world><physics>. Check the type attribute and any max_step_size or real_time_update_rate values. A missing type means the version default is used, which differs across releases.

3. Verify plugin loading. With the world running:

   gz plugin list

   Expected check: custom plugins appear by name without deprecation warnings. Risk: a mismatched plugin binary can crash the process, so run this before trusting simulation output.

4. Check thread configuration. In the world SDF or launch arguments, look for rendering_threads and physics_threads settings. Absence means a single-threaded fallback even on Gazebo 11+.

5. Verify ROS 2 bridge timing. In a separate terminal with ROS 2 sourced:

   ros2 topic echo /clock

   Compare timestamps to the simulation time reported in Gazebo. Also echo a sensor topic published by a loaded plugin to check for dropped messages.

Fixes tied to findings

Plugin not loading

If the plugin does not appear in the list, do not force-load it. Recompile the plugin against the installed Gazebo SDK headers and libraries for the current major version. Plugin API breaks between major versions; a binary built for Gazebo 9 is not safe in Gazebo 12. After rebuilding, restart the simulator and re-run the plugin list check.

Physics engine mismatch

Set the engine explicitly in the world SDF so behavior is reproducible. Example for Dart:

<physics name="default" type="dart">
  <max_step_size>0.001</max_step_size>
  <real_time_update_rate>1000</real_time_update_rate>
</physics>

For Simbody, change the type to simbody; for legacy ODE, use ode. Document the choice in the project README. Changing the engine changes results; do not compare motion studies across engines without recording the engine and solver parameters.

Step time and core utilization

If step time is high and cores are idle, add explicit thread counts matched to available hardware, for example:

<physics>
  <physics_threads>4</physics_threads>
</physics>

Risk: oversubscribing threads can increase latency rather than reduce it. Verify improvement by monitoring the real-time factor in the Gazebo UI before and after the change.

ROS 2 clock drift

Ensure ROS 2 nodes use simulation time by setting use_sim_time:=true in the launch configuration. Confirm message flow with ros2 topic echo and check that timestamps increase monotonically with simulation time. If drift persists, reduce bridge queue sizes and check network latency between the bridge and consumers.

Escalation criteria

• Plugin loads but causes a segmentation fault on world load. Escalate to the plugin author with the Gazebo version, SDK version, and a backtrace.
• Identical SDF produces different joint constraint errors on two machines with the same Gazebo version. Escalate after confirming identical physics engine type, solver parameters, and step size. Cross-engine differences are expected; cross-machine differences with the same engine suggest timing or uninitialized parameters.
• ROS 2 closed-loop control remains unstable after clock sync and latency checks. Escalate with topic latency measurements and the control gains in use.

Limitations and practical verification

Physics engine selection produces non-deterministic results across engines. Do not assume ODE, Simbody and Dart are interchangeable. The plugin API is not forward-compatible across major Gazebo releases; recompilation is required. Exact SDF element names and defaults can vary by release, so confirm against the installed version's documentation before committing changes.

Practical verification after changes:

• Run gz plugin list and confirm no warnings.
• Check the world SDF for an explicit physics type and thread settings.
• Echo /clock and a plugin-published sensor topic to confirm message flow and timestamp alignment.

Keep the physics engine and thread configuration in version control alongside the SDF so reproductions are explicit.