Enabling Moleculer Circuit Breaker: Decision Guide, Trade‑offs & Validation

Why a Circuit Breaker Matters in Moleculer

When a microservice repeatedly fails to reach a downstream dependency, the failures can cascade through the system, overwhelming other services and causing a domino effect. Moleculer’s built‑in Circuit Breaker middleware automatically tracks consecutive failures per service and, once a threshold is reached, opens the breaker. Calls then return immediately with a BreakerOpenError, giving the downstream services time to recover.

Decision: Enable the Circuit Breaker?

Use the breaker when:

• Services call external APIs or databases that might become unavailable.
• You need to protect the system from cascading failures.
• Your actions are idempotent or you can safely retry.

Do not enable it if:

• All calls are guaranteed to succeed (e.g., local in‑memory operations).
• You cannot tolerate the immediate failure response during recovery.
• You rely on non‑idempotent writes without compensating actions.

Configuring the Breaker: Options & Trade‑offs

Moleculer exposes a breaker object in the service definition. The most common properties are:

• threshold – number of consecutive failures before opening.
• timeout – time (ms) to mark a request as failed if no response is received.
• resetTimeout – wait time (ms) before attempting to close the breaker.
• enabled – global flag to turn off the middleware.

Below is a compact comparison of typical configurations.

Option	Threshold	Timeout	Reset Timeout	When to Use
Default	5	60 000	30 000	Standard protection with moderate sensitivity.
Low Threshold	2–3	5 000	10 000	Fast protection for critical services; watch for false positives.
High Threshold	10–15	120 000	60 000	Allows more failures to surface before opening; use when transient errors are common.
Global Disable	–	–	–	During debugging or in a tightly controlled environment.
Per‑Service Override	Variable per service	Variable per service	Variable per service	Fine‑grained control for heterogeneous workloads.

Trade‑offs to Consider

• Latency vs. Resilience – When the breaker is open, calls return instantly with an error, adding no extra latency but also no attempt to recover.
• False Positives – A low threshold may block legitimate traffic during brief outages.
• Monitoring Overhead – Each open/close event is emitted; if you have many services, this can increase event traffic.
• State Consistency – The breaker state is kept in memory per node. In a cluster, a node may have a closed breaker while another has it open, leading to inconsistent behavior.
• Non‑Idempotent Operations – If your action modifies state, an immediate failure may leave the system in an unexpected state. Use compensating actions or design idempotent writes.

Concrete Implementation & Validation

Below is a minimal service that demonstrates the breaker in action.

module.exports = {
  name: "payment",
  version: 1,

  // Enable the breaker with custom thresholds
  breaker: {
    threshold: 3,          // open after 3 consecutive failures
    timeout: 5000,          // 5 s request timeout
    resetTimeout: 20000    // 20 s before retrying
  },

  actions: {
    // Simulate a flaky external call
    charge: {
      async handler(ctx) {
        // Throw an error to trigger the breaker
        throw new Error("Simulated external service failure");
      }
    }
  },

  // Listen for breaker events for debugging
  events: {
    "breaker:open": (payload) => {
      console.log(`Breaker opened for ${payload.service}`);
    },
    "breaker:close": (payload) => {
      console.log(`Breaker closed for ${payload.service}`);
    }
  }
};

**Validation Steps**

1. Run the service with moleculer run payment.js (requires Node >= 14).
2. Invoke the action three times using a Moleculer client or curl -X POST http://localhost:3000/api/payment/charge.
3. After the third failure, the breaker emits breaker:open and subsequent calls immediately return a BreakerOpenError (HTTP 503 if exposed via REST).
4. Wait resetTimeout (20 s) and invoke the action again; the breaker will attempt to close and the action will retry.
5. Verify metrics by enabling Moleculer’s metrics middleware and checking the circuitbreaker_opened counter.

**Limitations**

• In a multi‑node deployment, each node tracks its own breaker state. Use a shared state store or a global breaker if consistent behavior is required.
• Setting the threshold too low can cause legitimate traffic to be blocked during transient outages. Tune based on observed error patterns.
• Non‑idempotent actions may require additional error handling or compensating logic when the breaker opens.

Monitoring and Alerting

Moleculer emits breaker:open and breaker:close events. Integrate with your observability stack (Prometheus, Grafana, ELK) to alert on frequent opens. Example Prometheus query:

sum(rate(circuitbreaker_opened{service="payment"}[5m]))

A sudden spike indicates potential instability in the downstream dependency.

Conclusion

Enabling the Circuit Breaker in Moleculer is a straightforward way to guard against cascading failures. By tuning threshold, timeout, and resetTimeout per service, you can balance resilience against the risk of false positives. Always validate in a staging environment and monitor breaker metrics to ensure the configuration aligns with your operational expectations.