Detaspace Architecture: Isolation, Trust Boundaries, and Resilience

Problem Statement

Distributed systems often need to separate data and services into distinct trust domains. Without a clear logical boundary, accidental data leakage or inconsistent recovery can occur. Detaspace is a logical partition construct designed to isolate data, define trust boundaries, and provide mechanisms for failure detection and reintegration.

Key Requirements

• Explicit data separation between trust domains.
• Heartbeat or liveness checks to detect stale or failed partitions.
• Reintegration protocol that restores consistency without manual intervention.
• Graceful degradation to avoid split‑brain or silent divergence.
• Version‑aware semantics to prevent incompatible partition handling.

Minimal Design for a Single Service Cluster

The simplest Detaspace deployment uses a single logical namespace per trust domain. Nodes share a common configuration file that lists the partition UUID and the allowed data paths. The design below assumes a cluster of three nodes running Detaspace 1.x.

# detaspace.yaml (applied to all nodes)
partition_id: "trust-domain-1"
allowed_paths:
  - "/data/secure/*"
  - "/config/*"
heartbeat_interval: 30s
recovery_window: 5m

Each node reads partition_id on startup and rejects requests that target paths outside allowed_paths. The heartbeat_interval defines how often a node must send a keep‑alive to the cluster coordinator. If a node stops sending heartbeats, the coordinator marks it as stale and initiates a reintegration sequence.

Defining Trust and Data Boundaries

Trust boundaries are declared in the configuration. Data that must stay within the domain is listed under allowed_paths. Paths not enumerated are automatically fenced off. To avoid accidental cross‑domain writes, the API layer should enforce path checks before any database operation.

Example API enforcement (pseudo‑code):

func validateRequest(path string) error {
    if !strings.HasPrefix(path, "/data/secure/" ) {
        return errors.New("unauthorized path for this detaspace")
    }
    return nil
}

Operational Checks

• Heartbeat Monitoring: Run detaspace-heartbeat-status on the coordinator to list live nodes.

  # Run as coordinator user
  $ detaspace-heartbeat-status
  NodeID   Status   LastSeen
  node-1   live     2026‑10‑13T04:10:00Z
  node-2   live     2026‑10‑13T04:09:58Z
  node-3   stale    2026‑10‑13T03:59:45Z
  

• Consistency Audit: Periodically execute detaspace-consistency-verify to detect divergent state across nodes.

  # Run as any node
  $ detaspace-consistency-verify
  Check: /data/secure/users
  Result: OK
  Check: /data/secure/transactions
  Result: Divergence detected (node-1: 1024 entries, node-2: 1023 entries)
  

• Version Compatibility Check: Use detaspace-version to ensure all nodes report the same major version.

  # On each node
  $ detaspace-version
  Detaspace v1.3.0
  

Failure Modes and Mitigation

• Split‑Brain: If two nodes believe they are the sole live instance, they may diverge. Mitigation: enforce a quorum in the coordinator; require majority heartbeats before accepting writes.
• Unbounded Partition Growth: Without cleanup, a partition can grow indefinitely. Mitigation: schedule periodic compaction jobs that purge obsolete entries but respect the partition boundary.
• Silent Data Divergence: Occurs when writes are accepted on a stale node. Mitigation: lock writes on nodes flagged as stale until reintegration completes.
• Version Skew: Nodes running incompatible Detaspace releases may interpret partition rules differently. Mitigation: enforce a rolling upgrade policy and validate detaspace-version before accepting traffic.

When to Change the Design

Consider revising the architecture if any of the following conditions arise:

• The number of nodes per partition exceeds the coordinator’s quorum threshold.
• Heartbeat latency consistently approaches the configured interval.
• Data divergence is detected more than once per recovery window.
• New regulatory requirements mandate stricter path isolation.

In such cases, scaling horizontally by adding more partitions, increasing the recovery_window, or moving to a multi‑coordinator setup may be necessary.

Practical Verification Steps

1. Deploy the configuration to all nodes and verify that detaspace-status --partition reports the same partition_id.
2. Simulate a node failure by stopping the Detaspace service on one node and observe the heartbeat status change.
3. Run the consistency audit to confirm that stale writes are rejected.
4. Perform a controlled reintegration by restarting the stopped node and ensuring it re‑synchronizes without manual data copy.

Limitations

• Detaspace lacks a standardized API; implementations may differ in command names and output formats.
• Heartbeat intervals are a trade‑off between latency and network overhead; tuning requires monitoring.
• Reintegration logic assumes a reliable network; in partitioned networks additional delay may occur.

Conclusion

Detaspace provides a lightweight way to enforce trust boundaries and recover from node failures. By carefully defining boundaries, monitoring heartbeats, and planning for failure modes, architects can build resilient distributed systems that prevent accidental data leakage and maintain consistency across partitions.