Norg: Minimal‑Design Architecture for Java‑Native Object Graph Serialization

Problem & Takeaway

When persisting or transmitting complex object graphs—especially those with cycles—developers often fall back on Java’s built‑in serialization or third‑party solutions that add overhead or security risk. Norg offers a lightweight, annotation‑driven API that preserves graph structure, supports versioning, and enforces a strict class whitelist, making it an attractive choice for mission‑critical services.

Requirements

• Serialize arbitrary object graphs, including cycles, without infinite recursion.
• Maintain backward compatibility via explicit schema versioning.
• Minimize runtime overhead: use annotations, no heavy reflection during normal operation.
• Enforce security: restrict deserialization to known classes.
• Provide clear operational metrics: memory usage, GC pressure, throughput.

Smallest Suitable Design

At its core, Norg consists of three components:

1. Annotation Processor – @NorgSerializable marks classes and fields.
2. Serializer – walks the graph, assigns unique IDs, writes a header with a schemaVersion and a classRegistry map.
3. Deserializer – reads the header, checks the whitelist, reconstructs objects using the ID map to re‑establish references.

Because Norg uses a header‑first approach, the entire graph must fit in memory. No streaming API is provided, so the design is ideal for graphs that comfortably fit within the JVM heap.

Trust & Data Boundaries

1. Whitelist Enforcement – Before deserialization, Norg checks that every class encountered in the header is present in a developer‑supplied Set<Class>. If a class is missing, deserialization aborts with NorgSecurityException.

2. Header Integrity – The header contains a CRC32 checksum of the serialized payload. This guards against tampering and partial corruption.

3. Version Pinning – The schemaVersion in the header must match the runtime library’s supported version. Mismatches trigger a NorgVersionMismatchException.

Operational Checks

To validate a production deployment, run the following diagnostics:

java -jar norg-demo.jar --run-tests

• Test 1: ObjectGraphRoundTripTest – verifies cycle preservation.
• Test 2: MemoryProfile – uses jcmd to capture heap snapshots during serialization of a 1M‑node graph.
• Test 3: SecurityWhitelistTest – attempts to deserialize a payload containing an unknown class and expects NorgSecurityException.

Expected checks:

• Heap usage < 70% of max heap.
• GC pause times < 5ms per cycle.
• Whitelist test passes, no exception thrown when whitelist is correct; exception thrown when whitelist is missing the class.

Failure Modes

Mode	Cause	Impact	Mitigation
GC pressure	Large graphs allocate many temporary objects during traversal.	Throughput drop, possible OutOfMemoryError.	Increase heap, enable incremental GC, or split graph into sub‑graphs.
Whitelist breach	Deserialization of unknown class.	Arbitrary code execution risk.	Strict whitelist, runtime checks, audit logs.
Version mismatch	Client writes with newer schema version than server.	Deserialization failure.	Implement version negotiation or fallback.
Corrupted header	Network packet loss or disk corruption.	Deserialization abort.	CRC check, retry logic.

When to Change the Design

Consider revising the architecture if:

• Streaming is required – e.g., graphs > 10M nodes or memory‑constrained environments.
• Performance bottlenecks – GC spikes exceed thresholds despite tuning.
• New security policies demand per‑field encryption or signing.
• Interop with non‑Java systems becomes necessary, requiring a more standard format (e.g., Protobuf).

In such scenarios, a hybrid approach—using Norg for small, trusted graphs and a streaming serializer for large, external data—can be adopted.

Concrete Example

Below is a minimal Java example that demonstrates a cyclic graph, serialization, and secure deserialization.

import io.norg.*;

@NorgSerializable
class Node {
    @NorgField
    String name;
    @NorgField
    Node next;

    Node(String name) { this.name = name; }
}

public class Demo {
    public static void main(String[] args) throws Exception {
        // Build a cycle: A → B → C → A
        Node a = new Node("A");
        Node b = new Node("B");
        Node c = new Node("C");
        a.next = b; b.next = c; c.next = a;

        // Serialize to byte array
        byte[] data = Norg.serialize(a);

        // Prepare whitelist (only Node is allowed)
        Set<Class> whitelist = Set.of(Node.class);
        Norg.setWhitelist(whitelist);

        // Deserialize
        Node deserialized = (Node) Norg.deserialize(data);
        System.out.println(deserialized.next.next.next.name); // prints "A"
    }
}

Key points:

• Annotations keep the API lightweight; only annotated fields participate.
• The header includes schemaVersion = 1 and the class registry containing Node.
• During deserialization, Norg verifies the whitelist before instantiating any object.

Comparison with Alternatives

Feature	Norg	Java Serialization	Kryo
Cycle support	Yes (ID mapping)	Yes (ObjectStream)	Yes (handles references)
Reflection overhead	Low (annotation‑driven)	High (full reflection)	Moderate (opt‑in)
Security	Whitelist + CRC	None (vulnerable)	Optional (custom class resolver)
Streaming	No	No	Yes (ObjectOutputStream)
Versioning	Explicit header	Implicit (serialVersionUID)	Optional (custom schema)

Practical Verification Steps

1. Run unit tests – ensure round‑trip integrity.
2. Profile memory – jcmd VM.native_memory summary before/after serialization.
3. Simulate corruption – truncate the byte array, confirm CRC failure.
4. Whitelist test – remove Node.class from whitelist, confirm NorgSecurityException.

All checks should pass in a healthy environment. If any fail, revisit the corresponding mitigation strategy.

Conclusion

Norg delivers a compact, secure, and version‑aware solution for Java object graph serialization. Its minimal API and explicit trust boundaries make it suitable for high‑integrity services that can tolerate in‑memory graph processing. When scaling beyond these constraints, consider augmenting Norg with streaming adapters or migrating to a format that inherently supports large‑scale data.