Designing CI Pipelines Around Codeac Incremental Analysis

Requirements

Teams adopting Codeac typically need fast feedback on every pull request while keeping the CI runtime predictable. The core requirements are:

• Analyze only the files that changed since the last successful scan.
• Produce machine‑readable results (SARIF) for downstream tooling such as GitHub code scanning or custom dashboards.
• Persist the incremental state across ephemeral CI runners without manual intervention.
• Detect when the cached state is no longer valid (e.g., dependency upgrades, configuration changes) and fall back to a full analysis automatically.

Smallest Suitable Design

The minimal architecture that satisfies the above consists of three components:

1. Scanner binary – the Codeac CLI invoked with --incremental.
2. Cache volume – a directory (default .codeac/cache) that lives beside the repository checkout and is mounted into each CI job.
3. Artifact store – CI‑native cache (GitHub Actions actions/cache, GitLab CI cache:, CircleCI save_cache) that backs up .codeac/cache between runs.

No separate service, database, or message queue is required. The scanner reads the cache on start, computes a fingerprint of the current dependency graph (lock files, package.json, go.mod, etc.), and decides whether to reuse the previous analysis or discard the cache and run a full scan.

Trust and Data Boundaries

Boundary	Data Flow	Trust Assumption
Source → Scanner	Source files, lock files, config	Scanner runs in the same trust zone as the build; no external secrets are read.
Scanner ↔ Cache Volume	Incremental index, file‑level hashes	Cache is considered trusted only while its fingerprint matches the current dependency graph. A mismatch triggers invalidation.
Cache Volume ↔ CI Cache Service	Serialized cache tarball	CI cache service is treated as an untrusted storage layer; integrity is verified by the fingerprint check on restore.
Scanner → SARIF Consumer	SARIF JSON	Downstream tools trust the SARIF output only if the scan completed without cache‑invalidation errors.

Because the cache can be restored on a different runner, the fingerprint verification is the single point that prevents stale results from being accepted.

Operational Checks

Add the following steps to every CI job that runs Codeac:

1. Restore cache – use the CI provider’s cache action with a key that includes the hash of all lock files (e.g., codeac-${{ hashFiles('**/package-lock.json', '**/yarn.lock', '**/go.sum') }}).
2. Run incremental scan:

   codeac scan --incremental --format sarif --output results.sarif

   Run this in the repository root; the CLI automatically looks for .codeac/cache.
3. Validate exit code – Codeac returns 0 on success, 2 when the cache was invalidated and a full scan executed, and non‑zero on fatal errors. Treat 2 as a warning, not a failure.
4. Upload SARIF – feed results.sarif to the platform’s code‑scanning endpoint.
5. Save cache – push .codeac/cache back to the CI cache service using the same key.

A quick sanity check: after a successful run, inspect .codeac/cache/meta.json (if present) and verify that fingerprint matches the current lock‑file hash.

Failure Modes

• Cache corruption – If the cache directory is partially written (e.g., runner OOM), the fingerprint check will fail and a full scan runs. No false negatives are introduced, but pipeline time spikes.
• Stale cache restored – When the cache key does not include all relevant lock files, a mismatched fingerprint may go unnoticed. Mitigation: include every lock file pattern in the cache key.
• Disk pressure – Large monorepos can grow the cache to several gigabytes. Monitor du -sh .codeac/cache in a periodic job; set a retention policy (e.g., prune entries older than 30 days).
• Network‑bound cache restore – On self‑hosted runners with slow artifact storage, cache download may dominate the job. Consider a local shared volume for the cache if the runner fleet is stable.

Conditions That Would Change the Design

• Multi‑repo analysis – If a single Codeac invocation must cover several repositories, a shared cache service (e.g., S3 with versioned objects) replaces the per‑repo CI cache.
• Policy‑driven full scans – Compliance rules may require a full scan on every merge to main. Add a conditional step: if: github.ref == 'refs/heads/main' then run codeac scan --format sarif without --incremental.
• Remote scanner – For environments that forbid installing the CLI, a thin wrapper can POST source archives to a hosted Codeac API; the trust boundary then shifts to the API endpoint and its authentication tokens.
• Cache encryption requirement – If the cache must be encrypted at rest, enable CI‑provider encryption or encrypt the tarball before upload (e.g., gpg --symmetric) and decrypt on restore.

Verification Checklist (Run Before Merging)

1. Create a test branch, modify a single source file, push, and confirm the CI job finishes in significantly less time than a baseline full scan (target ≤ 30 % of full‑scan duration).
2. Delete .codeac/cache locally, run the same command, and verify the log shows Cache invalidated, performing full analysis and exit code 2.
3. Introduce a dependency version bump in package.json, push, and ensure the pipeline runs a full scan (fingerprint mismatch).
4. Inspect the uploaded SARIF for the expected rule IDs; spot‑check that findings from unchanged files are still present.

If any step behaves unexpectedly, treat the pipeline as unverified and revert to a full‑scan‑only configuration until the root cause is identified.

Limitations

• Incremental mode only tracks file‑level changes; cross‑file refactorings that affect analysis results in untouched files are not re‑evaluated until a full scan runs.
• Cache growth is unbounded by default; large histories can exceed runner disk quotas.
• The fingerprint algorithm is opaque; custom build steps that generate code at build time may not be captured unless their outputs are part of the lock‑file hash.

These limits are documented in the Codeac CLI reference (version ≥ 2.4) and should be revisited when the codebase exceeds ~500 k lines or when CI runtime budgets tighten.