Designing Eclipse JDT Incremental Compilation: Light‑Weight Architecture and Operational Safeguards

Problem Statement

When developers edit Java files in Eclipse, the IDE must re‑compile only the changed files to keep the workspace responsive. The incremental compilation feature of the Java Development Tools (JDT) must therefore detect edits, schedule a background job, resolve classpaths, and report errors without disrupting the user experience. The challenge is to build a minimal, robust architecture that balances performance, correctness, and maintainability.

Requirements

• Detect file changes in the workspace and trigger recompilation only for dirty resources.
• Run compilation in a background job that does not block the UI or interfere with other tasks.
• Resolve classpaths accurately, reacting to changes in .classpath files or project references.
• Report compilation errors via Eclipse markers and offer quick‑fixes (e.g., clean workspace).
• Guard against memory exhaustion during large recompiles.
• Provide clear operational checks and diagnostics for developers and maintainers.

Smallest Suitable Design

The core of the architecture is a Singleton Job that aggregates dirty resources. The design uses four key components:

1. File Change Listener – implements IResourceChangeListener to capture POST_CHANGE events.
2. Dirty Resource Queue – a thread‑safe ConcurrentLinkedQueue<IResource> that collects changed files.
3. Compilation Job – a single Job instance scheduled with low priority, which pulls resources from the queue and invokes org.eclipse.jdt.internal.compiler.batch.Compiler via the public org.eclipse.jdt.core.compiler.batch.BatchCompiler API.
4. Classpath Listener – implements IClasspathChangedListener to flush the queue when a project’s classpath changes.

When the listener receives a change event, it adds the affected IFile to the queue and schedules the job if it isn’t already running. The job waits a short debounce period (e.g., 500 ms) to coalesce rapid edits, then processes all queued files in a single compiler pass. This minimal design avoids per‑file jobs, reduces context switches, and keeps memory usage predictable.

Trust and Data Boundaries

• Workspace Metadata – The job reads the workspace’s .metadata to determine project structure and classpath entries. Changes to .classpath files trigger the classpath listener.
• Compiler API – The job uses only the public batch compiler API, ensuring no internal JDT state is mutated directly.
• Markers – Errors are reported via IMarker objects attached to the affected files, guaranteeing that the IDE’s error view remains the single source of truth.

Operational Checks

To maintain stability, the architecture includes runtime checks:

• Job Scheduling – The job is scheduled with Job.SHORT priority and Job.LONG for large compilations. It checks Job.getCurrentJob() to avoid duplicate runs.
• Memory Guard – Before starting, the job queries Runtime.getRuntime().maxMemory() and aborts if the estimated compile footprint exceeds 70% of the limit, posting a marker that suggests increasing heap size.
• Classpath Validation – The job verifies that every referenced project is resolved. If a project is missing or its .classpath is malformed, a marker is created on the project root.
• Error Reporting – All compiler diagnostics are converted to markers with severity IMarker.SEVERITY_ERROR or IMarker.SEVERITY_WARNING and a quick‑fix that runs org.eclipse.core.resources.ResourcesPlugin.getWorkspace().getRoot().refreshLocal() to clean stale metadata.

Failure Modes and Mitigation

• Stale Classpath Entries – If a referenced JAR is removed from the filesystem but still listed in .classpath, the compiler throws FileNotFoundException. The job catches this and creates a marker on the project root with a quick‑fix to remove the entry.
• Missing Dependencies – Unresolved types generate compiler errors. Markers are attached to the offending files, and the quick‑fix offers to add the missing dependency via the org.eclipse.jdt.ui.wizards.NewJavaProjectWizard.
• Corrupted Workspace Metadata – If the workspace’s .metadata is inconsistent (e.g., missing org.eclipse.jdt.core settings), the job logs a warning and prompts the user to run Clean Workspace from the Project menu.
• OutOfMemoryError – Large projects may exhaust the JVM. The memory guard above prevents this by aborting early and advising the user to increase the heap via -Xmx.

When to Redesign

The minimal design works well for typical single‑user workspaces. However, consider redesigning under these conditions:

• Multi‑User or Shared Workspaces – Concurrency becomes a concern. Introduce per‑project jobs or use a thread‑pool with per‑project locks to avoid race conditions.
• Very Large Projects – A single job may become a bottleneck. Split the queue into per‑project subqueues and schedule parallel jobs with careful resource limits.
• Custom Build Configurations – If projects use non‑standard build paths or external tools, extend the classpath listener to incorporate those paths before compilation.
• Performance Regression – If users report UI lag, profile the job to identify hotspots and consider incremental source scanning (e.g., only re‑parse changed files).

Concrete Example: Plugin Skeleton

public class IncrementalCompileActivator extends AbstractUIPlugin {
    private static IncrementalCompileActivator instance;
    private final DirtyQueue queue = new DirtyQueue();
    private final CompileJob job = new CompileJob(queue);

    public void start(BundleContext context) throws Exception {
        super.start(context);
        instance = this;
        IWorkspace workspace = ResourcesPlugin.getWorkspace();
        workspace.addResourceChangeListener(new ChangeListener(queue, job), IResourceChangeEvent.POST_CHANGE);
        workspace.addClasspathChangedListener(new ClasspathListener(queue));
    }

    public void stop(BundleContext context) throws Exception {
        instance = null;
        super.stop(context);
    }

    public static IncrementalCompileActivator getDefault() {
        return instance;
    }
}

class ChangeListener implements IResourceChangeListener {
    private final DirtyQueue queue;
    private final CompileJob job;
    ChangeListener(DirtyQueue q, CompileJob j) { this.queue = q; this.job = j; }
    @Override
    public void resourceChanged(IResourceChangeEvent event) {
        try { event.getDelta().accept(new ResourceDeltaVisitor(queue)); } catch (CoreException e) { e.printStackTrace(); }
        job.schedule();
    }
}

class CompileJob extends Job {
    private final DirtyQueue queue;
    CompileJob(DirtyQueue q) { super("Eclipse Incremental Compile"); this.queue = q; }
    @Override
    protected IStatus run(IProgressMonitor monitor) {
        List<IFile> files = queue.drain();
        if (files.isEmpty()) return Status.OK_STATUS;
        // Build options
        Map<String, String> options = JavaCore.getOptions();
        // Compile
        try { BatchCompiler.compile(files, options, null, monitor); }
        catch (Exception e) { return new Status(IStatus.ERROR, "org.eclipse.jdt.core", e.getMessage(), e); }
        return Status.OK_STATUS;
    }
}

Running this plugin in a test workspace shows that only modified files trigger the CompileJob and that errors are surfaced as markers. Use the Eclipse console to verify job priority by inserting System.out.println("Job running: " + getName()); inside run().

Verification Checklist

• Create a small plugin project and install the activator.
• Open a Java file, edit it, and observe that only the changed file is compiled.
• Introduce a missing import and confirm a marker appears with a quick‑fix.
• Delete a referenced JAR and verify a marker on the project root suggesting a clean workspace.
• Measure memory usage during a large compile; ensure the job aborts if memory exceeds the threshold.

Conclusion

The architecture above delivers a lightweight, reliable incremental compiler for Eclipse JDT. By centralizing change detection, batching recompiles, and enforcing strict operational checks, it keeps the IDE responsive while providing clear diagnostics. When projects grow or collaboration patterns shift, the design can be expanded with per‑project jobs or enhanced concurrency controls, preserving the core principles of minimalism and robustness.