Dynlink Plugin Loading and Version Compatibility for OCaml Zero‑Downtime Updates
0 reputation · 01 Oct 2020, 16:10 UTC
0 reputation · 01 Oct 2020, 16:10 UTC
Goal: update a small OCaml service without stopping traffic by loading a new version of a plugin via Dynlink while the host process continues running.
Constraint: Dynlink.loadfile can load a plugin, but once a module is loaded there is no documented way to unload or replace it; the host remains tied to the original plugin’s code and data.
Uncertainty: whether rebuilding the plugin’s .cmxs file and re‑loading it with the same Dynlink call (perhaps after setting allow_unsafe_modules) yields a safe update, or if any change to core modules forces a full process restart.
End with specific questions: Can a plugin be safely reloaded after its .cmxs is rebuilt without restarting the host? Does enabling allow_unsafe_modules permit replacement of an already‑loaded plugin? If core modules change, is a rolling restart behind a load balancer the only viable option for zero‑downtime migration?
34775 reputation · 01 Oct 2020, 19:08 UTC
No – a plugin that has already been loaded with Dynlink.loadfile cannot be safely replaced or unloaded while the host process stays running. The runtime keeps the original code and data alive, and there is no API to discard it. Enabling allow_unsafe_modules only turns off safety checks; it does not give you an unload/reload capability.
Dynlink.loadfile performs incremental linking via the #load and #run primitives; once a module is linked it remains in the runtime image.unload operation; the only way to remove a module’s code is to let the process exit.LinkError or segmentation faults.allow_unsafe_modules disables the safety check that prevents loading of modules that mutate the runtime state, but it does not change the linking model.If the plugin is pure and does not retain mutable global or thread‑local state, you can load a newly built .cmxs under a fresh module name and route new requests to it while letting existing requests finish using the old version. The old module stays resident, but as long as no references to it remain it will be garbage‑collected when the process ends. This pattern works only when:
Whether the plugin holds mutable global variables or thread‑local storage that must be cleared before the new version can be used safely. If such state exists, the only safe zero‑downtime approach is to run two host instances behind a load balancer and perform a rolling restart.
ocamlopt -shared -o plugin_new.cmxs plugin.mllet _ = Dynlink.loadfile "plugin_new.cmxs"Any change to the host’s own compiled modules (the binary that calls Dynlink) requires a full process restart, because the runtime image of the host cannot be updated incrementally. In that case a rolling restart behind a load balancer is the only viable zero‑downtime strategy.
Use comments to ask for clarification. Post a solution as an answer.
6,260 reputation · 01 Oct 2020, 21:02 UTC
One detail worth adding to the answer above: even with allow_unsafe_modules set, loading a rebuilt .cmxs doesn't shadow the original module. Both versions end up linked into the runtime, and any closure the host captured before the reload still points at the old code. So you can get a split-brain situation where new requests hit v2 while in-flight work silently runs v1 — including v1's global state, which is now duplicated rather than shared.
If hot-swapping matters more than keeping everything in one process, the more robust pattern is moving the plugin into its own process and talking to it over a Unix socket or similar IPC. The host then just reconnects after the plugin process restarts, which gives you real unload semantics the Dynlink API can't. Otherwise, for core-module changes, a rolling restart behind a load balancer is the honest answer — Dynlink was designed for extension, not replacement.