Portable shell escape dependency declaration for LaTeX packages

Question

Package authors sometimes need to invoke external tools during compilation through \write18, but there is no widely adopted metadata mechanism to declare that dependency so engines and distributions can enforce or prompt for the correct permission level. While pdfTeX, XeTeX, and LuaTeX expose command-line flags to control shell escape, their default security postures differ, and the command whitelist available in restricted mode is not standardized across engines or distribution versions.

LuaTeX adds further ambiguity because its os.execute and io functions are governed by a security layer that does not align directly with the traditional \write18 restricted mode. A document that relies on shell escape may therefore behave differently depending on the engine and the local TeX installation, without any explicit warning to the user.

When a package genuinely requires shell escape—for example, to run a conversion tool at build time—the author cannot portably signal that requirement so the build system or user knows to adjust engine flags. This leaves the permission boundary implicit and dependent on distribution defaults.

Is there a documented or emerging convention for declaring shell escape dependencies in package metadata that works consistently across pdfTeX, XeTeX, and LuaTeX? How should authors reconcile LuaTeX's native os security layer with the engine-agnostic \write18 restricted mode? What configuration strategy keeps a project compilable when the local distribution defaults to restricted shell escape?

Tasadduq Burney · Answer

Short answer
There is no standardized package-metadata field or engine-agnostic convention for declaring shell-escape requirements across pdfTeX, XeTeX, and LuaTeX. A \sty file cannot force the engine to enable \write18; the document or build system must pass the appropriate command-line flag. The portable approach is to check at load time, fail or degrade gracefully, and document the required flag.
Confirmed facts and assumptions

pdfTeX, XeTeX, and LuaTeX support flags such as --shell-escape, --no-shell-escape, and --shell-restricted, but default postures and restricted-mode command whitelists vary by engine and TeX distribution version.
pdfTeX exposes \pdfshellescape, and the pdftexcmds package offers a cross-engine abstraction for querying shell-escape status. Exact return values for restricted mode are build-dependent.
LuaTeX controls os.execute, io.popen, and related operations through its own security layer. That layer is related to but not identical to traditional \write18 restricted mode, so permission in one does not reliably imply permission in the other.
No engine or package manager currently enforces invented metadata keys such as RequireShellEscape.

Assumption: you are using a recent TeX Live or MiKTeX installation and can add pdftexcmds as a package dependency. If you cannot add dependencies, use the engine primitive directly where available.
Recommended package strategy

Load pdftexcmds and check shell-escape availability at package load time.
If shell escape is required and unavailable, issue a clear warning or error naming the missing external tool and the flag the user needs, for example --shell-escape.
If shell escape is optional, provide a non-shell fallback path and proceed without it.
Document the dependency in the package README and in the .dtx documentation, not in a metadata key that engines ignore.
For LuaTeX, additionally test os.execute or io.popen under the intended build flags, because \write18 status alone does not confirm Lua I/O access.

Build configuration
Configure CI or Makefile targets to compile with each target engine using both --shell-escape and --no-shell-escape. This verifies that the package either works with the flag or degrades as intended without it. Do not rely on restricted mode for portability; its allowed-command list is not standardized.
Verification
pdftex --version
lualatex --version
xelatex --version
Then compile a minimal test document under each flag and print \pdfshellescape or the pdftexcmds equivalent. In LuaTeX, run a harmless os.execute probe under the same flags to observe the native security layer.
Missing diagnostic
Which TeX engine and distribution version are you targeting? If you are using LuaTeX exclusively, the check should focus on os.execute rather than \write18; the recommendation otherwise stays the same.

Portable shell escape dependency declaration for LaTeX packages

1 answer

Short answer

Confirmed facts and assumptions

Recommended package strategy

Build configuration

Verification

Missing diagnostic

Your perspective belongs here.

0 question comments