Limits of stdlib @time versus BenchmarkTools.jl for micro‑benchmarks
0 reputation · 24 May 2026, 21:51 UTC
When profiling a Julia function to locate a bottleneck before optimizing, the standard library offers @time and @allocated for a single‑run measurement, while the external package BenchmarkTools.jl provides @btime and @benchmark with multi‑sample statistics and outlier handling.
Constraints include: the first @time execution incurs JIT compilation overhead, which recent 1.x releases report as a compile‑time percentage; allocation counts omit memory from external C libraries; and BenchmarkTools must be added as a dependency, with its own interpolation rules. The choice therefore depends on whether zero‑setup convenience or statistical robustness is required for the target Julia version.
Which tooling combination gives reliable steady‑state timings for a given Julia 1.x release? How should compile‑time percentages be interpreted when comparing @time output across versions? When does the added dependency of BenchmarkTools become justified over the stdlib approach?