GOPROXY caching for private modules: avoiding repeated downloads on a low-traffic service
0 reputation · 22 Nov 2021, 19:28 UTC
0 reputation · 22 Nov 2021, 19:28 UTC
I'm running a low-traffic Go service (Go 1.22 toolchain) that depends on a few modules hosted in a private Git repository. My current setting is the default GOPROXY=https://proxy.golang.org,direct, and GOPRIVATE covers the private repo path.
My understanding is that when the public proxy returns 404/410 for a private path, the toolchain falls back to direct mode, which means every clean build in CI re-downloads the module from the origin instead of hitting a cache. For a workload that rebuilds infrequently but from scratch each time, that adds network cost and latency I'd like to avoid.
What I can't pin down from the official documentation is the intended configuration boundary here: whether a self-hosted or authenticated proxy is expected to cache private modules, how authentication is supplied to it, and what cache invalidation semantics (if any) apply when a private module version is re-tagged or updated. The docs describe fallback behavior but not the caching contract for private paths.
GOPRIVATE + direct fetching the only intended path?34275 reputation · 23 Nov 2021, 04:58 UTC
Your reading of the fallback behavior is correct, and the boundary you're looking for is this: the Go toolchain itself has no authenticated caching path for private modules. GOPRIVATE + direct fetching is the default intended path. If you want proxy caching for private modules, you must run (or subscribe to) a proxy that implements the Go module proxy protocol, handles authentication itself, and is listed in GOPROXY ahead of direct. The public proxy at proxy.golang.org will never cache your private paths — it returns 404/410 and the toolchain falls through.
Supported by the protocol, not built into the toolchain. The distinction matters:
$GOPROXY/<module>/@v/<version>.zip, etc.). Any server implementing it can be a proxy, including one that fronts your private Git host..netrc entry matched to the proxy host, or network-level controls (VPC allow-listing). The Go tool does not forward your Git credentials to a proxy, and GOPRIVATE only controls which paths bypass the public proxy and checksum database — it does not authenticate anything.A typical configuration:
GOPROXY=https://goproxy.internal.example,https://proxy.golang.org,direct
GOPRIVATE=git.example.com/myteam/*
GONOSUMDB=git.example.com/myteam/* # implied by GOPRIVATE, shown for clarityNote that GOPRIVATE paths also skip checksum-database verification, which is what allows a private proxy to serve them without sum.golang.org integration.
There is no invalidation contract, because Go treats module versions as immutable. A proxy caches by module@version; once v1.2.3 is cached, the proxy is entitled to serve that zip forever. If you re-tag v1.2.3 to a different commit, proxies and local module caches that already have the old content will keep serving it — and worse, consumers who fetched before and after the move can end up with different code for the same version. The correct workflow is to publish a new version (v1.2.4 or a pseudo-version) rather than move a tag. If a tag move has already happened, the recovery is to have consumers run go clean -modcache and purge the version from the proxy's storage — treat this as an incident, not a routine operation.
Run the download with verbose tracing:
go mod download -x git.example.com/myteam/modThe output shows the URL each module is fetched from — your proxy's host versus a direct VCS fetch. Two caveats: this only shows the source on a cold cache (modules already in $GOMODCACHE aren't re-fetched), so clear the cache or use a fresh CI workspace for a meaningful check; and -x output is verbose, so grep for the module path. If you run your own proxy, its access logs are the more durable signal — they show whether CI actually consulted the cache before any direct fetch occurred.
One honest consideration: a self-hosted proxy is another service to operate. For a handful of private modules and infrequent clean builds, the simpler alternative is caching $GOMODCACHE between CI runs (most CI systems support keyed caches; key on go.sum). That eliminates repeated origin downloads without any new infrastructure. The private proxy earns its keep when you have many CI jobs, many developers, or want a single audited authentication point for module access.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.