Question
Teleport auth server role cache TTL causing stale authorization after role update
Nia Sage
0 reputation · 05 Nov 2022, 11:26 UTC
111.8K views1
Loading…
0 reputation · 05 Nov 2022, 11:26 UTC
Teleport auth server caches role mappings for users and services to reduce load on backend storage such as DynamoDB or etcd. The cache is populated on first lookup and refreshed only after a configurable TTL, reported as 5 minutes by default in Teleport v13.
Role changes made by an administrator are not reflected in authorization decisions until the cached entry expires or the auth server is reloaded. No automatic cache invalidation occurs on role update events.
This creates an unresolved trade-off between immediate consistency for access revocation and backend load and performance. The goal is to clarify the documented caching behavior and its impact on authorization timing.
What is the propagation window for role additions and removals under the default auth_cache_ttl? Is cache invalidation on role updates part of the design, or is TTL-based expiry the intended behavior? How does the TTL interact with new authorization checks versus existing authenticated sessions?
The propagation window for role additions and removals in Teleport is determined by the auth_cache_ttl setting. Under the default configuration in Teleport v13+, this window is 5 minutes. This means a role update made in the backend may take up to five minutes to be reflected in authorization decisions made by the auth server.
TTL-based expiry is the intended behavior for the Teleport auth server. The system does not perform automatic cache invalidation upon role update events. This design choice is made to protect the backend storage (such as etcd or DynamoDB) from high-frequency read requests that would occur if every authorization check required a fresh database lookup, especially in large-scale environments.
The TTL interacts differently depending on the type of request:
If your security requirements demand faster propagation than the 5-minute window, consider the following actions:
teleportauth_cache_ttl in the auth server configuration. Note that very low values will increase the load on your backend storage.Note: The specific behavior and configuration keys may vary between Teleport Open Source, Enterprise, and Cloud. Verify against your version-specific documentation.
Use comments to ask for clarification. Post a solution as an answer.
4,990 reputation · 05 Nov 2022, 12:53 UTC
Teleport’s auth server rebuilds its role mapping cache only on a time‑to‑live (TTL) basis. The default role_cache_ttl is 5 minutes in v13. Until that TTL expires, the auth server continues to serve the cached role set for all lookup requests.
When a role is added or removed, the change is invisible to the auth server until the next lookup after the TTL. Existing session tokens, however, are immutable: they carry the exact role set that was in effect at the time of token issuance. Consequently, a user who loses a role will still be able to access resources that require that role until the token expires (typically 24 h) or the session is revoked.
# Verify the current TTL
$ tctl auth get --output=yaml | grep role_cache_ttl
# Wait 5 minutes, then retry access
$ tctl auth login && tctl auth get-roles