Short-lived OIDC tokens versus long-lived service account tokens for Kubeflow least-privilege authentication

Direct Answer

For Kubeflow service accounts on Kubernetes 1.24+, use bound service account tokens with a 1-hour TTL (the default) projected via the TokenRequest API, not legacy non-expiring secrets. For user-facing workloads (notebooks, pipelines), pair short-lived OIDC access tokens (5–60 minutes from your IdP) with a sidecar refresh mechanism (oauth2-proxy, kubelogin) so users don't re-authenticate. Scope IAP at the GCP project/load-balancer level and enforce namespace isolation with Kubernetes NetworkPolicies plus namespace-scoped RoleBindings—never ClusterRoleBindings. Automatic revocation on role change is not native: existing bound tokens remain valid until expiry. Implement a Kyverno policy or admission webhook that deletes or cordons pods using a service account when its RoleBindings change.

Token Lifetime Trade-offs

Service Accounts (Workload Identity)

Kubernetes 1.24+ issues bound tokens with configurable expiration (default 1 hour) via the TokenRequest API. Mount these via projected volumes (tokenVolumeProjection in the pod spec) with an explicit expirationSeconds (e.g., 3600). Avoid mounting the legacy secret token that never expires. For Kubeflow Pipelines and Katib, use an init container that writes a short-lived OIDC token to a shared emptyDir volume, with a sidecar refreshing before expiry.

User-Facing OIDC Tokens

Enterprise IdPs (Okta, Azure AD, Keycloak) typically issue access tokens valid for 5–60 minutes with refresh tokens lasting days. Configure a sidecar (oauth2-proxy or kubelogin) to exchange the refresh token automatically. This keeps the access token short-lived without user interaction. Assumption: your IdP supports refresh tokens and the sidecar can reach the token endpoint from the pod network.

Scoping IAP to Prevent Cross-Namespace Escalation

IAP policies apply at the GCP project or load-balancer level, not at the Kubernetes namespace level. A user granted IAP access can reach the Kubeflow ingress, but Kubernetes RBAC still governs API access. To prevent escalation:

1. Use namespace-scoped RoleBindings only—never ClusterRoleBindings for Kubeflow users.
2. Apply NetworkPolicies that deny cross-namespace pod-to-pod traffic by default; allow only explicit ingress/egress rules per namespace.
3. Configure the RBAC authorization webhook (e.g., kubernetes-rbac-authorizer) to honor only groups prefixed with kubeflow: from the OIDC groups claim.

Caution: Misconfigured NetworkPolicies can allow pod-to-pod bypass even when RBAC denies access. Test with kubectl auth can-i <verb> <resource> --as=<user> --as-group=<group> across namespaces.

Automatic Credential Revocation on Role Change

Kubernetes RBAC changes take effect immediately for new TokenRequest calls. Existing bound tokens remain valid until their exp claim. No built-in mechanism revokes them early. Options:

• Kyverno policy (or custom admission controller) that watches RoleBinding/ClusterRoleBinding changes and deletes pods referencing the affected service account. For stateful workloads (notebook servers), prefer cordon/drain over deletion.
• Short TTL + high refresh frequency: set bound token TTL to 15–30 minutes and ensure sidecars refresh aggressively. This bounds the window of stale access.
• TokenReview API: build a controller that calls TokenReview on each request (adds latency) or periodically audits active tokens.

Verification Steps

# 1. Confirm Kubernetes version supports bound tokens (1.24+)
kubectl version

# 2. Inspect service account token projection
kubectl get sa <sa-name> -n <ns> -o yaml
# Look for tokenVolumeProjection in pod specs using this SA

# 3. Test IAP + RBAC interaction
kubectl auth can-i get pods --as=[contact removed] --as-group=kubeflow:team-a -n other-namespace

# 4. Validate OIDC token lifetime
echo <id-token> | base64 -d | jq .exp
# Compare with IdP policy

# 5. Verify revocation behavior
# Modify a RoleBinding, then attempt API call with existing token before/after TTL expiry
kubectl auth can-i ... --token=<bound-token>

One Missing Diagnostic Detail

Which Kubeflow distribution and Kubernetes version are you running? (Vanilla, GKE Kubeflow, AWS Kubeflow, Azure Kubeflow, or a managed service like Vertex AI Pipelines?) The default authn/authz stack—IAP, Istio, OIDC provider, and whether workload identity is enabled—changes the recommended token strategy and the exact commands to verify it.