Keras–TensorFlow concurrency: Can one shared model serve requests predictably?

A service built with Keras 3 on the TensorFlow backend needs to determine whether a single shared model instance can handle concurrent inference requests without unpredictable tail latency. The integration boundary matters because Keras delegates execution to the active backend, while the TensorFlow runtime controls graph execution, thread pools, and accelerator resource scheduling. No Keras-level contract establishes bounded latency for concurrent predict calls.

The target is TensorFlow 2.x with Keras 3, but equivalent deployments may use the JAX or PyTorch backend, where the same sharing pattern may behave differently. Thread configuration, eager versus compiled execution, GPU memory growth, and request-handler overhead can confound the result. A useful evaluation must compare sequential and concurrent latency for the same instance, including p50 and p99, while distinguishing backend contention from application scheduling.

Which Keras 3 and TensorFlow versions should define the supported baseline? What benchmark can distinguish backend-level contention from request-handler overhead before choosing a shared-instance deployment model?