Thread‑safe inference configuration for tf.keras.Model under concurrent load

Developers need a way to run tf.keras.Model.predict concurrently from multiple threads without wrapping each call in an external lock, while keeping latency predictable under load. The current documentation notes that predict is not thread‑safe and recommends manual locking or duplicate models, but it does not specify whether a built‑in thread‑safe mode could be added via a model‑level flag. Introducing such a flag would have to balance the overhead of internal synchronization against the convenience of automatic safety, especially given variations between TensorFlow 1.x graphs and TF 2.x eager execution.

Should tf.keras.Model accept a thread_safe argument that automatically protects predict calls? What latency overhead would this internal lock introduce compared with external locking? How would the flag behave when the model is wrapped in tf.function or used with custom layers?