Exporting TensorFlow SavedModel with Custom Serving Signatures for Production

Why Explicit Signatures Matter

When you ship a model to production, the serving system must know exactly which tensors to feed and which to read back. A SavedModel can contain multiple signatures—named entry points that expose a concrete function’s inputs and outputs. Explicit signatures give you:

• Deterministic input shapes and dtypes that the client must match.
• Multiple entry points in a single model (e.g., serving_default, encode, decode).
• Control over preprocessing and post‑processing that lives inside the graph.

Without them, the default signature generated from a Keras model.call may be too permissive or not match the client’s request format.

How TensorFlow Builds a Signature

In TF 2.x, a @tf.function decorator turns a Python callable into a tf.function object. By supplying an input_signature composed of tf.TensorSpec objects, you force the function to accept only tensors of a specific shape, rank, and dtype. When you call get_concrete_function(), TensorFlow compiles that specification into a ConcreteFunction that can be exported.

Example:

import tensorflow as tf

# Simple Keras model
model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(224, 224, 3)),
    tf.keras.layers.Conv2D(32, 3, activation='relu'),
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(10, activation='softmax')
])

# Explicit serving function
@tf.function(
    input_signature=[tf.TensorSpec(shape=[None, 224, 224, 3], dtype=tf.float32)]
)
def serving_fn(x):
    return model(x)

# Export
tf.saved_model.save(
    model,
    'export_dir',
    signatures={'serving_default': serving_fn.get_concrete_function()}
)

Here None allows variable batch size, but all other dimensions are fixed. The exported directory now contains a saved_model.pb and a variables/ folder.

Inspecting the Exported Model

Use the saved_model_cli tool to verify the signature definition:

saved_model_cli show \
    --dir export_dir \
    --tag_set serve \
    --signature_def serving_default

Output will list input and output tensor names, shapes, and dtypes. This is the contract your client must adhere to. If you exported multiple signatures, replace serving_default with the desired name.

Common Mistakes and Limits

• Dynamic shapes beyond batch size: tf.TensorSpec(shape=[None, None, 224, 224, 3]) is illegal. Use tf.RaggedTensorSpec for variable‑length dimensions.
• Custom layers or objects: If you added a custom metric or layer, register it with tf.keras.utils.get_custom_objects() or pass custom_objects to tf.saved_model.load().
• Large models: Protocol buffers are limited to 2 GB. For bigger models, split the graph or use experimental_io_device options.
• Python‑side preprocessing: Anything outside TensorFlow ops is invisible to the exported graph. Move preprocessing into the @tf.function body.

Verifying the Signature in Python

Load the model in a fresh process and call the signature to confirm it works:

loaded = tf.saved_model.load('export_dir')
infer = loaded.signatures['serving_default']
print(infer(tf.zeros([1, 224, 224, 3])))

Ensure the output tensor names match those reported by saved_model_cli. If you see a KeyError, the signature name is wrong or not exported.

Deploying to TensorFlow Serving

Run the official Docker image, mounting the exported directory as a model:

docker run -p 8501:8501 \
    --mount type=bind,source=$(pwd)/export_dir,target=/models/my_model \
    -e MODEL_NAME=my_model \
    tensorflow/serving

Once running, send a request via REST or gRPC. For REST, a minimal curl example:

curl -X POST \
    -d '{"signature_name": "serving_default", "instances": [[0.0, 0.0, ...]]}' \
    -H "Content-Type: application/json" \
    http://localhost:8501/v1/models/my_model:predict

Replace the array with a valid float32 image of shape [1,224,224,3]. A successful response will contain the softmax probabilities.

Version Compatibility Note

SavedModel format versions advance with major TensorFlow releases. A model saved with TF 2.15 may not load on a TF 2.10 server. Always match the TensorFlow Serving version to the training environment or use tf_upgrade_v2 to migrate.

Bottom Line

Exporting a SavedModel with explicit, concrete signatures gives you a clear contract for production inference, reduces runtime errors, and enables multi‑entry‑point models. Validate with saved_model_cli, test in isolation, and deploy with TensorFlow Serving to get a robust, scalable inference pipeline.