Choosing Between nbconvert --execute and Papermill for Automated Jupyter Notebook Runs

Decision and Constraints

You need to run Jupyter notebooks unattended—in CI pipelines, scheduled jobs, or orchestration tools—and you may want to inject different parameter values for each run without editing the notebook source. The two built‑in, supported ways to execute a notebook headlessly are:

1. jupyter nbconvert --execute (part of the Jupyter core)
2. papermill (a Netflix‑originated tool now maintained in the nteract ecosystem)

Both execute arbitrary code with the runner’s permissions, so you must trust the notebook source or run it in a sandboxed environment.

Option Comparison

Feature	nbconvert --execute	Papermill
Runtime dependency	None beyond a standard Jupyter installation	Requires papermill (and its nbclient dependency)
Parameter injection	No native support; you must read environment variables or edit cells	Yes – tag a cell with parameters metadata and override via CLI -p or Python API
Output artifact	Executed notebook (same format) – can also convert to HTML, script, etc.	Executed notebook with a record of the overridden parameters
Error handling	Aborts on first exception; raises a generic NbConvertError	Raises PapermillExecutionError that includes the failing cell index and still writes a partially executed notebook for debugging
Typical use	Simple re‑execution or format conversion when parameters are not needed	Parameterized runs, CI/CD, orchestration (Airflow, Dagster, cron)

Trade‑offs

• Simplicity vs. functionality: nbconvert is already present; if you only need to re‑run a notebook without changing inputs, it adds no extra dependency.
• Parameter management: Papermill’s parameters cell keeps the notebook runnable interactively (defaults stay intact) while allowing clean overrides from the outside.
• Debuggability: Papermill writes the partially executed notebook on failure, making it easier to inspect state; nbconvert stops and leaves you with only the error trace.
• Maintenance: Papermill releases are less frequent than Jupyter core; pinning a version in CI is recommended to avoid surprises with nbclient upgrades.

Concrete Implementation and Validation

Sample notebook layout

Create a notebook demo.ipynb with two cells:

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": ["parameters"]
      },
      "source": ["threshold = 0.5\n"],
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "source": ["result = threshold * 2\nprint(f'Result: {result}')"],
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}

The first cell is tagged parameters and provides a default threshold. The second cell computes a simple derivative.

Running with Papermill

Assuming you have a CI runner or local shell with papermill installed (pip install papermill):

# Run with default threshold (0.5)
papermill demo.ipynb output_default.ipynb
# Run with overridden threshold
papermill demo.ipynb output_override.ipynb -p threshold 0.9

Where to run: Any directory where you have read access to demo.ipynb and write access for the output notebooks. Permissions: The notebook code executes with the same user privileges as the runner; ensure the environment is trusted or sandboxed.

Expected check: Open output_default.ipynb and output_override.ipynb with jupyter nbconvert --to stdout --execute output_*.ipynb or view them in Jupyter; the printed result should be Result: 1.0 for the default and Result: 1.8 for the override.

Risk: If the notebook contains malicious code, it will run with the runner’s rights. Never execute untrusted notebooks in a privileged CI job.

Running with nbconvert --execute

jupyter nbconvert --to notebook --execute demo.ipynb --output executed_default.ipynb

This command re‑executes the notebook but ignores any parameter‑injection mechanism; it will use the default threshold = 0.5 from the parameters cell, producing the same output as output_default.ipynb. No -p flag exists, so you cannot change the value without editing the notebook.

Validation via a Python test

To prove that Papermill actually changed the notebook’s behavior, you can read the output notebooks with nbformat and assert on the cell outputs:

import nbformat

def get_last_output(path):
    nb = nbformat.read(path, as_version=4)
    # Assume the last cell contains the print output
    last_cell = nb.cells[-1]
    for out in last_cell.get('outputs', []):
        if out.get('output_type') == 'stream':
            return out.get('text', '').strip()
    return ''

assert get_last_output('output_default.ipynb') == 'Result: 1.0'
assert get_last_output('output_override.ipynb') == 'Result: 1.8'
# Ensure they differ
assert get_last_output('output_default.ipynb') != get_last_output('output_override.ipynb')

Run this script in the same environment where the notebooks were executed; it will raise an AssertionError if the outputs are not as expected.

Failure case demonstration

Introduce a deliberate exception in the second cell (e.g., 1/0) and run Papermill:

papermill demo.ipynb output_fail.ipynb -p threshold 0.9

Papermill will raise PapermillExecutionError that includes the failing cell index, and output_fail.ipynb will contain the outputs from the first (parameters) cell and the traceback in the second cell. The same failure with nbconvert --execute yields a generic NbConvertError and does not preserve the partially executed notebook.

Limitations and Practical Verification

• Both tools execute notebooks sequentially; hidden state (e.g., imports that mutate globals) or reliance on execution order can still break automation.
• Papermill inserts a new cell after the tagged parameters cell; if your notebook logic depends on the parameters cell being the very first cell, verify that the inserted cell does not shift indices you rely on.
• To verify that parameterization worked, compare the output notebooks programmatically (as shown) or diff the JSON cells that contain the computed results.
• If you need both execution and format conversion, a common pattern is to run with Papermill (or nbconvert) to get an executed notebook, then use nbconvert --to html on that artifact for reporting.