Choosing between torch.save file fixtures and in‑memory tensor factories for credential‑free PyTorch tests

Answer to the Core Questions

1. Overall test runtime for 500 unit tests

• When tensors are large (e.g., > 100 MiB) or models require heavy preprocessing, torch.save fixtures reduce runtime by avoiding repeated construction. Typical savings are 30‑70 % for such cases.
• For small to medium tensors (< 10 MiB) or when tests frequently mutate data, in‑memory factories are faster because they skip disk I/O. Runtime savings are usually < 10 %.

2. Peak memory with multiple workers

• torch.save fixtures keep a single copy on disk; each worker loads the same file into memory, so peak memory grows linearly with the number of workers times the tensor size.
• In‑memory factories allocate fresh tensors per worker, but each allocation is independent; peak memory is roughly the same as a single worker plus overhead for parallel execution. For 10 workers, memory can be 10× the per‑worker size.

3. Version‑compatibility safeguards

• Prefer torch.save only when the fixture was created with the same major/minor PyTorch version that the CI uses. Add a pytest.mark.requires_torch_version("1.13") marker or store the version in the file name.
• Use torch.jit.save for scripted models to reduce binary size and improve portability.
• Keep a small requirements.txt that pins the PyTorch version used to generate fixtures.

Recommended Strategy for Your CI Pipeline

1. Profile a sample test suite. Run the 500‑test suite once with in‑memory factories and once with file fixtures, recording total runtime and max‑rss per worker.
2. Set a threshold. If runtime improvement > 20 % and peak memory < 2 GiB, adopt file fixtures for that test group.
3. Automate cleanup. Store fixtures in tests/fixtures/ and delete them in a pre‑commit hook or CI job that runs git clean -fdX tests/fixtures/ after tests finish.
4. Version guard. Add a small helper function:

   def load_fixture(path: str, torch_version: str):
       assert torch.__version__ == torch_version, "Fixture incompatible"
       return torch.load(path)
   

5. Use relative paths. Reference fixtures via Path(__file__).parent / "fixtures" / "model.pt" to avoid hard‑coded locations.

Missing Diagnostic Detail

To fine‑tune the recommendation, please provide the typical size of the tensors or models used in your tests. This will help decide whether the file‑based approach offers a measurable benefit over in‑memory factories.

Quick Reference Table

Aspect	File Fixtures	In‑Memory Factories
Runtime (single worker)	↓ for large data	↑ for large data
Peak memory (10 workers)	10× tensor size	≈10× tensor size + overhead
Reproducibility	High (deterministic)	High (seeded RNG)
Version safety	Requires same PyTorch minor	No external dependency
Disk usage	High for many fixtures	Zero