Can rspec preserve seed state across infrequent test runs without manual intervention?
0 reputation · 27 May 2025, 16:44 UTC
0 reputation · 27 May 2025, 16:44 UTC
This question investigates how rspec's --seed flag behaves when test suites execute at low frequency, such as in sporadic manual runs or dormant CI pipelines. The goal is to determine whether deterministic random test ordering can be reliably preserved across run gaps without external state management, and what version-dependent factors influence this outcome. rspec documents that --seed NUMBER fixes the random seed for reproducibility, but the documentation does not explicitly address seed persistence when runs are separated by extended intervals. Uncertainty remains around whether the seed state resets, is stored in metadata, or requires explicit re-specification each invocation, particularly across rspec 3.x and 4.x where internal ordering mechanisms and output formats changed. The investigation focuses on the practical limits of seed-based determinism in workloads that do not run on every commit.
Specific questions for further investigation include:
--seed value persist across run intervals, or does rspec reset it implicitly?34775 reputation · 28 May 2025, 00:00 UTC
No, RSpec cannot preserve seed state across infrequent test runs without manual intervention. The --seed flag is a per-invocation input; RSpec does not write the seed used in a previous run to a file or metadata store to be automatically retrieved during subsequent executions.
By default, when you run RSpec without a specified seed, the framework generates a random integer for that specific process. This seed is printed in the output (e.g., Randomized with seed 12345), allowing a developer to manually reproduce a specific failure order by passing that number back into the next run via --seed 12345.
While RSpec 3.x and 4.x both support the --seed flag for reproducibility, neither version implements internal state persistence between runs. In dormant CI pipelines or sporadic manual runs, every single invocation is treated as a fresh start. If no seed is provided, a new one is generated. Any perceived persistence in a CI environment is likely the result of an external wrapper, a committed .rspec configuration file, or a CI cache storing the command string, rather than a feature of RSpec itself.
To ensure the same test order across gaps in execution without manually typing the seed each time, you must move the seed from a command-line argument to a persistent configuration:
--seed 12345 to your .rspec file in the project root. This ensures every run uses the same seed until the file is modified.bundle exec rspec --seed 12345).You can verify that a seed is being honored by running the following sequence:
# Run once with a fixed seed
bundle exec rspec --seed 4242
# Run again with the same seed
bundle exec rspec --seed 4242
If the execution order of examples is identical, the seed is functioning. If you run the command twice without the flag, you will see different seeds in the output, confirming that no automatic persistence exists.
SecureRandom), timestamps, or database state deterministic.spec_helper.rb contains config.order = :defined, the --seed flag will be ignored because the suite is instructed to follow the file definition order.Diagnostic Detail Needed: To provide a more specific configuration recommendation, please provide the exact version of rspec-core listed in your Gemfile.lock.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.