A minimal Selenium 4 architecture: explicit waits, disposable sessions, and trust boundaries

Intermittent Selenium failures rarely come from Selenium itself. They come from three engineering decisions made early: how tests synchronize with the page, how driver sessions are created and torn down, and how much the suite trusts whatever the browser loads. This note describes the smallest design that holds up under those three pressures, what it costs, and the conditions that force a bigger one.

Scope and assumptions

• Selenium 4.6 or later, which bundles Selenium Manager for driver resolution. Behavior varies across 4.x minor releases, so confirm details against the changelog for your exact version.
• Selenium 4 speaks the W3C WebDriver protocol end to end. The legacy JSON Wire dialect is gone, so capabilities must use standard keys or vendor-prefixed ones such as goog:chromeOptions.
• Examples use the Python bindings and Chrome; the same structure applies to the Java and C# bindings.
• Local execution only. Grid enters later, as a deliberate design change.

The smallest suitable design

Three decisions carry most of the stability.

1. Explicit waits, and nothing else

Use WebDriverWait with expected conditions or small custom conditions. Do not use fixed sleeps, and do not mix implicit waits with explicit ones — Selenium's own documentation describes that combination as unsupported, with compounded and unpredictable timeouts. Implicit waits stay at their default of zero.

2. Let Selenium Manager resolve drivers

Since 4.6, Selenium downloads a driver matching the installed browser on first use. That removes third-party driver managers and most version-skew failures. It needs outbound network access on first run; behind a corporate proxy, set the standard proxy environment variables and verify behavior against your version's changelog. If CI cannot allow that, pin browser versions and manage drivers explicitly — but treat that as the exception.

3. One disposable session per test

Fresh driver, fresh temporary profile, quit() in teardown regardless of outcome. Shared sessions leak cookies, service workers, and scroll position between tests, and they turn one failure into a cascade.

The shape of a minimal fixture:

# conftest.py — run under pytest. Requires selenium >= 4.6 and a
# locally installed Chrome. Selenium Manager downloads the matching
# driver on first use, which needs outbound network access.
import pytest
from selenium import webdriver

@pytest.fixture
def driver():
    options = webdriver.ChromeOptions()
    options.add_argument("--headless")
    options.add_argument("--window-size=1280,1024")
    # Vendor-prefixed capability, W3C style:
    options.add_experimental_option("prefs", {
        "download.default_directory": "/tmp/selenium-downloads",
    })
    drv = webdriver.Chrome(options=options)
    drv.set_page_load_timeout(30)
    yield drv
    drv.quit()  # runs even when the test fails

And a test that waits on a condition instead of sleeping:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def test_order_row_appears(driver):
    driver.get("https://staging.example.com/orders/1234")  # placeholder URL
    row = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located(
            (By.CSS_SELECTOR, "tr[data-order-id='1234']")))
    assert "paid" in row.text.lower()

presence_of_element_located only guarantees the element is in the DOM. If the next step clicks it, wait for element_to_be_clickable instead — matching the condition to the interaction is what keeps waits honest.

Trust and data boundaries

The driver session is the trust boundary. Everything the browser loads is untrusted input, and the automation can be steered by it:

• Profiles. Never point a test at a real user profile. A session running against your daily browser profile exposes its cookies and logged-in state to whatever the test navigates to. Temporary profiles cost nothing and cap the blast radius.
• Credentials. Test accounts only. Inject secrets at runtime from your CI secret store, never in code or capabilities.
• Downloads. Treat them as hostile files: dedicated download directory, scan before use, never auto-open.
• Page content. page_source and element text are data. Do not paste them into shell commands, queries, or logs without escaping.
• Network exposure. Local sessions bind to the machine running the tests. When you later run a Grid, node ports are reachable over the network — restrict access to the grid subnet.

Operational checks

Three checks keep the design observable:

1. Record capabilities at session start. driver.capabilities includes the browser version; write it into the test report so version skew between environments is visible instead of inferred.
2. Probe session liveness before retrying. A cheap call such as driver.current_url inside a try/except for WebDriverException distinguishes a dead session (restart) from a page problem (retry the step).
3. Capture artifacts on failure. Screenshot plus page source, plus the failing locator and capabilities. This is the primary debugging signal for flaky runs.

def capture_failure(drv, name):
    drv.save_screenshot(f"artifacts/{name}.png")
    with open(f"artifacts/{name}.html", "w", encoding="utf-8") as fh:
        fh.write(drv.page_source)

Run the suite both headless and headed on a schedule. Headless mode is not behaviorally identical to headed for rendering, media, and some input paths, so a suite that only ever runs headless hides environment-specific failures.

Failure modes and their fixes

Symptom	Typical cause	Mitigation
StaleElementReferenceException after a re-render	A WebElement cached across a DOM update	Re-locate inside the wait condition; never store elements across renders
Passes alone, fails under parallel load	Fixed sleeps tuned to a fast machine	Replace sleeps with condition-based waits
Timeouts far longer than configured	Implicit and explicit waits combined	Explicit waits only; leave implicit waits at zero
Sessions fail to start after a browser upgrade	Version skew or legacy capability keys	Let Selenium Manager re-resolve; use W3C keys and vendor prefixes such as goog:chromeOptions
Grid nodes slow or crashing	Unbounded sessions per node	Bound concurrency per node; quit() in teardown, always

When the design must change

• Cross-browser coverage. When one machine can no longer hold the browser matrix, move to Selenium Grid or a hosted provider. You inherit queueing, node management, and a version matrix — budget for them.
• Parallel scale. Grid again, with bounded concurrency per node; resource exhaustion is the failure mode to watch from day one.
• Network interception, emulation, console logs. The standard WebDriver API does not cover these. CDP works but is Chrome-specific and browser-version-specific — it can break on browser upgrades. The BiDi APIs are the standards-track answer but still evolving across 4.x releases. Prefer a standard API wherever one exists, and isolate CDP use in a small, clearly marked layer.

Verifying the result

1. Run a minimal test with WebDriverWait against a page that updates its DOM after load, and confirm it passes with no sleep calls anywhere in the path.
2. Deliberately break a locator and confirm the screenshot and page source artifacts appear.
3. Execute the suite headless and headed and reconcile any differences.
4. Check the Selenium changelog for your exact 4.x version to confirm Selenium Manager behavior and capability names.

This design holds until coverage or scale forces a Grid. The part worth keeping regardless is the artifact discipline: when the next flake arrives, the screenshot, page source, and recorded capabilities turn a morning of guessing into a five-minute diagnosis.