Default X-Pack Security in Elasticsearch 8.0 and Integration Test Authentication Requirements

Answer the question first

Use the auto‑generated elastic password and the node’s CA certificate (or the printed SHA‑256 fingerprint) to authenticate your test client. This keeps the cluster in the same secured state as production while avoiding the need for real production secrets. If you want to avoid the password reset step on every run, create an API key once per test environment and reuse it. Disabling xpack.security.enabled removes TLS, authentication, and audit logging, so any test that expects 401/403 responses or security headers will no longer exercise those paths.

Why this works

• The first‑boot bootstrap of 8.x generates a CA, node TLS certs, enables HTTPS, and sets a random password for the built‑in elastic superuser.
• Clients can trust the auto‑generated CA by pointing to config/certs/http_ca.crt or by using the SHA‑256 fingerprint printed in the startup log.
• Using the elastic password is safe for CI because it is unique to the node and not shared with production.
• API keys or service‑account tokens give fine‑grained privilege control and can be created via the REST API once per test environment.

Practical steps for a local test run

1. Start a fresh 8.x node (e.g., with Docker or Testcontainers). The log will contain:

   Elastic password: <random>
   CA fingerprint: <sha256>
   HTTPS endpoint: https://localhost:9200
   

2. Extract the password and CA fingerprint (or copy config/certs/http_ca.crt if you run the node locally).

3. Configure your test client:

   client = Elasticsearch(
       hosts=[{"host": "localhost", "port": 9200}],
       http_auth=("elastic", <password>),
       ssl_cafile="config/certs/http_ca.crt",  # or ssl_show_warn=False + ca_cert_fingerprint
   )
   

4. If you prefer an API key, run:

   curl -X POST "https://localhost:9200/_security/api_key" \
        -H "Content-Type: application/json" \
        -u elastic:<password> \
        --cacert config/certs/http_ca.crt \
        -d '{"name": "test-key", "role_descriptors": {}}'
   

   Use the returned api_key in your client’s api_key parameter.

5. Run your integration tests. Verify that a request without credentials returns 401 and that the client correctly handles TLS handshake failures when the CA is invalid.

What disabling xpack.security.enabled does

• Turns off HTTPS on the HTTP layer – all traffic becomes plain HTTP.
• Removes authentication and authorization checks; every request succeeds regardless of credentials.
• Audit logging is disabled, so you cannot validate that your client emits the correct audit events.
• Any test that asserts on security‑related response headers (e.g., WWW-Authenticate) will always fail or be meaningless.

When to use a test user instead of elastic

If your code path includes role‑based access checks, create a dedicated test user with the minimal required roles and use its credentials in the same way as the elastic password. This will surface permission errors that the superuser would otherwise hide.

Final recommendation

Keep security enabled, authenticate as the generated elastic user or a pre‑created API key, and trust the node’s CA. Avoid disabling security unless you are explicitly testing the “no‑security” path. This approach preserves TLS, authentication, and audit logging while keeping the test environment free of production secrets.

Missing diagnostic detail

Do you run your tests against a single-node Docker container or a multi‑node cluster? The bootstrap process differs for multi‑node setups, affecting how you retrieve the password and CA certificate.