Question
Stub at the Network Boundary vs. Test-Mode Config Inside the Extension
Ash Lane
0 reputation · 25 Oct 2020, 05:21 UTC
32.5K views1
Loading…
0 reputation · 25 Oct 2020, 05:21 UTC
When integration-testing a VS Code extension that calls a remote service, the team must decide where to enforce the credential-free contract. One approach runs a local stub server that asserts the exact request shape the extension sends; the extension points at the stub via a test-scoped endpoint setting. The alternative adds a test-only configuration path inside the extension that bypasses authentication parsing and returns canned responses, activated by a dedicated setting such as extension.testMode.
Both the Extension Test Runner and @vscode/test-electron can launch the extension against either arrangement, but they do not inject secrets. The stub approach keeps the network boundary honest and reproduces the same code path used in production, yet it cannot reveal tenant-level policy errors or real-service latency. The internal test mode exercises configuration handling and authentication parsing logic, but it diverges from the production request flow and may mask serialization bugs.
The constraint is that CI must run with zero production credentials present, and the test must fail loudly if the extension accidentally falls back to a default endpoint.
35775 reputation · 25 Oct 2020, 05:47 UTC
The approach utilizing a local stub server at the network boundary provides higher confidence that the production request contract is honored without requiring production credentials.
The stub method forces the extension to execute the exact network code path used in production, including serialization and request shaping. The internal test mode, conversely, bypasses authentication parsing entirely, testing configuration logic rather than the complete production network interaction. This divergence risks masking subtle serialization errors that would manifest against a real service.
To refine the recommendation, confirm whether the extension has a mechanism to explicitly log or throw an error when it cannot resolve the configured remote endpoint, rather than defaulting to an unconfigured endpoint.
Use comments to ask for clarification. Post a solution as an answer.
35,775 reputation · 25 Oct 2020, 11:13 UTC
To satisfy the zero-credential CI constraint and prevent silent fallback, the extension should treat an undefined endpoint as a hard activation error rather than a runtime warning. In activate(), validate workspace.getConfiguration('extension').get('endpoint') and throw if falsy—this makes the failure deterministic and testable without network mocks. The integration test then only needs to launch the extension in a clean workspace (no .vscode/settings.json, no env var) and assert that vscode.extensions.getExtension('publisher.name')?.exports remains undefined or that a known command rejects with a configuration error. Pair this with a stub-server test that injects the endpoint via process.env.EXTENSION_ENDPOINT=http://127.0.0.1:port (supported by both @vscode/test-electron and the Extension Test Runner) and asserts the exact request shape on the stub. Finally, add a packaging script that greps the bundled out/ for any production hostname literal; if found, fail the build. This three-layer approach—activation guard, stub contract test, and static fallback scan—covers the CI requirement without shipping test-mode code paths.
vscode