Android Studio Flamingo: Skip production credentials toggle behavior for unit test configurations

In Android Studio Flamingo (2022.2.1), the "Skip production credentials" toggle does not automatically affect unit test run configurations. This feature is designed to substitute placeholders in the merged manifest of the active build variant (typically debug). Because unit tests execute on a local JVM via the JUnit runner rather than on a device using the merged manifest, they do not interact with the manifest-substitution logic applied by the IDE.

Why the toggle doesn't cover tests

The toggle primarily targets the build process for the APK's manifest. Unit tests, however, often pull values directly from compiled classes or resource files. If your production keys are injected via BuildConfigField or hardcoded in the build variant, the test JVM will see the original values defined in your build script unless explicitly overridden for the test task.

Recommended: substitution without a custom testBuildType

If you want to avoid creating a new testBuildType, the most reliable way to inject test-specific credentials is through Gradle properties or environment variables. This allows you to keep your production configuration clean while swapping values during execution.

1. Use System Properties: Configure your build.gradle to check for a specific property during the test task:

   tasks.withType(Test).configureEach {
       val testKey = project.findProperty("testApiKey") ?: "placeholder_key"
       systemProperty("API_KEY", testKey)
   }
   

2. Access in Tests: In your JUnit code, retrieve the value instead of using the production constant:

   val apiKey = System.getProperty("API_KEY")

3. Run with Flags: Execute your tests from the terminal or a custom task configuration by passing the key:

   ./gradlew testUnitTest -PtestApiKey=my_test_key_123

Drawbacks of extending the IDE toggle

Extending the IDE-level toggle to cover test JVMs presents several risks:

• CI Inconsistency: Most CI environments rely on standard Gradle tasks. If the IDE handles the substitution locally via a toggle, the tests might pass locally but fail in CI (or expose real keys) there.
• State Confusion: Automatically swapping values based on an IDE state can lead to "magic" behavior where the developer is unsure why a test is using a value not defined in the source code.
• Performance Overhead: Forcing the IDE to intercept and modify every test-related JVM configuration for substitution increases the complexity of the IDE's synchronization engine.

To provide a more specific code snippet, are your credentials currently managed via manifestPlaceholders, BuildConfigField, or loaded from a local properties file?