Karma JUnit reporter alert filtering for CI/CD noise reduction
0 reputation · 24 Mar 2020, 19:06 UTC
0 reputation · 24 Mar 2020, 19:06 UTC
Currently, using failOnTests: false prevents the CI process from exiting with an error code, but the JUnit reporter still captures all test states in the resulting XML file. This results in CI tools parsing and displaying every skipped case, making it difficult to distinguish between critical regressions and intentional test deactivation.
How can the karma-junit-reporter be configured to exclude specific test statuses like 'skipped' from the XML output? Is there a native configuration property to limit the reporter to only 'failed' events without writing a full custom JavaScript reporter class?
35275 reputation · 25 Mar 2020, 01:41 UTC
There is no native property that limits the reporter to only failed events. However, karma-junit-reporter provides a filter option (in recent versions) that lets you exclude tests from the generated XML based on any condition, including skipped status. This is the intended way to achieve what you need without writing a custom reporter.
In your karma.conf.js, add a filter function to the junitReporter configuration. The function receives a test result object (with fields like success, skipped, suite, description) and returns true to include the test in the XML, or false to exclude it.
// karma.conf.js
module.exports = function(config) {
config.set({
reporters: ['junit'],
junitReporter: {
outputDir: 'test-results',
outputFile: 'junit.xml',
filter: (test) => !test.skipped // Exclude skipped tests
}
});
};To include only failures (and exclude skipped and passing tests), use:
filter: (test) => !test.success && !test.skippedNote that failOnTests: false only prevents the CI process from exiting with a non-zero code; it does not affect what the reporter writes to XML.
If the XML also contains browser console messages (in system-out/system-err) that contribute to alert fatigue, you can disable console capture globally with:
client: {
captureConsole: false
}This removes all console output from the JUnit report, which may be acceptable if you rely on other sources for debugging.
The filter option is not present in all versions of karma-junit-reporter. Verify your installed version with npm list karma-junit-reporter and check its README. After adding the filter, run a test that includes skipped cases and inspect the generated XML to confirm they are absent.
Which version of karma-junit-reporter are you currently using? This determines whether the filter option is available. If you're on an older version without it, the alternative is to post-process the XML in your CI pipeline (e.g., with a script) to strip entries where skipped="true".
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 25 Mar 2020, 04:46 UTC
While filtering skipped tests reduces CI noise, it is important to verify how your specific CI parser handles the resulting XML. Some JUnit parsers expect a consistent count between the total tests executed and the sum of passed, failed, and skipped cases defined in the <testsuite> attributes.
tests attribute in the suite header remains unchanged while individual <testcase> elements are removed via a filter, some tools may flag the report as malformed.Before deploying this to a production pipeline, verify the filtered output against your CI tool's JUnit plugin to ensure that removing skipped entries doesn't inadvertently trigger a "Report Parsing Error" or miscalculate the overall success percentage.