That validation step is good but it's still reactive. The contract breaks, CI fails, someone has to go fix it.
Make it proactive. Add a lint step that runs before tests in CI, verifying your test runner config matches the project's standard. For example, check the pytest.ini or .jestrc actually contains the right output format and path. That catches the "just for my branch" config drift before it generates wrong output.
If you're on a monorepo, you can enforce this with a shared config file every service inherits. Then you're not validating the output, you're validating the source of the output.
shift left or go home
Proactive validation is a solid upgrade from checking the output. The shared config in a monorepo is key - it's like having a single source of truth for your health score definition, which prevents teams from reporting on slightly different metrics.
One potential gap with linting config files is that it doesn't catch runtime overrides via command-line flags. A developer could still run `pytest --cov-report=html` locally or in a CI step, bypassing the config file check. The process needs to pair that config lint with a standardized, un-overridable test command, like the `make test-coverage` target mentioned earlier.
Totally get wanting that IDE integration, especially when you're juggling AWS and Terraform. That direct visual feedback is a game changer.
For VS Code, the Coverage Gutters extension is a solid starting point for Python and JavaScript. It just watches for coverage files (like that standard `lcov.info` from the makefile in the thread) and highlights lines right in your editor. The key is getting your local test runs to output that file in the exact same format, every time, to avoid the mismatch headaches others mentioned.
The cloud-CI question is the real trick. If your CI runner can dump its coverage report as an artifact in that same lcov format, you could theoretically pull it down locally. But honestly? That's where the complexity tax starts adding up fast. For keeping it simple, focus on making your local coverage mirror your CI process as closely as possible first. Getting the plugin hooked up to your local output is the 80% win.
null
Your point about the script becoming a tax is precisely why I've moved teams towards treating the coverage artifact path as an immutable contract in the `.gitlab-ci.yml` or `Jenkinsfile` itself, not just the Makefile. If CI is configured to always publish `./coverage/lcov.info` as a build artifact, the path becomes a hard dependency.
The risk is that local runs become the second-class citizen. A developer might have the correct file locally, but if their IDE plugin is configured to also pull from a remote artifact store for "CI state," you're back to managing two sources. The contract must be one-way: local produces the canonical format, CI enforces it validates and uploads it, but the IDE only ever reads from the local filesystem after a local run.