Hey folks, been diving deep into GitHub Advanced Security this past month, specifically the CodeQL scanning performance. I was curious about how scan times scale with project size, so I ran a little benchmark across a few of our internal repos.
I measured the full CodeQL analysis runtime (setup, build, analysis) for each project, using the default actions workflow on GitHub-hosted runners. Here's what I found for a mix of JavaScript/TypeScript and Python projects:
* **Small project (~10k LOC):** Scan completed in about 4-5 minutes. Pretty snappy!
* **Medium project (~50k LOC):** Scan time jumped to 12-15 minutes. The increase felt more than linear.
* **Large monorepo (~200k LOC):** This one took 28-32 minutes. The setup and compilation phases started to dominate here.
The key takeaway for me was that after ~100k lines, the time isn't just about lines of code—it's heavily influenced by the complexity of the dependency graph and the build process. A simple 200k LOC Python service was faster than a 150k LOC TypeScript app with a complex Webpack build.
Has anyone else done similar timing tests? I'm wondering if tweaking the `paths` or `paths-ignore` configuration in the workflow to focus scans on changed directories could shave off significant time for larger projects. Also curious if self-hosted runners with more resources make a dramatic difference.
— Kevin
Benchmark or bust
I'm a dev at a small fintech shop, we run CodeQL scans on our main web app and a few microservices in production.
Based on my setup:
1. **Analysis time scaling:** My experience matches yours. Our 75k LOC Python service scans in 18-20 minutes. The 40k LOC React frontend takes 22-25 minutes because the build step adds significant overhead.
2. **Cost control:** The GitHub-hosted runner time is the real cost. For us, moving from the default 'large' to running on a self-hosted runner with more RAM shaved about 30% off the large project scan time. That saved real money over a month of daily scans.
3. **Configuration impact:** The `paths-ignore` setting for things like `node_modules` and generated docs is a must. Not using it added 7-8 minutes to our frontend scan. Also, setting `threads: 0` to use all vCPUs on the runner helped more on Python than JavaScript.
4. **The real limitation:** You hit it. For large projects, the initial database creation (the "setup" phase) dominates. Once that's cached, subsequent scans on the same codebase with small diffs are way faster, but any significant change triggers a near-full rebuild.
I'd stick with CodeQL for now if you're already on GitHub and your scans are under 30 minutes. The security findings are integrated and the cost is bundled if you already pay for Advanced Security. If your scans are creeping past 45 minutes regularly, tell us if you can use self-hosted runners and whether you mostly need SAST or secret scanning.