As a cost analyst, my primary lens for evaluating any tool is its total operational expenditure, particularly within complex, resource-intensive environments. I’ve been tasked with providing a financial and operational assessment of our current application security tooling, and Checkmarx is a candidate for our primary code scanning solution. However, our codebase is a single, large monorepo housing over 200 microservices and libraries, with a combined history exceeding five years.
The marketing materials and sales engineering demos are, unsurprisingly, optimized for greenfield repositories. I am seeking concrete, production-tested insights from teams operating at a similar scale. My preliminary analysis raises several cost and performance concerns that standard pilots fail to capture:
* **Scan Initialization & Incremental Analysis:** How does the engine handle the first full scan of a massive codebase? Is the time-to-first-result measured in hours or days? More critically, for subsequent scans, does the incremental analysis truly function as advertised, or do merge conflicts and deep histories force frequent near-full re-scans, consuming excessive compute cycles?
* **Infrastructure & Hidden Operational Costs:**
* For the self-managed (on-premises) deployment, what are the true resource requirements for the management console and scan engines? I'm particularly interested in sustained CPU/memory consumption under load, not the minimum specs for a proof-of-concept.
* For the SaaS offering, is the pricing model based on lines of code, scans, or concurrent users? Have you encountered steep nonlinear cost increases as the monorepo grew? Are there data egress or storage fees for retaining historical scan data?
* **Pipeline Integration Overhead:** In a monorepo with high commit frequency, can the scanner be integrated into CI gateways for pull requests without becoming the critical path bottleneck? What is the average scan latency for a typical diff, and what compute resources (e.g., powerful CI runners) are required to keep it under, say, ten minutes?
* **Noise-to-Signal Ratio & Triage Costs:** The financial impact isn't just license fees; it's the engineering hours spent triaging findings. In a large, polyglot monorepo, how effective is the out-of-the-box configuration at suppressing false positives or irrelevant issues in legacy components? Have you had to invest heavily in creating and maintaining custom query filters or rulesets?
My objective is to build a realistic TCO model that includes not just the licensing line item, but the associated cloud/compute costs, storage, and the often-overlooked productivity tax on development teams. Any data points on your scaling journey, performance tuning, or unexpected cost drivers would be invaluable.
-- Liam
Always check the data transfer costs.
Good to see a focus on operational costs from the start - that's often where the real pain points emerge.
On your first question about scan initialization and incremental analysis: in my experience with large legacy repos, the initial full scan is a multi-day event that requires significant resource planning. The incremental scans *can* work well for routine commits, but you're right to be skeptical. Major merges or refactoring across shared libraries often triggered what felt like partial re-scans that still took hours. The compute cost of those "in-between" scans added up.
I'd suggest asking them for concrete metrics on scan time variance based on change type, not just best-case scenarios. The delta between scanning a single service change versus a core library update is massive.
Stay factual, stay helpful.
Multi-day initial scans? Sounds about right. The real joke is the "partial re-scans" you mentioned. With a monorepo, touching a shared lib can light up the whole dependency graph. Their "incremental" logic often fails to grasp that, and you end up paying for a 70% scan anyway. Asking for variance metrics is smart, but good luck getting a straight answer. Their data tends to vanish when the sales cycle ends. 😏
Just my two cents.