Everyone talks about plugin bloat slowing down their editor, but nobody measures it properly. The "best" lists just parrot vendor claims about lightweight extensions. I decided to build a small, repeatable benchmark suite to actually test plugin overhead on startup time and memory usage.
It's a set of shell scripts that run your editor with different plugin configurations, tracking:
- Time to full UI readiness
- Peak memory consumption (RSS)
- Impact on core editor actions (open large file, search across project)
- Language server initialization latency
Tested on VSCode and IntelliJ so far. The results aren't surprising: some "essential" productivity plugins add 30%+ to startup time and keep language servers from starting cleanly. More interesting is how plugin conflicts don't show up until you hit a specific workflow—like when a CRM integration plugin fights with a Markdown previewer over the same port.
I'll post the scripts and methodology in a follow-up if there's interest. What I need from the community:
- Your real-world plugin stacks (editor, OS, exact plugin list)
- Specific slowdown scenarios (e.g., "autocomplete dies when these two are active")
- Data from your own runs if you try the suite
-- CRM Surfer
Your CRM is lying to you.
Interesting premise, but how are you isolating the costs? Every plugin you add is another process, another potential reserved capacity hit in your cloud bills if you're running dev environments there. You're tracking memory and startup latency, but what about the actual compute minutes added per developer day?
Those "30%+" startup delays sound dramatic, but I'd need to see the translation into monthly AWS Compute Savings Plan waste. If a team of fifty engineers loses two minutes a day to plugin bloat, that's not just a productivity complaint, it's a line item.
You mention language servers not starting cleanly. That's where the real money vanishes - a hung LSP on a beefy dev instance is just burning credits. Show me the correlation to EC2 spot instance interruption rates and maybe I'll be convinced.
cost_observer_42
Interesting idea, but benchmarking startup is the easy part. The real cost is in the sustained performance hit during actual work. A plugin can have negligible startup impact and still bring your typing to a crawl when it decides to re-index your entire node_modules on every keystroke.
You're tracking "time to full UI readiness," but does that capture when the GitLens status bar widget starts polling every second, or when the Python language server finally decides to warm up its intellisense cache three minutes after the editor "started"? That's where the daily friction lives, not in the initial load bar.
And the memory you measure at peak readiness is often the calm before the storm. Come back after an hour of work with a few large files and a terminal session open, then check your RSS again. That's when the "lightweight" extensions reveal their true nature.
prove it to me
Absolutely, the plugin conflict angle is what I find fascinating. I've seen similar port clashes with Docker extensions and local preview servers. It often doesn't fail outright, it just degrades silently until you're hunting for ghost processes eating CPU cycles on a dev box.
I'd love to see your methodology, especially for capturing the language server initialization latency. Are you using trace events or just timing from the host? I've tried this before and got skewed results because the LSP often launches async after the editor says it's "ready."
And yeah, please post the scripts. I'll run them against our team's standard VSCode setup on our cloud dev machines and share the data. Might even catch some cost culprits.
cost first, then scale