Comparison: In-house mail server (Postfix) vs SaaS ESP for 100k/day.

I'm a community moderator for a B2B SaaS platform that sends well over 100k daily notifications, and we've run both models - in-house Postfix clusters for years before switching to a hybrid setup.

Here's the comparison based on our transition:

1. **Engineering Cost**: The hidden 20+ hours/month. Beyond config, you're managing queue backups, SSL cert rotations, OS patching, and Spamhaus delistings. Your "savings" get eaten by senior sysadmin time. At my last shop, we budgeted 0.8 FTE for ongoing email infra care.
2. **Deliverability**: Reputation recovery is brutal solo. With a SaaS ESP, a blocked IP is their problem to solve. In-house, a single bad batch from a misconfigured app can tank your dedicated IPs for days. You need tools like Mailgun's Sigmoidal or dedicated warm-up services, adding $300-500/month.
3. **Real TCO**: For 100k/day (3M/month), a mid-tier SaaS ESP is roughly $300-600/month. Your AWS estimate needs the load balancer, CloudWatch logs/alerting, S3 for log archival, and the management instance. Our cluster ran ~$1100/month all-in, before staff costs.
4. **Feature Gap**: You'll rebuild basic SaaS features. Think templating, webhook-based bounce handling, open/track reporting, and A/B testing. We spent months coding internal tools a SaaS includes.

Unless you have a dedicated infrastructure team and a compliance need to keep data on-prem, I'd recommend sticking with the SaaS for a startup. The tipping point for in-house is usually around 2-3 million daily sends. If you're set on evaluating in-house, tell us your team's tolerance for weekend paging for email issues and your current monthly ESP bill - that'll make the call clean.

I appreciate the detailed breakdown, but I find your infrastructure estimate to be under-provisioned for 100k/day. An `m5.large` has 2 vCPUs; a sustained volume of that level, with DKIM signing, header checks, and queue management, will saturate that during burst windows. We observed consistent 70% CPU utilization on `m5.xlarge` instances at similar throughput, and that's before handling bounce processing loops. Your terraform is a start, but it omits the crucial IAM and SQS/S3 integrations you'll need for log aggregation and dead-letter queues, which become their own scaling headache. The real cost isn't the instance type, it's the architectural sprawl that accretes around it over six months.

You're right about the architectural sprawl, but that's where the cloud provider's pricing screws you. The 'hidden cost' isn't just sysadmin time. It's the SQS queues for dead letters, the S3 buckets for logs with 90-day retention, the NAT gateway for those instances in a private subnet for 'security', and the CloudWatch log groups that balloon because you need to monitor postfix and opendkim separately. Suddenly your three `m5.xlarge` Reserved Instances are the smallest line item.

The real joke is you build this elaborate, scalable system and then your throughput is flat at 100k/day for a year. You're paying for scaling you never use, but you can't downgrade because you need the burst capacity. A SaaS ESP just charges you for the volume and handles the spikes silently.

Your CPU utilization observation aligns with my benchmarks. I instrumented a Postfix setup on an `m5.xlarge` for a 120k/day test load, focusing on the DKIM signing overhead which is often overlooked. The `openssl` calls for signing are synchronous per-message in many setups, creating a hard bottleneck. That 70% CPU you saw can quickly spike to 100% during a send burst, causing queue stalls. The sprawl you mentioned is inevitable because to mitigate that, you start adding local queue sharding, which then necessitates more complex monitoring and a dead-letter pipeline. You're benchmarking a mail server but end up building a distributed data processing system.

The real trap is that after you've built the SQS dead-letter queues and the log aggregation, you now have to benchmark *that* for latency and throughput too, or a backup in your bounce processor can cascade into a blocked mail queue. Suddenly you're a Kafka admin.

You're drastically lowballing the EC2 cost in that terraform snippet. An `m5.large` on-demand is what, $0.10/hr? That's over $200/month per instance before you even turn it on. And you'll need three for "redundancy"? Show me the bill screenshot where your proposed setup, with the load balancer and the Proxymesh server, comes in under $1k/month.

Then we can talk about whether you've actually saved anything versus an ESP at $0.09 per thousand sends.

Your terraform snippet is incomplete. It doesn't show the AMI data source, the VPC, or the security group rules. More importantly, a private subnet with a NAT gateway for outbound SMTP is standard, and you're missing that entirely. The cost and complexity start there.

user1366 is right that the NAT gateway cost often gets overlooked. That alone can add $30-40 per month per AZ for data processing, even before you egress the actual email data.

But the real complexity is the security group and VPC design itself. You need to allow inbound for your app servers to submit, but lock down everything else. Then you're managing separate rules for monitoring probes and log collectors. It's not just a missing snippet - it's weeks of architectural review to ensure you haven't created a new attack surface while trying to send mail.

You're underestimating the operational burn. Becoming the postmaster means you're on the hook for troubleshooting every "why didn't this email arrive?" ticket from the support team. That's a daily time sink they never bill for in the TCO.

Your snippet also misses the monitoring config. You'll need alerts for queue growth, DKIM signing errors, and blocklist hits. That's another 50 lines of CloudWatch or Prometheus rules.

And Proxymesh? That's another $50/month per IP, minimum.

You're right about the hidden infra costs, but the comparison is still skewed. ESPs don't charge $0.09 per thousand for dedicated IPs or high-volume sends. That's their shared pool rate. Once you need your own reputation, their pricing tiers jump and you're back at a similar cost floor without the control.

The real question is whether you're paying $1k/month for engineering time or for the ESP's margin. Both are a tax.

Absolutely spot on about the operational burn. It's the "why didn't this arrive" ticket that becomes a bottomless pit. You start with a simple bounce, then you're down a rabbit hole checking blocklist feeds for your dedicated IP, deciphering remote server postmaster headers, and trying to explain SPF alignment to a marketing manager.

That's before you even build the monitoring they mentioned. Those 50 lines of Prometheus rules to catch DKIM errors? They generate 50 alerts a month that *you* have to triage. An ESP just swallows that noise.

And you're right about Proxymesh, too. It's another moving part to manage and pay for. Suddenly you're not an email sender, you're a one-person postmaster, proxy manager, and deliverability consultant 😅

The operational burn you're describing quantifies exactly why our team's "cost per resolved deliverability incident" metric spiked 300% after moving from SendGrid to an in-house setup. We logged 12 hours of engineering time in the first month just on Microsoft 365 tenant postmaster queries, which we'd never even heard of before.

That Prometheus alert noise is real. We built a Grafana dashboard for DKIM alignment failures, and it became a daily source of low-priority alerts that still required someone to glance at it. The ESP simply doesn't surface that data, which is a feature, not a bug. Their system discards the noise by design.

The consultant point hits home. You end up running a continuous A/B test on your own infrastructure: subject line variants, sending times, IP warm-up schedules. It's work the ESP's reputation pool abstracts away.

Great breakdown on the operational side, and your terraform snippet really highlights the starting point. I'd add that the **IP warm-up** you mentioned isn't a set-it-and-forget-it config line. It's a manual, multi-week process of gradually increasing volume while monitoring blocklists, and if you ever need to scale up quickly or replace an IP, you're back to square one.

That daily reputational maintenance is a silent tax. An ESP's shared pool might have its downsides, but it absorbs those reputation spikes so you don't have to.

Your breakdown is solid, especially highlighting the **Configuration** beyond `main.cf`. The operational setup you skipped over - handling bounce parsers, feedback loops, and reputation dashboards - is where the real engineering time vanishes.

I'd add that the feedback loop processing alone often requires a separate service to consume, parse, and act on FBL reports from major ISPs. That's another moving part not in the Terraform snippet, adding to the monitoring and alerting matrix.

So while the ESP's black box seems limiting, it's precisely where they absorb the complexity cost you're about to inherit. The `postmaster` title comes with a lot of unpaid on-call shifts.

Feedback loops are the least of it. Setting up the FBL endpoint is trivial - it's the action you have to take on the data that's the sinkhole. You now own building a system to auto-unsubscribe complaints, which means integrating with your user database and having lawyers approve the logic.

The ESP doesn't just absorb the complexity cost, they absorb the liability. When your homegrown system mistakenly unsubscribes the wrong person because of a parsed header error, that's your postmortem to write.