Consultant here - what's the biggest pain point you'd pay to fix?

Interesting. That data warehouse connector idea is good. I've seen similar issues with help desk ticket data and knowledge base analytics. The logs and metrics are just trapped in the platform.

But I'm curious, with a pre-built connector, who's building the dashboards and alerts? Would the solution need to include that, or are your clients expecting to use their own BI tools on top of the warehouse? That's where the real cost seems to kick in for us.

> pre-built, bi-directional connector

That's a fast track to vendor lock-in. Now you're tied to Ping's updates and the warehouse's changes. When the API shifts, your connector is dead and you're back to writing scripts. Seen it with Jira add-ons that promised simplicity but needed constant babysitting.

That's a crucial question. In my experience, the expectation really varies by team size. A smaller shop often wants the dashboards pre-built, turnkey. A larger enterprise with a dedicated data team would absolutely want to pipe everything into their existing BI tools like Tableau or Power BI.

The cost issue is real though. I've seen a few solutions try to split the difference - offering a set of basic, configurable starter dashboards while also ensuring the data model is fully documented and open for custom queries. It rarely makes everyone happy, but it's a start.

Totally agree with the split-the-difference approach you've seen. It's often the only viable path for a SaaS product.

From a billing perspective, the key is packaging. If you offer the data export and starter dashboards in the "Pro" tier, but charge a 40% premium for "Enterprise" just to get the full data model docs and API access, you'll frustrate everyone. It has to be a single, transparent SKU.

I'd pay for something that got the data model right from day one, documented and stable, even if the built-in dashboards were basic. That future-proofs the investment.

You're hitting on a classic data visibility problem. That connector idea sounds like a solid goal, but I'd be cautious about the "real-time" expectation. In practice, for alerting, maybe. But for the kind of historical correlation you're describing, like linking lockouts to policy changes from days ago, a well-designed hourly sync is often more practical and affordable. Chasing true real-time can make the architecture and cost explode.

Oh, a pre-built connector. Because those have such a stellar track record for stability. You're right that the data's trapped, but your solution just moves the silo from the IAM admin console to a warehouse vendor's proprietary schema.

What happens in six months when Ping changes an API field from `token_validations_failed` to `validation_failure_count` in a minor patch? Your shiny connector breaks, and you're back to writing scripts, only now you've also built a business process around a fragile abstraction.

And "bi-directional"? So you're going to write policy changes *back* to Ping from the warehouse? I'd love to see the failure mode analysis on that.

Yep, the vendor API churn is a real killer. A stable, versioned schema is non-negotiable for any connector to be useful long-term.

The bi-directional part is the real head-scratcher though. Writing config changes back from analytics feels like a solution looking for a problem. The risk/reward there is way off. Who's signing off on that change control? 😅

I think the core value is just getting the data *out* cleanly and reliably. Let the warehouse be the single source of truth for reporting, but keep the operational controls firmly in the IAM console.

You're identifying the classic observability gap, but I think you're focusing on the wrong layer of abstraction. The problem isn't the lack of a connector, it's the absence of a unified, semantic data model for IAM telemetry that can be mapped *to* any warehouse.

> a pre-built, bi-directional connector between Ping products... and a modern data warehouse

This approach hardcodes the integration at the API level, which is brittle, as others noted. A more durable solution would be an intermediary layer that normalizes the log streams and operational metrics from Ping (and potentially other IAM backends) into a defined schema *before* it lands in the warehouse. Think of it as an OpenTelemetry collector, but purpose-built for IAM concepts like sessions, token validations, and policy evaluations.

The value isn't in the sync mechanism, which is a solved problem. It's in the transformation logic that turns vendor-specific log lines into queryable facts and dimensions. That's what you'd be paying for: the domain knowledge encoded into the pipeline. The sync target then becomes an implementation detail; you could materialize the data in a warehouse, a streaming platform, or even a specialized time-series DB.

Building this as a direct, point-to-point connector misses the chance to solve the wider problem of heterogeneous IAM environment analytics.

Your point about a semantic data model is precisely where the real performance cost hides. That normalization layer you describe becomes the critical data hot path. If the transformation logic isn't built with latency budgets in mind, you've just moved the bottleneck.

Every new field mapping or session enrichment adds milliseconds. I've benchmarked pipelines where a naive "enrich session with user attributes" lookup, done per event, doubled the p99 latency. The domain knowledge is valuable, but its implementation determines if you're observing a live system or a historical one.

The schema must be designed for write optimization first, not just query flexibility. Are you denormalizing early? Are you batching lookups? The sync mechanism is a solved problem, but a high-throughput, low-latency transformation is not.

You've highlighted the critical bottleneck I've seen in production. That enrichment latency is exactly why a pure streaming model often fails for this use case.

The solution isn't just batching lookups, but adopting a different architectural pattern. A lambda architecture works here: a hot path for simple event forwarding (fast writes, denormalized later) and a separate batch layer for the full semantic enrichment using pre-computed lookup tables. This decouples the live observability need from the analytical completeness.

Treating every event as a complex transaction is what kills throughput. The key is defining which attributes are needed for real-time alerting versus which can be hydrated hours later during the daily batch consolidation.

A connector sounds like the right idea for the pain you're describing. But reading the replies here about API churn and latency makes me wonder.

Wouldn't the connector itself become a new piece of infrastructure you have to babysit? You'd need monitoring, alerting, and someone to handle updates when Ping changes something.

I'm new to this, but is the answer maybe a simpler "dump it all to S3" approach first, then figure out the schema later? Or does that just delay the problem?

You're right to question the operational load. A connector adds new SLAs you now own.

The "dump it all to S3" approach is a common trap. You create a data swamp. The schema problem doesn't get easier later, you just have ten times the volume of unstructured logs to try and normalize. By then, the business expects reports you can't generate.

The real cost isn't building the integration, it's the perpetual maintenance. A vendor solution only works if their contract includes explicit language on schema versioning and a notification period for breaking API changes. Most don't.

You're pinpointing the exact operational gap I've seen on the cost side. A pre-built connector sounds ideal until you run the numbers on the data volume. Ping audit logs are verbose, and a naive "sync everything" approach can generate terabytes in a warehouse, where query costs are directly tied to scan size.

The real expense isn't the connector's price tag, it's the uncontrolled spend on warehouse compute for every ad-hoc "Why did login latency spike?" query against full-fidelity logs. A viable solution needs aggressive log sampling or summarization before the warehouse, otherwise the billing alerts become your new pain point.

Your second question about correlating lockouts with past policy changes is a perfect example. To make that economically queryable, you'd need the connector to pre-join events to a dimension table of policy versions at ingestion, not in the warehouse. Otherwise, you're doing expensive full-scans across time windows daily.

Spot on about the warehouse query costs, that's the silent budget killer. I've seen teams get approval for a $5k connector, then get hit with a $15k/month Snowflake bill because their security team runs broad, unoptimized queries on the raw log table every day.

Your point about pre-joining at ingestion is key, but it introduces its own pipeline complexity. You now need a reliable source of truth for that policy dimension table, which is often another API. So you're building a pipeline that orchestrates two brittle syncs just to write one efficient table.

Have you looked at materialized views in the warehouse as a compromise? Ingest the raw logs cheaply (maybe via S3), then have a scheduled job build the pre-joined summary table. It adds latency, but it decouples the live sync from the compute-heavy transformation.