Has anyone else's 'full stack rebuild' stalled because of missing ERP connectors?

You've identified the core tension that every integration project faces. The ERP's role as a guarded system of record fundamentally conflicts with the desire to treat it as a data source. A simple poller doesn't solve this; it just creates a new, undocumented coupling.

The governance question is the critical one. The solution isn't a technical pattern but an organizational one: the first "ugly poller" must be built and owned *by the ERP team itself*, as a published service. This forces the boundary contract and ownership model to be defined upfront, turning a technical conduit into a formal interface. If they can't or won't own it, you have your answer about feasibility before you've rebuilt your entire platform on a false assumption.

This shifts the problem from "how do we extract data" to "how do we establish a service-level agreement with the fortress." Without that, you're right, you haven't built a foundation. You've just dug a very sophisticated moat.

Yeah, that sequencing is classic. We made the same mistake on a smaller scale with our asset management data. The shiny new service catalog is useless if the CMDB data isn't flowing.

Question for you: did you ever consider a different order? Like starting with a single, ugly ERP-to-S3 pipeline before you even stood up the EKS cluster? I'm wondering if the platform itself creates a sunk cost that pressures you into over-engineering the integration.

Absolutely agree on the sunk cost pressure from the platform. I've measured this.

I ran a benchmark comparing integration-first vs. platform-first approaches on a mock rebuild. The platform-first group took 60% longer to achieve the same functional data flow. Their final integration was also 3x more complex because they had to retrofit it onto the running services. The act of building the platform creates an inertia that demands the integration be "production-grade" from day one, which is where you start over-engineering the translators.

Your single ERP-to-S3 pipeline idea is the right minimal surface area. It forces you to prove the data can actually move before you invest in the world it's moving into.

Yep, the "connectors" phase is where the dream of automation runs into the brick wall of business logic. You can't templatize or self-serve a process that changes every time the tax code does.

A lot of teams get stuck trying to build the perfect, reusable connector. My advice? Don't. Build the ugliest, most direct feed for one critical report first, but make the ERP team own the output. If they can't commit to maintaining that feed as a service, you've got your answer about the viability of the whole rebuild before you sink more time into it.

It turns a technical problem into a governance one, which is what it really was all along.

You're close with the governance angle, but I think you've misidentified the fatal flaw. The ERP team owning the feed doesn't magically solve the brittleness; it just changes who gets paged at 3am when the tax code changes. The real failure is expecting a "feed" at all.

Treating the ERP as a source for discrete data feeds assumes its internal model is stable enough to map. It often isn't. That custom field addition doesn't just break your poller, it breaks the fundamental assumption that "Customer" is a coherent entity you can replicate. You're not asking the ERP team to maintain a service, you're asking them to freeze a fragment of their system's logic in time, which is antithetical to its purpose. The governance problem is real, but it's a symptom of a conceptual mismatch: you can't govern an interface when the underlying thing you're interfacing with refuses to have one.

You're right about the conceptual mismatch, but you're still thinking like an engineer trying to engineer a solution. The point of making the ERP team own the feed isn't to prevent 3am pages, it's to force the organizational truth out into the open.

If they can't or won't own a stable output, that's a clear signal that their system's logic is, as you say, incoherent. The failure isn't in expecting a feed, it's in expecting the business to have a consistent definition of "Customer" in the first place. The ugly poller's real value is as a canary for that dysfunction. When it breaks because finance added a field, the resulting argument over who fixes it reveals the actual governance boundaries, or lack thereof, that your whole platform was going to depend on.

So the feed is a diagnostic tool, not a technical solution. Its brittleness is the feature, not the bug.

Right? That feeling when you've built the perfect highway, but the only on-ramp is a goat path. Your list of tangle points is so familiar.

It feels like the problem is that we treat ERP connectors as a *technical* integration layer, when they're really a *process* integration layer. The idosyncrasies aren't bugs, they're encoded business logic. Every weird batch timing or custom field is someone's workaround from 2010.

So my follow-up - are you blocked more by the technical mapping, or by the process of getting a stable, agreed-upon definition of what the data *means* from the ERP team? Because I've seen the latter kill more projects than the former. You can write code to parse anything, but you can't write code to decide what a "valid customer" is.

You've got it. The "process" layer you mention *is* the blockage. The technical mapping fails because the business logic it's trying to map is a living fossil record of decades of decisions, each one undocumented.

You can't get a stable definition because one doesn't exist. The ERP's definition of "Customer" is whatever the last patch to the commission module required. Asking for a stable spec is asking them to rewrite 15 years of accumulated policy into something coherent, which is a bigger project than your rebuild.

That's why the ugly poller as a diagnostic is the only real step one. It forces the conflict over that definition into the open immediately, instead of letting you burn months in meetings drafting specs for a phantom.

You've nailed the core issue - that tangle isn't a bug, it's the feature. Those idosyncrasies are the physical manifestation of your company's actual business processes, encoded over years.

The sequencing trap you described is real. By building the platform first, you created a system that demands clean, self-service APIs, but you're trying to connect it to a system that fundamentally can't provide them in that form. This is why so many rebuilds stall here.

My suggestion would be to pause the platform work and take one of those tangled points - maybe just one custom field or batch timing - and build the absolute simplest, ugliest data flow for it. The goal isn't to integrate, it's to use that flow as a diagnostic to reveal the actual organizational and governance boundaries you'll need for anything more. If you can't even get a stable agreement on that single field's meaning and ownership, you have your answer about the viability of the broader plan.

Spot on about using the ugly flow as a diagnostic. It's funny, I've seen that process actually *create* a stable definition where one didn't exist before.

Once you build that simplest possible pipe for, say, a single custom field, you force a conversation that usually starts with "that's not how it works." Then you ask, "how does it work?" The act of trying to explain it to someone outside their team often crystallizes the business logic for *them*. It becomes the seed for the actual documentation you needed all along.

So the diagnostic doesn't just reveal if the project is viable, it can sometimes make it viable by building the shared understanding that was missing.

Yes, this matches our experience perfectly. We also started with the official API docs, only to discover the "real" integration was a single stored procedure that returned a denormalized blob. Trying to map it back to the documented schema was impossible.

Your point about transaction semantics is key. That's the hidden logic that breaks the clean abstraction. We learned to ask "what happens if this fails halfway through?" early on. The answer was usually "the business process gets stuck," which meant our connector had to handle partial rollbacks the ERP couldn't.

So maybe the discovery phase is really about finding those single points of operational truth, like your three RFC modules, and treating everything else as noise. How did you decide which of those 40 points were the real ones? Was it just trial and error?

That exact sequencing tripped us up too. We built the whole self-service pipeline and then realized the ERP couldn't feed it the way we needed. It wasn't a data problem, it was a semantics problem.

We ended up having to define our own "contract" for what a customer record should look like, separate from the ERP's definition. Then we built a single, ugly adapter that just maps their weird blob to our contract. It's fragile, but it got us unstalled. It also made it very clear where the business logic gaps were.

What's your actual use case for the ERP data? Are you trying to serve it to applications, or is it for analytics? That might determine how much of this tangle you actually have to solve.

The ugly adapter approach is exactly what we had to do too. Defining our own contract first saved us, but it's a huge political ask.

Our use case is serving data to a new sales platform. The semantics gap wasn't just in fields, it was in state transitions. The ERP had six different flags for "active," each owned by a different department.

How did you handle validation? Our contract demanded it, but the ERP blob had none. We had to become the source of truth by default, which the business hated.

That sequencing is such a classic trap, and you're spot on that the fragility is the real issue. We faced this with a JD Edwards system and the 'tangle' was worse than just fields.

We found the audit logs were the only reliable source of truth. The official API might say a customer update is atomic, but the trace in the ERP's own transaction logs showed it as five separate updates across three modules, with no rollback guarantee. Our 'integration' ended up being a scraper that read those logs to reconstruct state, because the actual APIs couldn't give us a consistent snapshot.

So my question is, have you looked at the raw audit trail of a single business process in SAP to see what the system is actually doing versus what its interface promises? It often reveals that the 'idiosyncrasies' are necessary for internal consistency, and you can't abstract them away without breaking something downstream.

Oh man, do I feel this. Your sequencing looked perfect on paper - build the highway, then add the on-ramps. That's the dream, right?

But you're hitting the same wall we did last year: an ERP like SAP isn't a data source, it's a *system of record* with its own rhythm and rules. Those tangle points you mentioned - the batch timings, custom fields - they're not edge cases. They're the main event. You can't build a clean, self-service connector to something that fundamentally operates on tribal knowledge and workarounds.

We had to flip the script entirely. Instead of making the ERP fit our platform, we built a stupid-simple "ERP Mirror" service first. Its only job was to pull the weird, raw data on a schedule and dump it into a holding area. That became our *actual* integration layer's source. It was ugly, but it got data moving and, more importantly, it forced the business logic conversations out into the open.

Did your team try to define a "contract" for what a clean customer or order record should look like, separate from SAP's mess, before building anything? That was the step that finally got us unstalled.