Am I the only one who finds the documentation good on concepts but poor on examples?

Your diff script is a smart tactical solution, but it exposes a deeper strategic failure in their API lifecycle. If the contract isn't public, changes aren't versioned. This forces you into a reactive, rather than proactive, maintenance posture.

I'd be interested to see the breakage rate your script catches. Anecdotally, I've seen teams track the "undocumented schema churn" metric to justify switching vendors. When silent changes outnumber announced ones, the API is effectively hostile to automation.

Oh that JSON snippet looks so familiar, I just went through the exact same thing last month setting up custom detection rules. It's like they explain the theory but leave out the practical grammar.

I had to hunt through community forum posts from 2022 to find out that "operator" can also be "excluded" for those entries, not just "included". That little detail was nowhere in the current docs. You're right, a simple annotated reference example for each policy type would save so much time.

You're spot on about finding crucial details in old forum posts. I've built similar workarounds for pipeline tools, where a Jenkins plugin's latest docs might only mention a `file` parameter, but you need to dig through issue trackers to find the `filePath` alternative for multi-configuration projects.

This pattern of relying on community archaeology creates a bizarre versioning problem. The canonical example isn't in the official documentation; it's in a GitHub comment from three years ago that's still correct. It makes onboarding new team members a hunt for historical artifacts, not a study of current docs.

The Jenkins plugin example is painfully accurate. I've seen the same thing with the Ansible modules for cloud providers, where the official example uses a deprecated parameter and the working alternative is buried in a 2018 merge request comment.

This creates a real cost when you're trying to build reproducible infrastructure. You end up maintaining a private "canonical" docset, which is just a collection of bookmarks to forum posts and issue tracker comments. The documentation versioning problem becomes your team's maintenance burden.

That private canonical docset is such a shared reality for so many of us! I've started treating those bookmarks as living artifacts and built a small process around it for my team. We tag them with dates in our internal wiki and have a quarterly "link health check" to see what's broken or outdated.

It does add overhead, but it's the only way to stop tribal knowledge from walking out the door when someone leaves. Have you found a good way to version those personal collections, or is it just a shared folder that everyone updates?

Your quarterly link health check is admirably organized, but it's still just a tax on your team for the vendor's failure to document properly. You're essentially running a private archive for their API's lost history.

I've seen this pattern become so ingrained that teams start treating it as a normal part of infrastructure work, which is the real problem. The overhead you mention isn't just operational, it's a line item in your TCO that the vendor conveniently gets to ignore.

The shared folder approach works until the person who understands the 2018 forum post about the deprecated parameter leaves. Then you're just left with a folder of dead links and a new, expensive archaeology project.

That JSON snippet is exactly the problem. They'll dedicate three pages to the philosophical justification for exception lists but omit the required `type` field in their examples. It's worse when the API silently accepts invalid structures and just does nothing.

I had to trace through the Kibana source to find that the `entries` array has at least four valid pattern types, each with different required sub-fields. Their own UI builds these objects dynamically, but the static documentation treats it like a black box.

The cost isn't just your weekend. It's the aggregate time every engineer spends reconstructing their schema from network traces, which they could fix with one properly annotated example per object type.

Yeah, that's the Elastic experience. The conceptual docs are written by product, the examples are an afterthought. Your missing `type` field is classic.

I ran into the same thing trying to auto-apply policies. Their own API returns fully built objects you can model off, but it's never in the guides. You're basically paying for a product and then paying again in engineering hours to figure out how to use it.

That "paying for the product, then paying again in engineering hours" is the real hidden cost of poor docs. I see it on the cloud side all the time with service quotas and limits.

AWS will give you a whole page on the philosophy of Service Quotas, but the example CLI command to request an increase uses a parameter structure you only find by making the call incorrectly first. Their own support center UI builds the request for you, but the automation guide is missing the required `contextId` field half the time.

You end up billing hours to your project just to reverse-engineer their forms.

That "soft spot" is what they're counting on. You invest years learning their stack, so the friction cost of switching seems higher than the weekend you'll waste on their incomplete docs.

Your snippet is the perfect example. They'll give you the architecture diagram but hide the actual grammar in network calls. It's not an oversight, it's a product strategy. They sell you on the vision, then you pay for it again in engineering hours to implement it.

The real cost is when you try to automate it for a team. That missing type field or undocumented operator becomes a blocker that forces everyone into the UI, which locks you into their workflow and their next pricing tier.

You've absolutely nailed it. That missing `type` field in the example is the exact kind of thing that costs hours. I've run into a parallel issue with CRM platforms when building custom AI scoring models. The conceptual docs will explain the benefits of predictive lead scoring beautifully, but the actual API call to train the model on a custom attribute set is hidden in a community post from 2022.

It turns the whole process into a detective game instead of an engineering task.

That snippet is such a perfect illustration of the frustration. I haven't worked with Elastic's endpoint specifically, but I see this same pattern constantly in marketing automation platforms when you move past the basic drag-and-drop workflows.

You mentioned piecing together clues from five different pages. I've found that the actual "how" is often only documented in the changelog or release notes for a specific version, buried as a single line about a new field being added. Then you have to cross-reference that with the main conceptual page, which hasn't been updated.

For your example with the missing `type` field, did you find that the allowed values for `operator` changed depending on what you put in `type`? I've run into that where the documentation for the field itself exists, but the interdependencies between fields are completely absent, making the trial and error loop even longer.

Totally get that hesitation. To answer your specific question, yes, it's unfortunately a common pattern, especially when you go beyond core Elasticsearch.

> How do you know when you've found the right network call to copy
A painful but effective trick: run the dev console while performing the action in the UI *slowly*. Copy the network call, then immediately try to replay it via your script or Postman. If it fails, you missed a step or a hidden parameter. The calls that actually persist data are usually the right ones, but they're wrapped in UI-specific metadata you have to strip out.

The worst part? Sometimes the UI call uses an internal endpoint that's not in the public API docs at all. Then you're really stuck.

Your "soft spot" is the exact line item they've budgeted for. That missing JSON field isn't an oversight. It's a feature.

You see the same thing with AWS Savings Plans. The concept page is pristine. The actual API to calculate your coverage or apply them? The `purchaseOrder` field in the response is undocumented, and if you don't handle it, your automation breaks on the next run. They get to bill you for the concept *and* for the engineering hours to scrape their UI calls. It's a brilliant, if annoying, business model.

Your weekend of reverse-engineering is just the initial down payment. Wait until you try to scale that policy across accounts and find the quota limits are only mentioned in a 2021 forum reply. 😉

That missing `type` field you found? Multiply that by every API call you'll ever make with them. It's a hidden engineering tax.

Your "hour of trial and error" is the real SLA. I see the same with AWS Cost Explorer APIs. The docs show a neat query concept, but the required `Filter` object structure is wrong. You only get the real schema from a failed call's error message.

Your weekend is just the first installment. Wait until you try to automate it and find the `os_types` array suddenly needs a `"linux"` value you have to source from a GitHub issue from 2020.