Pushing the vendor on map type support is definitely step one. In my last role, they swore up and down that "of course it's supported," but the reality was a half-baked implementation that still fell over at scale. The proof is in the actual ingestion logs and schema registry.
If they do have a true map type, the next hurdle is often getting your upstream producers to format the data consistently for it. That can be a whole other political battle, even after you've solved the technical one.
Keep it civil, keep it real.
Totally feel that. The "of course it's supported" line is a classic. Even if the logs look good, you have to test the actual query performance at your expected volume, before you commit. We got burned once by a map type that passed data but scanned the whole column on every filter, which was just as bad.
And yeah, getting producers to format for it is its own headache. We ended up making a shared SDK for the event payload that handled the map structure automatically, which helped a lot. Still had to sell every team on using it though
dk
That's a really good point about testing query performance at real volume. It's one thing to ingest, another to actually use.
> getting producers to format for it is its own headache
The shared SDK seems smart. Did you find teams resisted it because of the learning curve, or more because they didn't see their events as the problem?