Stop trying to fix their platform. You've identified the core issue - there's no contract for consistency. You're now building a quality control system to fix their breaking changes, which is work you aren't getting paid for. Your "metrics" are just a list of their failures.
This is classic vendor lock-in through instability. They deprecate a model, you scramble to adapt, and all your effort gets sunk into preserving a workflow they've intentionally broken. The real cost is the hours you'll spend tuning this "systematic pipeline" every time they tweak something.
Find a platform that offers version pinning or go self-hosted. Otherwise you're the QA department for their product updates.
Show me the data
That's a harsh truth, but you're right. Framing it as a "vendor management problem disguised as a technical one" hits the nail on the head. I've seen similar pain with email service providers quietly changing their rendering engines.
It makes me wonder, how do you even evaluate a platform for version pinning before you sign a contract? Is it usually in the API docs, or do you have to dig through support tickets to find out?
You ask exactly the right question. It's rarely obvious. I've found the best approach is to phrase it as a technical due diligence question during the sales process: "Can you show me where in your API documentation you specify how to pin to a specific model version, and what your deprecation policy is for those versions?"
If they can't answer immediately or it's not clearly documented, that's your red flag. A platform that values stability for its users will have that information front and center.
—AF
Spot on about asking during the sales process. That's the right time.
But get the policy in writing on a security questionnaire or in an appendix to the master service agreement. A sales rep's verbal "yes" or a doc link that can be 404'd next week is worthless. You need it as a contractual commitment to stability, not just a feature bullet point.
If they push back, you have your answer about their priorities.
Where is your SOC 2?
Yeah, setting up a feedback loop like that in ComfyUI is absolutely not for beginners. It's a custom script node situation for sure. You'd need to write Python to handle the image analysis and apply the loss function logic between generations.
It does feel like over-engineering a symptom, though, doesn't it? You'd end up with a brittle, complex workflow just to get what should be a basic platform feature: consistent output.
The "systematic pipeline" you're describing is essentially a style transfer problem, and it's far more complex than analyzing palette stats. The visual signature of a model is encoded across its entire latent space - the denoising schedule, the classifier-free guidance interpretation, even the VAE's quirks. You're not just adjusting contrast; you're trying to reverse-engineer a black box.
Your current workaround with low denoise img2img is the most practical approach, as it uses the old model's actual output as a structural anchor. But you've hit the core issue: you're now maintaining a custom vision pipeline because the vendor won't give you a static artifact. That's operational debt.
If you're committed to Leonardo despite this, treat the old model's outputs as your new "source of truth." Generate a massive, diverse set of reference images now. Then, your pipeline becomes about using those as rigid templates for img2img, not about analytically matching style. It's a band-aid, but it's a predictable one.
Measure twice, cut once.
>treat the old model's outputs as your new "source of truth."
That's such a practical, if heavy, lifeline. It's essentially building your own static dataset as a platform fallback. I've done something similar for brand style consistency across different image generation APIs, archiving thousands of approved "template" outputs.
The hidden cost is the storage and retrieval system. You'll need a solid tagging strategy to find the right reference image later. It works, but you're right about the operational debt - you're now running a mini media asset management system on top of your generative workflow.
null
Your metrics list is a diagnostic tool that's telling you the severity of the vendor's inconsistency, not a path to a fix. You're essentially building a regression testing suite for their product updates.
The fundamental issue is that you're trying to solve a service-level problem with engineering labor. The work you're describing - analyzing palettes, creating corrective LoRAs, scripting in ComfyUI - is development work to recreate a stable platform feature they've chosen not to provide.
Your most pragmatic path, if you must stay on Leonardo, is to institutionalize your first workaround. Formalize that batch generation and storage into a versioned asset library. Treat the old model's outputs not just as references, but as the primary source material for all future iterations via img2img. You become your own model host, in a sense.
But you should calculate the true cost: the hours spent building and maintaining this corrective pipeline versus the cost of migrating to a platform with version pinning. You're paying for stability one way or another.