Help with prompt chaining for complex scenes

Yes, I've battled this exact problem for marketing concept art. Your instinct to build the scene in layers is right, but the key is to stop thinking of it as separate "generations" and start thinking of it as a single, evolving canvas.

You'll have more luck using a single, consistent model throughout. Switching models mid-chain almost guarantees style and detail mismatch. For blending, inpainting is your primary tool, not a last resort. After you generate your strong background, upload it to the Image Editor, mask a specific area (like the space for a fruit stall), and inpaint with a detailed prompt just for that stall. Repeat for key elements. This adds to the scene without overwriting the whole canvas.

For character consistency, that's tougher. If you need the same person in multiple spots, generate the character in isolation first. Then use that image as a custom element in your inpainting prompt when placing them into the scene. It's not perfect, but it helps anchor their appearance.

Yep, the custom element trick for character consistency is solid advice. That said, the problem I often run into is that the isolated character, when in-painted into a detailed scene, looks pasted in if the lighting doesn't match. You have to be really explicit in the inpainting prompt about the lighting direction and ambient color of the scene, or the character will look like a badly photoshopped asset. "Character in isolation first" works, but you need to bake in some of the final scene's lighting conditions even in that first gen.

You're on the right track with the layered approach, but I think the breakdown is in the sequence. Generating the background first is good, but your second step is too broad.

Instead of trying to add "crowd and stalls" in one go, you need to treat them as separate, iterative inpainting operations on that saved background canvas. Use a tight mask for each distinct vendor stall and generate them one by one with a specific prompt, like "a wooden fruit stall with melons and apples, oil lantern, dusk lighting". Do the same for small groups of people. It's slow, but it prevents the overwriting.

For consistency, absolutely use the same base model for every step, including all inpainting. Style changes will ruin cohesion. A parameter like Alchemy will help across the board.

I've had the best results by establishing a "master scene prompt" for lighting and mood, then deriving specific, shorter prompts from it for each inpainting step. This keeps the ambient color and shadow direction locked.

Right about the model consistency. That's non-negotiable. The risk isn't just style mismatch, it's a fundamental lighting and texture shift that makes the final composite look like a cheap collage.

Your point on the custom element for character consistency is good, but it introduces a TCO problem for this workflow. You're now managing assets - the character image, the background canvas, each inpainted layer. For a one-off piece, fine. For a series? You need a versioning system or your project folder becomes a mess.

The real caveat is that inpainting is a commitment. Once you start adding those stalls, you're locked into that background composition. You better be sure about your initial camera angle and perspective before you begin the chain, because you can't easily adjust it later.

Totally feel that TCO pain point. Managing those assets turns into a full-on project management exercise after the second or third image, doesn't it? I've started tagging exports with a simple version code right in the filename, like "bg_v1_perspective_final_final.png", which is only half a joke.

And you're absolutely right about the background lock-in. It's like pouring the foundation for a house. I've burned hours because I didn't nail the light source direction in that first background generation. Once you inpaint a dozen elements with that baked-in lighting, trying to change the sun's position means starting over. Have you found a good way to spec that out beforehand? Maybe a super rough sketch?

Starting with your background first is smart, but the issue is trying to add too much in one step. The "crowd and stalls" prompt is too vague and the AI can't resolve all those elements at once without corruption.

Stick with one base model for the entire chain. Switching will break lighting and texture cohesion. For consistency, you'll need to inpaint elements individually with very specific prompts for each stall or character group, directly onto your saved background canvas. It's a slower, more manual process, but it prevents the overwrites and overlaps you're seeing.

Blending is almost entirely through iterative inpainting on that single canvas. Before you lock in that background, spend extra time getting the perspective and light source direction exactly right, as you can't easily adjust it later.

"TCO problem" is putting it nicely. It's asset sprawl. You're not building an image anymore, you're managing a mini project with dependencies.

> locked into that background composition

That's the real trap. Everyone talks about the inpainting steps, but the most critical prompt you'll write is the one for that first background. Get the focal length or horizon line wrong by a few degrees and the whole house of cards falls apart later. You can't fix foundational perspective with a masked prompt.

The "reliable workflow" you're asking for barely exists. You're trying to force a sequential assembly line onto a process that's fundamentally chaotic.

Everyone telling you to use the same base model is giving you the one piece of correct advice. The rest is just managing failure points. Character consistency across separate generations is a pipe dream without creating and managing custom assets, which is a whole other job. Blending? That's just hoping the inpainting algorithm doesn't introduce a glaring texture or lighting mismatch.

You're not building a scene, you're negotiating with a system that wants to generate a whole image at once. All this chaining is just a workaround for its limitations. My lesson learned? The more complex your chain, the higher the chance the final image looks like a committee designed it.

Yes, the single model advice is mandatory. The underlying noise patterns and texture libraries differ between models, so switching will break your scene's visual cohesion. I'd argue your current struggle isn't the chaining concept itself, but the granularity of your steps.

> Try to add crowd and stalls in a second gen

This is the core issue. "Crowd" and "stalls" are composite objects, not primitives. You need to decompose them further in your operational sequence. Think of it as a data pipeline: you establish your foundational layer (background), then you perform discrete, targeted transformations (inpainting operations) on specific coordinates of that canvas. Your prompt for each operation must be as specific as your initial background prompt.

For example, your workflow should look more like this:
1. Generate and finalize background canvas (perspective, lighting, castle).
2. Inpaint (masked) for "wooden textile stall with hanging rugs, lit by a lantern in dusk light".
3. Inpaint (masked) for "group of three people conversing near stall, backlit by dusk sky".
4. Repeat with distinct prompts for each major visual element.

Consistency for recurring characters *across different scenes* requires managing them as custom assets. Within a single scene, you achieve it by inpainting them in their intended location with a prompt that includes the scene's ambient lighting cues. Blending is a function of using the same model and a sufficiently detailed prompt for each inpainting mask; there is no separate "blending step." The system performs the blend based on the context of the surrounding pixels and your new prompt.

Forget a reliable workflow. You're trying to impose a rigid assembly line on a fundamentally chaotic process. The advice on using one model is correct, but the rest is just damage control.

Your core issue is thinking of "crowd and stalls" as a single step. That's a recipe for the overwrites you're seeing. You need to decompose further. Each vendor stall, each distinct character cluster, is a separate inpainting operation with a hyper-specific prompt on your locked background canvas. It's tedious, asset-heavy, and turns you into a project manager.

The real trap no one mentions enough is the irreversible commitment. That first background prompt is your foundation. Screw up the perspective or primary light source by a few degrees, and every subsequent inpainting step embeds that error. You can't fix foundational composition with a mask.

You're right about the negotiating part. It's less like engineering and more like steering a probabilistic system towards a local maximum, and every step adds more failure modes.

I do think the TCO analogy from earlier holds up. You're trading a higher upfront time investment in planning and asset management (the background canvas, character sheets) for a marginal increase in the probability of a usable final image. The question isn't "what's the workflow," but "what's the acceptable failure rate for the output?"

For me, the breaking point is usually around four or five inpainting steps. Past that, the accumulated entropy - the slight lighting mismatches, the texture drift - outweighs any gain in compositional complexity. The committee-designed look is inevitable.

This is really helpful. You mentioned generating the character in isolation first for consistency. Does that mean you need a separate tool to create that character asset, or can you do it within the same AI image generator using a really specific prompt?

You can absolutely do it in the same generator, it's just a different kind of prompt prison.

> generate the character in isolation first

This means creating a separate image, on a blank canvas, with a hyper-specific prompt for that single character. You're not inpainting yet, you're generating a discrete asset. Then you'd import that PNG and use it as a reference image (or img2img/inpainting source) when you place them onto your master background.

The caveat? You're now managing files, not just prompts. And the lighting on your isolated character *must* match the angle and intensity of your locked background, or they'll look pasted in. That's a whole other layer of spec to get right upfront.

So yes, same tool, but you're basically running a mini-project to create a sprite sheet before you even begin compositing.