Need to generate consistent character frames for video storyboards from text prompts. Compared DALL-E 3 (via API) and RunwayML (Gen-2 text-to-video, but used frame-by-frame).
**Key findings for SRE/ops workflow:**
* **DALL-E 3**
* Superior prompt adherence. Describes a "data center hallway with blinking red lights," you get exactly that.
* Character consistency across frames is a major struggle. Same character in pose A and pose B looks like two different people.
* Latency: ~12-15 seconds per image via API. Batch generation helps.
* Cost: $0.040 / image (1024x1024). Predictable.
* **RunwayML**
* Weaker on complex, specific prompts. "Blinking red lights" might be ignored.
* Better intrinsic character consistency if you use the same seed and a short video clip, then extract frames.
* Latency: Variable. Can be faster for a 4-second clip (~90 seconds), but then you have to extract frames.
* Cost: Credits system. Less predictable for high-volume frame generation.
**Verdict:**
* Use DALL-E 3 if your storyboard requires precise environments, objects, or scene composition. Manual character correction needed.
* Use RunwayML if your primary need is a consistent character across a short sequence and prompt details are flexible.
For automation, built a simple pipeline with the DALL-E 3 API and a consistency filter script. Example batch call:
```python
# Pseudocode for batch frame gen
prompts = load_storyboard_scenes("scenes.yaml")
for prompt in prompts:
response = openai.Image.create(
prompt=f"Storyboard frame, cinematic, {prompt}",
n=1,
size="1024x1024"
)
# Save, log, and trigger alert on failure
save_to_monitored_dir(response['data'][0]['url'])
```
RunwayML's API less suited for this discrete frame-by-frame approach. Their strength is temporal coherence, not static frame precision.
back to monitoring
I'm a creative director at a mid-size digital agency (around 80 people) where we produce explainer videos and client pitches. I run both DALL-E 3 via Azure OpenAI and RunwayML in our production workflow, specifically for generating and iterating on storyboard frames.
* **Character Consistency:** RunwayML has a clear advantage if your character is central to the scene. By generating a short 4-second video from a detailed character prompt and extracting frames, the core facial structure and attire remain coherent. DALL-E 3, even with meticulous prompt engineering, introduces noticeable variations in facial features and body proportions across a sequence, requiring manual correction in Photoshop.
* **Prompt Fidelity & Environment Control:** DALL-E 3 is decisively better for complex, specific environments. In our tests, prompts detailing "a 1960s kitchen with checkerboard linoleum and a yellow refrigerator" yielded correct details 9 times out of 10. RunwayML's video-centric model often simplifies or ignores specific environmental details in favor of motion and atmosphere.
* **Cost Predictability:** DALL-E 3's API cost is fixed per image ($0.040 for standard 1024x1024). For a 30-frame storyboard, that's $1.20. RunwayML's credit system becomes less predictable at scale; generating those same 30 frames via video clips can range from $0.84 to over $3.00 depending on clip length and resolution choices, making client billing projections fuzzy.
* **Operational Integration:** DALL-E 3's API fits cleanly into automated pipelines. We script batch generations and feed results directly into our asset manager. RunwayML's web interface and credit-based model introduces manual steps; you're often generating, downloading, and then extracting frames, which adds operator time.
My pick is DALL-E 3 for most agency work because our storyboards are environment-first and client feedback usually focuses on objects and setting. If your primary constraint is a hero character that must be visually stable across every frame without manual work, RunwayML is the only viable choice. Tell us whether your storyboards are environment-driven or character-driven, and if you're scripting batch operations or having an artist manually generate each scene.
Buyer beware, but with a spreadsheet.
Your cost breakdown is the critical part for ops. That $0.040/image is predictable, but the real budget killer is the rework.
You said DALL-E 3 requires "manual correction in Photoshop" for character consistency. That's a soft cost most forecasts ignore. If you need 20 frames and have to manually tweak 40% of them, you've added 2-3 hours of designer time. At agency rates, that single line item blows the API cost out of the water.
Runway's lower prompt fidelity might mean more generation rounds to get the environment right, trading one cost for another. You need to track that iteration count.
Metrics or it didn't happen.