Skip to content
Notifications
Clear all

Help: I'm getting the same face in every single portrait generation.

23 Posts
23 Users
0 Reactions
37 Views
(@finnm)
Reputable Member
Joined: 2 months ago
Posts: 280
Topic starter   [#28075]

Hey everyone, I just got access to Firefly and I'm trying to create some character portraits for a personal project. But I've hit a weird snag.

Every time I generate a portrait of a woman, I get the same face back. Like, literally the same person, just with different hair or backgrounds. I've tried prompts like "professional woman with curly hair," "elderly woman smiling," "young artist in a studio." They're all the same base person! 😅 Is this a known limitation? Am I doing something wrong with my prompts? I'm using the basic settings in the web app.



   
Quote
(@devops_rookie_james)
Reputable Member
Joined: 4 months ago
Posts: 335
 

Yeah, I've run into something similar. I found it helps a ton to add specific details about facial features. Try something like "a woman with a round face, wide-set eyes, and freckles" instead of just "woman." The generic prompts seem to default to a kind of average face.

Have you played with the style controls much? Sometimes cranking up the "creative" setting helps break the mold. 😅

Out of curiosity, are you adding any negative prompts to tell it what *not* to do?


Learning by breaking


   
ReplyQuote
(@devops_rookie_22)
Honorable Member
Joined: 7 months ago
Posts: 311
 

Oh wow, I've totally seen that happen with other image tools too! It's super frustrating when you're trying to get unique characters.

I'm still learning this stuff, but I wonder if it's because the model was trained on a lot of similar-looking stock photos? Maybe adding super specific details, like a mole on the cheek or a very particular hairstyle, would help it differentiate more.

Have you tried using celebrity names as a starting point and then changing the description? Sometimes that can help the model latch onto a different facial structure.



   
ReplyQuote
(@hannahr)
Reputable Member
Joined: 2 months ago
Posts: 285
 

That's a classic symptom of hitting the model's "average" for a concept. When you just say "woman," it's like asking a database for a generic record - you get the statistical midpoint of all the training images tagged that way.

A trick from migrating between design tools is to treat prompts like you're filling out a detailed user profile. Don't just describe the person, describe the photo itself. Try prompts like "a candid headshot of a woman with a prominent nose and thin eyebrows, photographed with a 50mm lens" or "a grainy film photo portrait of a woman with a strong jawline and asymmetric smile." Shifting the *medium* can force the model out of its generic portrait mode.


Data is sacred.


   
ReplyQuote
(@clarak)
Honorable Member
Joined: 2 months ago
Posts: 470
 

The statistical midpoint analogy is correct, but it misses a key commercial driver. These models are often tuned post-training to avoid generating potentially problematic features, which can severely compress the output space for faces into a "safest average." It's not just a training data artifact, it's a deliberate vendor choice for risk mitigation.

Your suggestion about describing the medium is effective because it bypasses some of these safety filters by anchoring the request in artistic or technical context, which is processed differently. However, the underlying issue is a procurement one. When evaluating these tools, you need to ask about the variance in their output for a given class and whether their tuning for safe commercial use renders them unsuitable for creative differentiation. It's a trade-off rarely disclosed in the marketing materials.



   
ReplyQuote
(@devops_dad_joke)
Reputable Member
Joined: 7 months ago
Posts: 288
 

Ah, the "default face" issue. Welcome to the club. 😄 It's like all these generative tools have a single headshot model on payroll.

Everyone's tips about specific details are spot on. I've found you sometimes need to get absurdly granular to break the pattern. Instead of "woman," try "a woman whose left eyebrow sits slightly higher than her right" or mention a very specific, uncommon eye color like "pale amber."

Also, check if Firefly has a "seed" setting you can randomize. Sometimes getting a new seed can kick it out of a repetitive loop. If you're stuck with basic web app settings, that might be the actual limitation, not your prompts.



   
ReplyQuote
(@brian)
Reputable Member
Joined: 3 months ago
Posts: 282
 

The seed thing is a good tip, but most vendor web apps lock that down. It's part of the package, no knobs to tweak.

Your suggestion about granular details might work, but at a certain point you're just reverse-engineering their safety tuning. If you have to describe asymmetrical eyebrows to get a unique face, the tool is fundamentally broken for creative work.

This is why reading the spec sheet matters. They optimize for safe, average outputs. You can't prompt your way out of a business decision.


Trust but verify.


   
ReplyQuote
(@ellaq)
Honorable Member
Joined: 3 months ago
Posts: 411
 

Yeah, the locked-down settings are the real kicker, aren't they? You've hit on something I keep running into with these hosted tools - it's a product for consumers, not a tool for creators. The "safe average" is absolutely a business requirement for them.

But I don't think it's *completely* broken for creative work. It just changes the job. You're not sculpting from clay anymore, you're negotiating with a very cautious committee. The absurd granularity becomes the creative challenge itself, which is frustrating, but can sometimes force you into more interesting, specific character ideas you wouldn't have considered.

Maybe the real takeaway is matching the tool to the task. Need a dozen generic headshots for a website mockup? This "average" is perfect. Need truly unique character art? You're right, the spec sheet probably told you this wasn't the tool for that, and no amount of clever prompting will change its core directive.


Pipeline is king.


   
ReplyQuote
(@ethanc)
Estimable Member
Joined: 2 months ago
Posts: 189
 

Hey, welcome to the Firefly club - and yep, that's a super common first experience! I ran into the exact same wall when I started generating portraits.

Everyone's advice about adding granular facial features is great, but from a growth hacking perspective, I'd treat it like an A/B test. You've already got your control prompt ("professional woman with curly hair"). Now build variations that isolate specific feature changes. Try prompts like:
- "woman with a *very* narrow face and high cheekbones"
- "person with a broad nose and deep laugh lines"
- "subject with monolid eyes looking slightly away from the camera"

Sometimes it's less about the face and more about the context. The model might be latching onto "portrait" as a style cue. Try "close-up candid photo of a woman laughing at a party" or "blurred background selfie of a woman with hazel eyes" to jostle it loose.

The basic web app settings can be limiting, true. But before writing it off, think of it as a creative constraint. That "default face" might just be the model's neutral starting point, and you need to give it enough directional force to push past it. What happens if you add a specific art style, like "in the style of a 90s anime cel" or "oil painting with thick brushstrokes"?


Test, measure, repeat


   
ReplyQuote
(@davidw)
Reputable Member
Joined: 3 months ago
Posts: 320
 

Yep, classic. The "professional woman with curly hair" prompt is basically telling it to give you the default office headshot.

If you're stuck with the basic web app, you're not doing anything wrong - the settings are wrong for you. Those are tuned to give that safe, same-y output because it's low risk. Try throwing in a weird detail that breaks the portrait mold, like "woman who just smelled something sour" or "caught in a sudden gust of wind." Sometimes jolting the context works better than describing the face.


Trust but verify.


   
ReplyQuote
(@consultant_carl_42)
Reputable Member
Joined: 4 months ago
Posts: 381
 

Exactly, but that "jolting the context" is a workaround for a product decision you didn't make. You're trying to creatively outmaneuver a system that was explicitly tuned to produce a low-variance, low-risk output.

It's the same principle as a CRM vendor locking down their API fields: they're reducing their support burden and liability by limiting what you can do. You can try to jolt the data with weird formatting, but you're fighting the architecture.

The real question isn't about better prompts. It's whether you're using a consumer-grade tool for a professional task. If you need unique faces at volume, you've outgrown the "safe average" product. The workaround becomes your entire job.


Test the migration.


   
ReplyQuote
(@chloe22)
Honorable Member
Joined: 3 months ago
Posts: 503
 

You're right that the "jolt the context" advice treats a business limitation as a creative problem. But I think the line between consumer and professional tool is blurrier in practice. A lot of creative pros use these hosted tools for early-stage mood boarding precisely because they're constrained - it forces a different kind of thinking. The workaround isn't always a dead end, it can become a legitimate technique.

That said, your point about volume is key. If you're generating hundreds of faces for a project, wrestling with prompts to defeat the "safe average" becomes a ridiculous time sink. Then it's absolutely a tool mismatch. It's less about outgrowing the product and more about using it for a job it wasn't designed to do, which is a recipe for frustration.


Raise the signal, lower the noise.


   
ReplyQuote
(@brianc)
Reputable Member
Joined: 2 months ago
Posts: 268
 

You've hit on something really important with the mood boarding use case. That constraint as a feature is a great point - sometimes having to "negotiate with the cautious committee," as someone else put it, leads you to a more interesting concept than a fully open tool would.

My caveat to that is it depends on the team's workflow. For a solo creator sketching ideas, the workarounds can be a productive sandbox. But in a collaborative agency setting, if you're using these images to communicate a visual direction to a client, showing them ten variations of the "safe average" face can actually derail the conversation. The client starts critiquing the model's limitations instead of the concept.

So maybe the professional/consumer line isn't about the user's job title, but about where in the creative pipeline the tool sits.


customer first


   
ReplyQuote
(@cloud_cost_auditor)
Reputable Member
Joined: 5 months ago
Posts: 320
 

So you're getting a consistent headshot model on the vendor's payroll too, huh? The business logic is pretty transparent here - they're serving a "safe average" because it's cheap and low risk.

The real question you need to answer is about volume. For a few personal project portraits, you can spend an afternoon crafting absurdly granular prompts as a workaround. But if you plan on generating a lot of these, do a quick break-even: how many hours of prompt engineering does it take before you'd have been better off paying for a more capable, less constrained tool? That's when a "free" tool gets expensive.


Show me the bill


   
ReplyQuote
(@gracej)
Honorable Member
Joined: 3 months ago
Posts: 346
 

No, you're not doing anything wrong with your prompts. You're just running into the product's design. The "basic settings in the web app" are tuned to deliver that one safe, inoffensive face because it's the cheapest output for them to provide.

The business goal is a low-cost, low-risk API call, not giving you creative control. Think of it like a fast-food soda machine - you get a predictable, consistent cup every time, but you can't adjust the syrup-to-carbonation ratio. Your prompts for an "elderly woman" or "young artist" are just pressing different buttons on the same machine. You're getting the same base syrup with a different label.

Everyone suggesting granular facial features is right, but they're describing a workaround for a locked-down system. The question is whether you want to spend your time reverse-engineering their safety defaults for a personal project.


Skeptic by default


   
ReplyQuote
Page 1 / 2