How do I make a model that doesn't default to 'pretty young woman'?

Just spent another evening wrestling with the default aesthetic of SD 1.5 and XL checkpoints. It's the CRM equivalent of every sales tool defaulting to a "high-energy, team-oriented, rockstar rep" persona. Exhausting.

I'm trying to generate concepts for a professional project—think seasoned engineers, diverse age ranges, realistic textures. Yet, without fail, the unbaked prompt "a portrait of a person" yields a 22-year-old with flawless skin, symmetrical features, and that particular SD "airbrushed fantasy" look. It's like HubSpot's default pipeline assuming every deal is a SaaS subscription.

What are the actual levers here? I've tried:
* **Negative prompts:** "young, pretty, beautiful, woman, child, teenage" helps, but feels like fighting the model's core bias. It's a workaround, not a fix.
* **Prompt weighting:** Heavily emphasizing "middle-aged," "weathered," "professional," "technical" sometimes works, but the composition often gets bizarre.
* **Different models:** Some deliberate merges or fine-tunes (like epicrealism) are better, but they often just swap "pretty young woman" for "pretty young woman with slightly more realistic pores."

Is the only real path here training a LoRA or embedding on a deliberately curated dataset of non-"pretty" faces? Or is there a smarter way to adjust the sampling or attention to de-prioritize this baked-in bias? I'm not looking for a one-prompt solution, but a method to shift the baseline for a whole project. The data portability issue here is real—these biases are embedded deep in the model weights.