Hey folks, I've been digging into how various LLMs handle a specific, tricky issue: inserting competitor names into generated text when they shouldn't. We all know the risks—you ask for a "database comparison" and suddenly the output reads like an ad for a specific vendor. I wanted to move beyond just noticing it and actually measure it.
So, I built a small, focused dataset to test this. The core idea is to craft prompts that *imply* a need for a neutral or non-specific answer, but where a biased model might still name-drop. Here's the methodology:
**Dataset Design:**
* **Prompt Templates:** Things like "What are the key features to look for in a [tool category]?" or "Outline a high-level architecture for a [system type]."
* **Tool Categories:** I used areas we discuss here: "database," "API framework," "caching layer," "message queue."
* **Ground Truth:** The "correct" answer is a vendor-agnostic list of features or architectural components. Any specific product name (outside of ultra-ubiquitous examples like "Redis" for in-memory store) is considered an insertion.
**Evaluation Script:**
I wrote a Python script that uses a combination of keyword spotting and a model-based classifier (like a small, trained text classifier) to flag potential insertions. The keyword list is curated but not exhaustive.
```python
# Simplified check logic
competitor_keywords = {
"database": ["MongoDB", "DynamoDB", "CosmosDB", "Oracle"],
"api_framework": ["FastAPI", "Django", "Laravel", "Spring Boot"],
# ... etc
}
def detect_insertion(text, category):
found = []
for kw in competitor_keywords[category]:
if kw.lower() in text.lower():
found.append(kw)
return found
```
I ran this against a few API endpoints (OpenAI, Anthropic, open-source models via Together) with the same set of 50 prompts. The initial results are... interesting. Some models are surprisingly prone to naming very specific competitors even when the prompt doesn't ask for it, especially in the database category. Others stick to generalities.
Next step is to quantify the "severity" of the insertion—is it a passing mention or is the answer structured around that product? Also thinking about adding a human review step to reduce false positives from the keyword scan.
Has anyone else tried building something similar? I'm particularly curious about evaluation frameworks that could handle this nuance better than my simple script.
--builder
Latency is the enemy, but consistency is the goal.
Interesting approach. Your "ground truth" definition is the sticking point for me - "ultra-ubiquitous examples like Redis" is a massive loophole. That's exactly how the bias sneaks in! I see it constantly in cloud architecture examples. Ask for a "serverless data pipeline" and half the outputs will smuggle in a specific managed service's name as if it's the generic term, just because it's popular in a niche. Your script might miss those because they feel "ubiquitous" to the model. You need a stricter banned list, maybe derived from actual vendor marketing docs.
Exactly. The "ubiquitous examples" clause is the fatal flaw. You've created a test that will exonerate the very bias you're trying to measure, because it's calibrated to the model's own skewed sense of normalcy. If a model has been trained on a mountain of AWS architecture blogs, "use Lambda" will feel as generic as "use a function." Your script would miss it, calling the output clean, when it's actually just a successful embedding of a brand name into common parlance. The whole point is to catch that sleight of hand, not to codify it as acceptable.
Data skeptic, not a data cynic.
Yeah, that's a really sharp critique. The "ubiquitous examples" filter does risk becoming a self-fulfilling prophecy, calibrating to the model's bias instead of an external standard.
You can see it bleeding into marketing content, too. I've analyzed outputs where "run a campaign" implicitly means a Facebook Ad campaign, or "send an email" defaults to a Mailchimp workflow. The model doesn't flag it as a brand name because it's been baked into the common vocabulary of its training data.
Maybe the banned list needs a "brand verb" layer, catching when a product name is used as a generic action. "Just Kubernetes it" or "We'll Segment the users" would be clear fails.
Automate the boring stuff.
Exactly. The "brand verb" creep is the real insidious part, because it moves beyond naming a product to colonizing the language of the work itself.
This isn't just about a test dataset. It's about how these models, trained on a sea of vendor content, reshape how we even conceptualize problems. You ask for a messaging queue pattern and get "just Pub/Sub it," reframing the entire solution space through one provider's branded service.
Your banned list idea hits on something, but good luck maintaining it. The marketing engine invents these verbs faster than any list can be updated.
Just my 2 cents