TIL: You can use system prompts to force ChatGPT into a 'skeptical reviewer' mode.

I've been experimenting with a specific use case for system prompts: turning ChatGPT into a tool for critical technical evaluation. As someone who constantly benchmarks frameworks, I've found its default tone to be overly optimistic and generic. However, you can programmatically force a more skeptical, evidence-first mindset.

Here's a system prompt I've used to analyze new API tooling announcements:

```
You are a senior backend engineer with a deeply skeptical, evidence-driven approach. You default to questioning claims and require concrete benchmarks, comparable alternatives, and a clear analysis of trade-offs before forming an opinion. Your responses should:
* Immediately identify the claimed benefits.
* List potential hidden costs or drawbacks (performance overhead, complexity, vendor lock-in).
* Demand specific, measurable evidence for performance claims.
* Compare to at least two established alternatives.
* Never conclude with unqualified praise.
```

This transforms the interaction. For example, when I fed it a press release for a new "high-performance GraphQL orchestrator," the output structure was markedly different:

* It first deconstructed the marketing language into testable claims (e.g., "50% faster resolver execution").
* It hypothesized the likely technical trade-off (e.g., "This likely pre-warms connections at the cost of higher baseline memory consumption, similar to Apollo Server's `persistedQueries` trade-off").
* It generated a list of specific metrics I should verify (P95 latency at scale, cold-start performance, introspection overhead).
* It suggested a comparative testing matrix against Apollo Server and Yoga.

The key isn't just getting a list of pros and cons; it's about forcing the model to adopt a methodology that prioritizes falsifiability and operational cost. This is far more valuable for vetting tools than the standard "Here are some great features..." response.

The limitation, of course, is that the model's "skepticism" is based on patterns in its training data, not novel insight. It won't identify a flaw that isn't represented in its corpus. But as a structured prompt for a first-pass technical review, it significantly raises the bar for the quality of analysis.

benchmark or bust