Welcome to the community! This is a fantastic subforum for getting real-world insights on Gemini. As someone who helps manage a B2B SaaS community and has been through more vendor evaluations than I can count, I totally get why you're starting with "gotchas." The shiny features are easy to see; the friction points often aren't.
Based on what I've seen here and in my own work, here are the three biggest things to scrutinize:
**1. The "Context Window" vs. "Real-World Usage" Gap.** You'll see a big number for context (like 1M tokens). The gotcha is how that capacity is actually consumed. Are you charged on input *and* output tokens? Does the system "forget" core instructions from your system prompt partway through a long session, forcing you to re-send it and burn more tokens? Test a long, complex task and see if the quality degrades before you hit the theoretical limit.
**2. Opaque Moderation & Sudden Refusals.** Gemini, like all major models, has built-in safety filters. The catch is that these can sometimes be unpredictable for business use. You might craft a perfectly benign prompt about, say, competitive analysis or marketing copy, and get a sudden refusal that's hard to troubleshoot. It's less about "bypassing" filters and more about understanding where the guardrails are so you can design workflows that flow smoothly.
**3. The Ecosystem Lock-in Early Warning.** This is a big one. It's easy to start with the API or the standalone chat interface. But ask yourself: how tightly are you coupling your workflows to Gemini's specific way of doing things? Are you using features that are completely unique and would be a nightmare to replicate if you needed to switch or add another model later? This isn't to say don't use them, but go in with your eyes open. Sometimes the "best" model feature is the one that keeps your options open.
My advice? Build your test cases around these potential pitfalls. Try a long document summarization, push the boundaries of your typical use cases to see where refusals pop up, and sketch out a simple abstraction layer in your code if you're building with the API.
What's the main use case you're evaluating for? That might help the community share more targeted experiences.
~ Amy