I've been conducting an extended evaluation of Notion AI's feature set as part of our firm's broader procurement review of AI-assisted productivity tools. A key use case we're assessing is ideation and initial content structuring. After rigorous, repeated testing of the 'brainstorm' feature across multiple prompt types and project contexts, I've observed a significant pattern of output redundancy that calls into question its utility for professional, iterative work.
My methodology involved creating a controlled test environment with distinct project pages. I executed the brainstorm command for prompts including:
* "Blog post topics for a B2B SaaS company focusing on compliance"
* "Innovative team meeting formats for a remote engineering team"
* "Potential features for a project management software update"
Across 15+ iterations per prompt category, the results consistently converged. For the B2B SaaS blog topics, for instance, the output cycled through minor variations of:
1. The Importance of SOC 2 in Vendor Selection
2. How to Conduct a Security Risk Assessment
3. Building a Compliance-First Culture
4. The True Cost of Non-Compliance
5. Future Trends in Data Privacy Regulations
Attempting to prompt for a "second layer" of ideas ("brainstorm more unconventional topics") simply yielded slight rephrasing of these core five, such as "Why SOC 2 Matters" or "Understanding Risk Assessment Frameworks." The lack of depth and genuine variety suggests the underlying model may be working from a constrained, optimized list for perceived high-value topics rather than generating novel combinatorial ideas.
This presents a tangible procurement and vendor evaluation concern. If the tool's output lacks sufficient diversity, its long-term value diminishes, impacting the ROI calculation for the annual seat commitment. It raises questions about the training data and the specific fine-tuning applied to the 'brainstorm' function.
I'm interested to hear if other members with a process-oriented approach have documented similar findings. Specifically:
* Have you identified any prompting techniques or context-setting methods that reliably produce a broader, less recycled idea set?
* In your vendor comparisons, have you found competing tools (e.g., within ClickUp, Coda, or standalone AI tools) demonstrate measurably better variance in ideation phases?
* Does this limitation factor into your renewal considerations or negotiation points regarding feature development roadmaps?
- Due diligence first.