You're not using it wrong. The `detail_level` parameter is a verbosity control, like others have said, not a depth control. It can't invent the cost implications you're after if they're not in your source.
I run into this in procurement all the time. If I only feed a vendor's own RFP response into a summary tool, I get a polished repeat of their sales points. To get the nuanced trade-offs, I have to curate the input: the vendor's doc plus our internal security review notes, the pricing annex, and a third-party comparison blog.
For your case, try feeding the whitepaper alongside the AWS Pricing Calculator page for Lambda/Fargate and a known benchmark article on cold starts. You're manually building the "balanced corpus" the tool needs to collate from. It's extra setup, but it's the only way to get past the shallow, promotional layer.
Ask me about my RFP template
That compliance example is exactly the kind of nuance I was missing. It's not just about pulling together different documents, it's about the specific gap between a rule and its implementation in your environment.
When you mention mapping `aws_s3_bucket.logging` to the CIS control manually, it makes me wonder: could you trick the tool into doing that analysis by also feeding it a mapping document? Like a separate file that explicitly links Terraform resource attributes to control IDs? Or would it still just retrieve and rephrase those links without understanding the *absence* of the attribute in your actual code?
You've correctly identified the configuration, but you're expecting analytical synthesis from a retrieval tool. The `detail_level="high"` parameter only increases the word count of the retrieved text chunks; it doesn't enable critical thinking.
Your example with Terraform provider docs is telling. If you ask it to compare `aws_lambda_function` to `aws_ecs_task_definition` based only on the HashiCorp docs, you'll get a feature list, not an architectural trade-off. The tool lacks a model of "cost" or "performance" unless you provide documents where those concepts are explicitly discussed in relation to those resources.
For your AWS scaling question, you'd need to build a corpus that includes the opposing viewpoints: the whitepaper, the pricing pages, and third-party benchmarks. The tool can then retrieve and concatenate those points, but it won't infer the implications of spiky traffic on its own. That synthesis is still your job.
infra nerd, cost hawk
Exactly, and that's the crux of the vendor evaluation problem. Even if you build this ideal "corpus" of opposing viewpoints, you're assuming you already know which documents contain the critical counterpoints.
> The tool can then retrieve and concatenate those points
But who curates that list? The procurement team that already knows about the notorious cold start benchmark? This creates a blind spot. It reinforces known unknowns and misses the "unknown unknowns" a critic would spot. You've automated a search for what you already know to search for.
It's a polished echo chamber.
Question everything
You've correctly identified the configuration, but you're expecting analytical synthesis from a retrieval tool. The `detail_level="high"` parameter only increases the word count of the retrieved text chunks; it doesn't enable critical thinking.
Your example with Terraform provider docs is telling. If you ask it to compare `aws_lambda_function` to `aws_ecs_task_definition` based only on the HashiCorp docs, you'll get a feature list, not an architectural trade-off. The tool lacks a model of "cost" or "performance" unless you provide documents where those concepts are explicitly discussed in relation to those resources.
For your AWS scaling question, you'd need to build a corpus that includes the opposing viewpoints: the whitepaper, the pricing pages, and third-party benchmarks. The tool can then retrieve and concatenate those points, but the synthesis is just adjacency. It can't infer the cost implication of a spike unless a document explicitly states "spiky traffic costs more on Fargate due to minimum task count."
No free lunch in cloud.
Right, that detail_level parameter got me too when I first tried it. It's just about how much of the surface text it repeats, not depth.
You're hitting the same wall I did with CDN configs. If you only feed the vendor's own best practices doc, you'll get a summary of their marketing. For the cost trade-offs you want, you have to feed it the counterpoints manually - like that Cloudflare vs Fastly pricing blog from last year, or your own billing data.
It's a fancy search engine, not a critic. You have to build the debate yourself first.
measure twice, ship once
Oh, that's a really interesting test you did! The part about the outputs being structurally identical even with different retrieval methods is kind of eye opening. It really shows the limit is in the design, not just the data you feed it.
So even if you manually add the pricing page and a benchmark, it's still just stitching sentences together? It can't actually connect the dots to answer a real question like "can my budget handle this?"
That makes me think of trying to use a report tool in Salesforce. If you only pull in the "Opportunity" object, you'll just get a list of deals. To see if you'll hit quota, you have to be the one to build the relationship to "Forecasts" and "Quotas" and ask the right question. The tool just shows the data you already linked together.
I guess the AI is doing the same thing, just with text chunks instead of database objects.
You've perfectly described the financial calculus these tools miss. The Lambda pricing page and your CloudWatch logs are just data points. The insight is in the delta between them, which requires an actual cost model.
I ran into this with DynamoDB on-demand versus provisioned capacity. The tool could regurgitate the pricing for a write request unit, but it couldn't take a week of application metrics and plot the crossover point where provisioned capacity becomes 40% cheaper. You have to build that spreadsheet yourself, mapping the irregular burst pattern to the monthly commit.
It's the difference between a dictionary and a translator. The tool has all the words, but it can't construct the sentence "This pattern is expensive."
Always check the data transfer costs.
The dictionary versus translator analogy is spot on for explaining the synthesis gap. It's the same limitation I hit when trying to compare support platforms for SLA management. The tool could list that Platform A has "automated breach alerts" and Platform B has "customizable priority matrix," but it couldn't construct the sentence "With your current ticket volume, the 5-minute alert delay in Platform A will cause you to miss your P1 SLA 3 times per month."
You have to be the one to bring in the volume data and the SLA policy doc to model that outcome. The tool just parrots the feature lists side by side.
Support is a product, not a department.
Exactly right. The 'opposing corpus' idea is the key, but it reveals the workflow problem: that curation step is the actual analysis. The tool just plays it back.
It reminds me of the product comparison threads here. If you only read the official release notes, you'll think every update is a breakthrough. You have to bring in the changelog, the support ticket spikes, and a user's rant about a broken API. The insight is in the tension between them, which the tool won't create for you. You have to create the tension first.
Keep it constructive.
I just started using a similar tool for security policy reviews and hit the same wall. You're not configuring it wrong.
The detail_level seems to just pull longer text snippets, like others said. When I asked it to compare MFA methods from vendor PDFs, it listed "SMS vs app-based" but didn't mention the SIM swap risk I read about later. That nuance wasn't in my source docs.
It makes me wonder, if the tool can't generate the insight itself, how do you even know which counterpoint documents to go find and feed it? That's the real work.
That SIM swap example is exactly the kind of thing I'd miss. It's not just about pulling longer snippets, you're right.
So if the real work is finding those counterpoint docs, how do you even start? Like for my ECR image scan setup, do I go find breach reports first, or wait until the tool gives a shallow answer and then go digging? Feels like you need to know the risk before you can teach the tool about it.
Exactly. That's the core of the workflow problem. You have to do the manual research to find the pricing page, the cold start blog, and the benchmark. But if you already know to go find those, you've already identified the critical trade-offs. The tool just becomes a mirror.
I saw this testing CI/CD runners. To get it to tell me when self-hosted runners become cheaper than GitHub's, I had to first calculate the inflection point myself by pulling my own data. Feeding that analysis back in just gave me a rephrase of my own spreadsheet. The insight generation is an echo.
-- bb
The `detail_level` parameter is almost certainly controlling verbosity, not analytical depth. You're expecting a cost model from a text summarizer.
The gap you're seeing is because those whitepapers don't contain the answer to your real question. They describe mechanisms, not financial implications. To get the cost trade-off during spiky traffic, you'd need to feed it at least three distinct document types: the AWS pricing pages, your own CloudWatch log data (or a representative sample), and a third-party analysis on cold starts. The tool can't synthesize a relationship that isn't explicitly stated in the corpus you provide.
It's like asking a search engine "should I buy this car?" after only giving it the manufacturer's brochure. You have to supply the opposing documents first - the repair forums, the depreciation charts, the insurance tables. The tool won't go find that counter-narrative for you; it just reflects the dataset you've assembled.
connected
Your issue with the `detail_level` parameter is a common misconception. It almost certainly just controls the length of the retrieved text snippets, not the analytical depth or the model's ability to synthesize. The tool is doing exactly what it's designed for: extracting and rephrasing statements present in your source documents.
The nuance you're looking for isn't there because your source corpus lacks the necessary opposing data points. An AWS whitepaper will never tell you that Lambda's cost model falls apart under your specific spiky traffic pattern. To generate the insight you want, you'd need to curate and feed at least three separate document types: the AWS pricing pages (for the raw cost variables), a relevant set of your own or sample CloudWatch logs (for the traffic pattern), and a third-party analysis on cold start performance (for the latency trade-off). The tool cannot invent relationships between these disparate data sets; it can only reflect connections explicitly described within the text you provide.
You're essentially asking the tool to perform a financial modeling task, which is far beyond semantic search and summarization. The shallow output is a direct indicator that your source material is insufficient for the question you're really asking.