Hi everyone, I'm new here and honestly a bit nervous to post. I've been tasked with finding better documentation and knowledge management tools for our engineering team. We're all AWS/Terraform focused.
I keep seeing NotebookLM mentioned. The idea of grounding an AI in our own internal docs (like Terraform modules and runbooks) sounds perfect for cutting down repeat questions. But I'm worried about moving from a cool demo to a real, 10-person team workflow.
Has anyone actually deployed it in a similar production setting? I'm especially curious about:
* How you handle access control for different document sets.
* If the grounded answers are reliable enough for troubleshooting steps.
* Any weird cost surprises at small team scale.
I'm trying to avoid another tool that just becomes a graveyard. A simple example of how you structured a source document would be amazing. Thanks for any insights!
Good question. We've been running a NotebookLM pilot for a three-team platform group (12 engineers) for about four months, grounded in AWS design docs and Terraform module READMEs. The short answer is it's useful for high-level orientation but has critical gaps for production troubleshooting.
On your specific points: Access control is its biggest weakness. It's effectively a flat document repository per 'notebook'. We had to create separate notebooks for 'team-private' vs 'shared' content, which fragments the knowledge base. For a 10-person team, you'll likely manage this manually, which doesn't scale.
The reliability of grounded answers is highly dependent on source structure. It works well for declarative 'what is this module for' questions. For procedural troubleshooting, like a runbook, it often hallucinates step ordering or specifics. We found you must format sources with extreme clarity. A simple example that worked for us was a Terraform module doc structured as:
```
## Module: s3_log_bucket
### Purpose: Creates an encrypted S3 bucket for application logs with lifecycle policy.
### Inputs (variables.tf):
- bucket_prefix (string, required)
- retention_days (number, default: 90)
### Outputs (outputs.tf):
- bucket_arn
- bucket_name
### Common Errors:
- "Error: prefix must match regex" => ensure bucket_prefix uses hyphens not underscores.
```
Cost hasn't been a surprise at this scale; it's negligible. The real cost is the maintenance overhead to keep source documents clean and the risk of engineers trusting a plausible but incorrect synthesized answer for a critical issue. It reduces simple repeat questions but isn't a set-and-forget solution. You'll still need a human-in-the-loop verification step for any operational guidance it gives.
No free lunch in cloud.
I agree with user1218's assessment on the core limitations, but I'd push back slightly on one point. For a 10-person, AWS/Terraform focused team, I don't think the access control issue is the primary blocker. It's inconvenient, but manageable with a single 'team-global' notebook. The larger problem is the tool's inability to interact with live systems.
You mentioned troubleshooting steps. That's where NotebookLM falls apart. A grounded answer can recite a runbook step, but it can't tell you if your specific VPC peering connection is stuck in `pending-acceptance` or if a Terraform state file is locked. That real-time context is everything.
Consider a hybrid approach. Use NotebookLM as a structured FAQ for your module documentation, but pair it with a CLI tool that can query your actual AWS environment. For a team your size, you might get more mileage from a simple, well-organized Confluence space with strict ownership until the agentic aspects of these tools mature. The graveyard you're worried about is filled with tools that promised more context than they could deliver.
Boring is beautiful
The reliability issue for troubleshooting is key, and it stems from a fundamental architectural mismatch. NotebookLM treats documents as static sources of truth, but effective troubleshooting requires reconciling documented intent with live system state. Your runbook might say "check the VPC peering connection status," but the answer's value depends on querying the AWS API *right now*.
Regarding your request for a source document example, structure is everything. A poorly formatted runbook yields vague answers. For grounding Terraform modules, we found success by enforcing a strict template in the source markdown. This creates predictable, citation-friendly blocks.
```markdown
## Module: vpc_baseline
**Purpose:** Provisions a foundational VPC with three private subnets, NAT gateway, and flow logs.
**Input Variables:**
- `cidr_block` (string): The primary IPv4 CIDR block (e.g., 10.0.0.0/16)
- `enable_flow_logs` (bool): Defaults to true.
**Common Error:** State lock on `aws_vpc.this` often indicates a stalled `terraform apply` in another workspace.
```
This format gives NotebookLM clear, isolated statements to ground in, which improves answer precision for "what is" questions. It still can't tell you if a state lock exists *today*, but it will accurately cite the documented common error. The cost for 10 users is predictable, but the real expense is the engineering time to refactor all your docs into this machine-optimal format.
Single source of truth is a myth.