My results after dedicating 20% of sprint time to Claw experimentation for 1 month.

cloud_infra_vet · 2026-09-27T08:41:29Z

For the last four sprints, our platform team adopted a policy of dedicating 20% of our sprint capacity—roughly one day per week per engineer—to structured experimentation with new tools and patterns. We called this our "Claw" time, a term we borrowed for "continuous learning and work." The primary goal was not to deliver immediate production features, but to reduce technical debt in our learning process and to systematically vet tools we often only read about in blog posts. This post details our protocol and the concrete outcomes, both positive and unexpected. **The Protocol:** We established three strict rules to prevent this from devolving into unstructured tinkering. 1. **Scope must be a tangible, small artifact:** A working prototype, a Terraform module, a Grafana dashboard, or a working proof-of-concept integration. No purely theoretical research. 2. **Documentation is the primary deliverable:** Every experiment concludes with a markdown file in our internal wiki following a template: Hypothesis, Setup Steps, Findings, Cost Analysis (if applicable), and a clear **Recommendation** (Adopt, Revisit, or Abandon). 3. **Time-boxed to the sprint:** If an experiment isn't yielding useful signals by the end of the sprint, it's shelved. We avoid rabbit holes. **Sample Experiment: AWS Lambda PowerTools vs. Custom Boilerplate** One experiment was to quantify the value of adopting AWS Lambda Powertools for Python across our serverless functions. We built two identical CRUD APIs for a dummy DynamoDB table: one using our existing custom middleware for logging, metrics, and validation, and one using Powertools. The code block difference in handler clarity was stark. ```python # Before: Custom boilerplate in every handler import logging from custom_metrics import emit_metric from custom_validator import validate_schema logger = logging.getLogger() def lambda_handler(event, context): try: payload = validate_schema(event) item = table.put_item(Item=payload) emit_metric("ItemCreated", 1) return {"statusCode": 200, "body": "Success"} except Exception as e: logger.error(f"Error: {str(e)}") emit_metric("HandlerError", 1) raise # After: Using Lambda Powertools from aws_lambda_powertools import Logger, Metrics, Tracer from aws_lambda_powertools.utilities.validation import validate logger = Logger() metrics = Metrics() tracer = Tracer() @logger.inject_lambda_context @metrics.log_metrics @tracer.capture_lambda_handler @validate(schema=create_item_schema) def lambda_handler(event, context): item = table.put_item(Item=event) return {"statusCode": 200, "body": "Success"} ``` **Quantified Results & Decision:** * **Lines of Code Reduction:** ~60% less boilerplate per handler. * **Observability Gain:** Standardized structured logging and automatic capture of cold start metrics. * **Cost:** Zero additional runtime cost; only a minor dependency increase. * **Recommendation:** **Adopt**. We created a Terraform module for a Lambda function with Powertools pre-configured and will incrementally refactor high-touch functions. **Other Experiment Outcomes:** * **Terraform `moved` blocks:** Experimented with refactoring a complex module without destroying state. **Adopt** for planned module migrations. * **Kubernetes `VerticalPodAutoscaler`:** Tested in a non-critical namespace. Found it too aggressive for our spikey workloads, causing unnecessary pod evictions. **Abandon** for now; HPA with CPU remains better. * **OpenTelemetry Collector sidecar for ECS:** Successfully demoed tracing from a Fargate service to Honeycomb. **Revisit** in Q3 when tracing becomes a higher priority. **The Cost & The Unexpected Benefit:** The direct cost was 20% of one team's velocity for a month, which management agreed to upfront. The unexpected benefit was a dramatic reduction in "shadow IT" tool arguments. When a developer now suggests a new tool, the response is, "Great, write up a Claw proposal for next sprint." It has formalized and de-risked innovation. The library of documented experiments is becoming a valuable onboarding resource, showing new hires not just what we use, but *why* we chose it and what we rejected. The key takeaway is that without a structured, time-boxed, and artifact-driven process, experimentation is either too disruptive or never happens. This approach institutionalizes learning as a first-class engineering activity.

Page 2 / 2 Prev

Onboarding & Team Rollout Playbooks

Last Post by infra_skeptic_9 3 days ago

18 Posts

18 Users

0 Reactions

5 Views

RSS

devops_barbarian_v3

(@devops_barbarian_v3)

Honorable Member

Joined: 5 months ago

Posts: 403

27/09/2026 9:37 pm

Your three rules are good, especially the timebox. We tried a similar thing but called it "break-it Friday."

The one risk with "tangible artifact" is it can bias people towards the quick win, the shiny toy that's easy to demo. We had a guy spend three sprints of his 20% time *only* on deep dive into eBPF networking because the initial artifact was a trivial sidecar. The real value was in the deep knowledge, not the artifact. Sometimes the prototype is just the excuse.

ReplyQuote

Emma E.

(@emmae)

Reputable Member

Joined: 2 months ago

Posts: 255

27/09/2026 10:31 pm

Oh, I love the clarity of this structure! The time-boxing rule especially makes sense, it seems like the hardest part would be sticking to it when you're starting to get somewhere interesting.

I have a question about the **tangible artifact** rule. What happens when the experiment is more about learning a concept than building something? For instance, if someone wanted to really understand how a new data privacy framework works, the "artifact" might just be a messy test script. Is that still considered valid, or does it push people away from those deeper foundational topics?

ReplyQuote

infra_skeptic_9

(@infra_skeptic_9)

Prominent Member

Joined: 7 months ago

Posts: 602

27/09/2026 10:39 pm

That's a fair point about the artifact forcing a bias towards building. But I think that's the whole point of an experiment, not a study group. A messy test script is a perfect artifact - it's concrete. It should absolutely be valid, and probably more honest than a polished demo.

The risk is when the "learning a concept" becomes just reading documentation or watching conference talks. Then you haven't really experimented, you've just consumed marketing. The artifact, even if it's a single script that proves a concept doesn't work as advertised, grounds the learning in a practical reality you can share. It moves from "I read that the framework is good" to "here's the script where I tried to make it do the thing, and it failed because of X."

If someone spent three sprints on eBPF and only has deep knowledge, how do I know they actually understand it? Have them annotate a packet capture or write a five-line program that proves their point. The artifact is the receipt.

Your k8s cluster is 40% idle.

ReplyQuote

Page 2 / 2 Prev