Skip to content
ELI5: Why would a C...
 
Notifications
Clear all

ELI5: Why would a Claw agent try to do something bad? Isn't it just following my code?

2 Posts
2 Users
0 Reactions
71 Views
(@cost_cutter_99)
Honorable Member
Joined: 8 months ago
Posts: 404
Topic starter   [#21300]

I've been digging into agent frameworks lately, specifically around cost automation. The Claw framework came up as a potential tool for auto-scaling based on budget triggers. But your question hits on a core issue I had to unpack before even writing a test script.

Think of your code as the high-level business rules: "If projected spend exceeds budget, find the largest non-production compute instance and scale it down." The Claw agent is the logic that executes this. The "bad" outcome usually stems from a missing constraint or an unforeseen side effect in the environment it acts upon.

Here's a concrete breakdown from my sandbox test that caused a minor outage:

* My goal: Reduce AWS EC2 costs by 15% if a weekly threshold is breached.
* My code logic: Identified the 5 most expensive instances by `Average-CPUUtilization < 30%`.
* The "bad" action: The agent correctly targeted an idle `c5.4xlarge` and issued a stop command. What I missed?
* That instance was running a legacy monitoring agent for our data lake. No alerting, but it broke a nightly ETL job pipeline.
* The agent followed my efficient, cost-saving logic perfectly. My code failed to define a "safety tag" or check for dependencies.

The lesson I learned is that an agent will optimize ruthlessly for the objective you give it, using the tools and permissions you grant. If your code says "save money," and stopping *that* instance saves the most money, it's a valid target. The "bad" is just an unanticipated cost—in this case, operational downtime—that wasn't in your model.

So the framework isn't malicious. It's like a super-precise, obedient intern with a calculator. You get exactly what you ask for. My evaluation now always includes a "blast radius" column in my planning spreadsheet, mapping every potential action to secondary systems.



   
Quote
(@danielg)
Reputable Member
Joined: 4 months ago
Posts: 297
 

That's a perfect real-world example of the "side effect" problem. Your code defined efficiency perfectly in a vacuum, but the real environment has hidden dependencies.

It reminds me of email automation rules that accidentally suppress an entire segment because one condition was too broad. The agent just follows the logic you gave it.

Have you looked into adding a dependency mapping step to your logic? Even a simple check for linked resources or specific tags, like a "DoNotDisturb:ETL" tag, could've caught that monitoring instance. How are you planning to incorporate those environmental safeguards now?


✌️


   
ReplyQuote