I've been reviewing a lot of code generation tasks lately, and a concerning pattern has emerged, especially around security-sensitive operations. The marketing for these "Claw" models emphasizes their reasoning capabilities, but that doesn't always translate to secure-by-default code.
Here's a reproducible case that made me raise an eyebrow. The prompt was straightforward: generate a Python function to hash a password using a salt.
**The Prompt:**
"Write a Python function to hash a password for storage. It should use a salt."
**The Generated Output (paraphrased):**
```python
import hashlib
import os
def hash_password(password):
salt = os.urandom(16)
return hashlib.sha256(salt + password.encode()).hexdigest()
```
**The Problem:**
While it technically uses a salt, this is **insecure and outdated**. It uses a fast, single-round SHA-256, which is vulnerable to brute-force attacks. The salt isn't stored with the hash, making it impossible to verify later. A proper solution should use a dedicated, slow key derivation function like `argon2id` or `bcrypt` via libraries like `passlib`.
**The Correct Answer (conceptually):**
A secure answer would recommend `passlib` or `bcrypt` and emphasize the need to store the salt *with* the hash. For example:
```python
from passlib.hash import argon2
def hash_password(password):
return argon2.hash(password)
```
The verification process is just as crucial.
This is more than a nitpick. It’s a fundamental failure to apply current security best practices, which is exactly what you'd hope a top-tier "reasoning" model would get right. It generated *working* code, but not *secure* code. In the B2B and SaaS space, where data breaches are catastrophic, this default behavior is a real liability.
Has anyone else run into similar issues with security-critical code generation? I'm particularly interested in cases involving authentication, encryption, or data sanitization where the model chose a simplistic or outdated approach.
Happy reviewing!
Keep it constructive.
Yeah, that's a classic example. I've seen similar stuff when testing different tools.
The real test is asking it to implement something like OAuth2 flow or secure file uploads. That's where they *really* stumble. The output often misses basic stuff like CSRF tokens or proper file validation.
It's all about prompting. If you just ask for a password hash, you get a bad one. But if you specifically say "use bcrypt with a work factor of 12," then it usually gets it right. The marketing sells the "reasoning," but you still have to give it the exact recipe.
Demo or it didn't happen