Skip to content
Notifications
Clear all

Has anyone tested Claw's API spec generation against the actual OAS validator?

3 Posts
3 Users
0 Reactions
21 Views
 danf
(@danf)
Estimable Member
Joined: 2 months ago
Posts: 168
Topic starter   [#20961]

I keep seeing these glowing reviews of Claw's "OpenAPI spec generation from database schema" feature. Everyone's so impressed it can spit out a YAML file. Great. Have any of you actually tried to *use* the output?

I'm not talking about a quick glance to see if it has `paths:` and `components:`. I mean running the generated spec through the official OpenAPI validator, or feeding it into a client generator like `openapi-generator` or `swagger-codegen`. Or, heaven forbid, trying to import it into something like Stoplight or Postman for actual contract testing.

My bet is you'll find a festival of non-compliance. I tried it on a moderately complex MySQL schema with some foreign key relationships and a few stored procedures it claimed to support. The resulting spec had:

* Incorrect `$ref` paths that didn't resolve (`#/components/schemas/MySchema` when the actual defined schema was under `#/components/schemas/MyTable`).
* Properties typed as `integer` for `VARCHAR` columns if the column name contained "id".
* `required` arrays that included every single column, nullable or not.
* Completely invented `example` values that didn't match the column's CHECK constraints or ENUM values.

The real kicker? It proudly generated `openapi: 3.0.3` at the top, but used syntax that was deprecated in 3.0.0. The validator threw more warnings than a linter on legacy code.

This isn't a minor nitpick. If the spec isn't valid, the entire value proposition collapses. You can't generate clients, you can't do automated contract testing, you can't use half the OAS tooling ecosystem. You just have a fancy, misleading text file.

So, before you post another "look at this cool spec I generated" screenshot, do the basic due diligence. Run `npm install -g @apidevtools/swagger-cli` and try `swagger-cli validate generated-spec.yaml`. Let's see those results. My guess is the failure rate is near 100% for anything beyond a trivial, two-table schema.


Anecdotes aren't data.


   
Quote
(@alexgarcia)
Honorable Member
Joined: 3 months ago
Posts: 496
 

You're making a critical distinction that gets lost in the marketing noise. The gap between "generates a YAML file" and "generates a *valid, usable* spec" is massive.

I've seen similar issues with their inferred enums - it often defaults to a generic string type when it can't perfectly map a CHECK constraint, which breaks downstream tools expecting the enum values. That invented example data is another subtle trap for testing.

Has anyone managed to get a clean validation pass from the official OAS tool, or is it always a patch job?



   
ReplyQuote
(@devops_contrarian_42)
Honorable Member
Joined: 6 months ago
Posts: 479
 

Exactly. The fanfare around "spec generation" is always about the initial dump. Nobody sticks around for the validation hangover.

You mentioned the `required` array including nullable columns. That's a classic sign of lazy inference. It's not mapping database constraints, it's just blindly listing every column. Makes the spec useless for any real client expecting proper partial updates.

If you can't even get the validator to pass, feeding it into a code generator is pure comedy. I tried the TypeScript client generation once. The broken `$ref`s just made it vomit empty interfaces.


Keep it simple


   
ReplyQuote