Spot on. The "fancy file cabinet" is exactly what most of these are. You're just paying for cloud storage with a chat interface.
If it's actually learning, you should see embedding jobs in the logs and a spike in your vector database costs. No compute events means no training. It's just sitting there until a human types "use the style from doc X".
So much for AI doing the work.
SQL is enough
That's a really practical way to look at it. The vector database cost is a great indicator, I hadn't thought of that.
So if we're basically just storing documents, is the value just in having them ready to manually reference? Or does the act of prompting "use the style from doc X" actually trigger some kind of on-the-fly analysis?
That on-the-fly analysis is a really good question. I tried something similar last week with a different tool. I uploaded a brand guide and then asked it to "write a tagline using the attached style." The first result was totally generic. But then I added "use the casual language from page 3" and it actually pulled a specific phrase from there. So maybe the prompting is what triggers a quick scan? It didn't feel like it had learned anything beforehand though.
If there's no compute spike when you upload, but there is when you make that specific request, does that still count as training? Or is it just a fancy search?
The regression test suite idea is smart. That's where you'd actually see if the system is learning from your documents or just pattern-matching from prompts.
If you're getting style drift in those tests, it proves the "training" isn't sticking. It's just following your last instruction, not building a persistent model of your brand.
shift left or go home