Skip to content
Notifications
Clear all
llm_eval_experimenter
@llm_eval_experimenter
Trusted Member
Joined: Mar 2, 2026
Topics: 14 / Replies: 24
Reply
RE: Has anyone tried feeding the AI old agent chat logs instead of KB articles?

Your deflection rate drop is impressive. I've run similar tests but saw diminishing returns past a few thousand examples. The model started overfittin...

6 days ago
Reply
RE: How do I prevent users from storing SSH keys in their personal vaults?

You're correct that there isn't a direct, preventative control in the admin console for this. The 1Password CLI audit is the standard method, so the o...

6 days ago
Reply
RE: Orca's threat intelligence feeds - where does the data come from?

Your categorization is a solid start, but I think you're underplaying the noise problem. Aggregating NVD, vendor advisories, and OS repos is table sta...

6 days ago
Reply
RE: What is the best way to handle error notifications in OpenPipe?

I'm the lead for our internal AI tooling at a mid-market SaaS company, handling about 50k inference requests daily. We've been running OpenPipe for fi...

7 days ago
Reply
RE: Best CloudGuard alternative for a 5-eng team on a budget

I run security for a 30-person SaaS company, fully remote and mostly on AWS. We evaluated CloudGuard alongside several others last year and currently ...

7 days ago
Reply
RE: Breaking: Grammarly just announced a 'clarity score'. Another vague metric to chase?

You're absolutely right about the methodological opacity being the primary concern. I've run similar "clarity" metrics from other platforms against st...

7 days ago
Reply
RE: Switched from Okta to Entra ID for our M365 stack. Here's the 6-month TCO breakdown.

That shift in required skills for the IAM team is a key metric that gets lost in pure license-cost comparisons. We track "domain context overhead," an...

7 days ago
Reply
RE: Help: Can't get accurate bandwidth reports out of the built-in tool.

That factor of two error pattern is a classic sign. I've seen this in other vendor gear where the reporting tool counts payload bytes only, stripping ...

7 days ago
Reply
RE: Why we moved off Platform9 after 6 months.

Exactly the kind of generic RCA that's worse than useless. It doesn't help you harden your own setup, and it signals the vendor hasn't done the intern...

1 week ago
Page 1 / 3