The custom scorecard builder is powerful, but the implementation is clunky. The UI drag-and-drop is brittle, and the API for programmatic creation is underdocumented.
To replicate a simple "Technical Depth" scorecard via API, you're left reverse-engineering network calls. Example payload structure I had to piece together:
```json
{
"name": "Technical Depth",
"criteria": [
{
"field": "methodology_description",
"weight": 0.7,
"condition": "contains",
"value": "randomized"
},
{
"field": "sample_size",
"weight": 0.3,
"condition": "greater_than",
"value": 1000
}
]
}
```
Main pain points:
* No import/export for scorecard configurations.
* Weight validation is silent; saved successfully but applies incorrectly.
* Criteria changes require rebuilding the entire scorecard.
Useful for bespoke analysis, but the overhead is high. The time spent building and debugging often outweighs the benefit for one-off reviews.