I've been tasked with implementing some data quality checks for a new pipeline we're building, and my team is evaluating Braintrust against a few other options (like Great Expectations and Soda Core). I'm trying to understand the actual developer experience for writing custom checks.
From what I've gathered, Braintrust uses a Python-first approach where you define checks as simple functions. This seems straightforward, but I'm nervous about how flexible it really is for complex scenarios. For example, in our current (messy) setup, I sometimes need to check that the ratio between two aggregated columns from different tables falls within an expected range.
In a platform like Great Expectations, I'd be defining Expectations, often with JSON configs, which feels a bit heavy. With Braintrust, would I just write a pure Python function that runs a query and returns a boolean? Could someone show a real-world example of a moderately complex custom check?
Here's a skeleton of what I *think* it might look like, but I'm unsure about the proper structure and how to integrate it:
```python
import braintrust
def check_revenue_ratio():
# I need to fetch data from two different sources/tables,
# calculate a metric, and see if it's within tolerance.
# Where does the database connection live? Is it injected?
total_rev = run_query("SELECT SUM(amount) FROM revenue")
refunds = run_query("SELECT SUM(amount) FROM refunds")
ratio = refunds / total_rev if total_rev > 0 else 0
return braintrust.check("refund_ratio", ratio, {"max": 0.02})
```
My main concerns are:
- How do you handle dependencies between checks?
- Is there a risk of writing inefficient checks that scan huge tables on every run?
- How does the feedback loop compare? Is it easy to see *why* a check failed, beyond just a pass/fail?
I'd love to hear from anyone who has moved from a more YAML/JSON-heavy framework to Braintrust's code-centric style. Does it make debugging easier, or does it just move the complexity into your Python scripts?