Having recently migrated a significant portion of our internal tooling and data pipelines to leverage Braintrust for AI development, my team has encountered substantial friction with the platform's permissions and role-based access control (RBAC) system. The documentation, while comprehensive in listing capabilities, lacks the concrete operational patterns necessary for implementing a secure and scalable multi-team environment. Our use case involves separate data science, ML engineering, and QA teams requiring distinct levels of access to projects, experiments, datasets, and the evaluation framework.
The core confusion stems from the interplay between **Organization-level roles** (Admin, Member) and **Project-level roles** (Admin, Writer, Reader). The inheritance model and the exact permissions boundary are not explicitly clear from the UI alone. For instance:
* Does a Project Admin have the ability to modify project membership, or is that solely the purview of an Organization Admin?
* Can a Project Writer invite new users to the project with a Reader role, or is that an admin-only function?
* What is the operational difference between an Organization Member and a Project Reader when it comes to accessing datasets linked across projects?
To ground this discussion, I'll outline our attempted structure and the specific roadblocks we hit. We aimed for a setup where:
* **ML Engineers** are Project Admins for their respective service domains, managing the full experiment lifecycle.
* **Data Scientists** are Project Writers, able to create and run experiments but not alter project settings or user access.
* **QA Analysts** are Project Readers across all projects, tasked with executing evaluations and viewing results without the ability to modify code or prompts.
Our configuration attempt via the CLI (`br` tool) looked something like this for project-level assignment:
```bash
# Attempting to grant Writer role to a user on a specific project
br projects users add --project "project-uuid" --user "user@company.com" --role "writer"
```
However, we observed inconsistent behavior where some users with the Writer role reported inability to create new experiments under certain conditions, which we suspect may be related to dataset permissions existing outside the project scope.
I am seeking detailed insights from other teams running Braintrust in production, particularly those with complex, multi-tenant requirements.
* What is your functional mapping of Braintrust roles to actual engineering and research personas?
* How do you manage permissions for shared datasets and evaluation suites that span multiple projects? Do you centralize these under a specific team's project or manage them at the org level?
* Have you encountered limitations that forced you to work around the native RBAC, perhaps by creating custom scripts or over-granting Organization Admin privileges?
* Are there any best practices for auditing access changes and maintaining security compliance, given the current tooling?
A comparative analysis with the permission models of other platforms we've used (like MLflow, Weights & Biases, or even cloud provider IAM) would be particularly valuable to understand the conceptual framework the Braintrust team intended.
Data over dogma
Yeah, that inheritance part really got me too when I started. I set someone as a Project Admin, but they still couldn't add users and had to come back to me, the Org Admin. So, from what I figured out, only Org Admins can actually manage memberships, even for projects. Is that what you're seeing?
For your separate teams, did you find a way to give the QA team read-only access to a project without making them Org members first? That's my next hurdle.
That's exactly right - only Org Admins can add or remove people, even from projects. It's a central governance choice that tripped us up at first too.
For your QA team, you can add them as external collaborators with the Reader role. They won't be Org members, but they'll have access to that specific project. Just go to the project settings and add their email addresses there. It works well for contractors or partner teams who don't need full platform access.
The catch is they'll still need an account on the platform, just not an Org membership. Have you tried the external collaborator route yet?