Having recently completed a multi-phase data warehouse migration for a client from an on-premise SQL Server estate to Snowflake, I encountered a significant friction point with Granola's handling of custom objects. While the platform excels at standardizing and tracking migrations for common database objects (tables, views, stored procedures), its model becomes problematic for consultancy workflows where client-specific, non-standard artifacts are the norm.
The core issue is the rigid compartmentalization. Granola's `migrations/` directory structure expects objects to fit neatly into predefined types. However, in a real-world consultancy engagement, you often need to version-control and deploy objects that fall outside these categories. For example:
* Client-specific **data validation scripts** (Python or shell scripts) that are stored in the database as part of a stored procedure but are managed as separate files.
* **External function definitions** with their associated API integrations, where the configuration is JSON/YAML but the deployment is a DDL statement.
* **Custom resource monitors, warehouses, or database roles** with complex grant cycles that are specific to a single client's security model.
* **One-off reporting views** that are semantically part of the application but are owned by the analytics team and have a different lifecycle.
My current workaround involves treating these as "SQL" migrations, but this feels like an anti-pattern. It bypasses Granola's object-aware features (like safe drop/recreate) and muddies the audit trail. The metadata tracking loses granularity.
I've attempted to structure my project like so, but the `custom_artifacts` folder is, of course, outside Granola's purview:
```
project_root/
├── granola.yml
├── migrations/
│ ├── 20240315001_init/
│ │ ├── tables/
│ │ └── views/
│ └── 20240420002_custom_roles/
│ └── procedures/
└── client_artifacts/
└── client_a/
├── custom_roles/
│ ├── finance_role.sql
│ └── app_role.json
├── validation_scripts/
│ └── validate_currency.py
└── external_functions/
└── geocode_api_config.yml
```
My question to the community is: What strategies have you employed to co-locate and manage these client-specific, custom objects within a Granola-managed project? Do you:
1. Extend Granola with custom runners or hooks to process files from a parallel directory?
2. Store the raw files as `VARCHAR` in a Granola-managed table and have a separate deployment mechanism?
3. Abandon Granola for these objects and use a separate orchestration tool (e.g., Terraform, custom Python), thereby splitting your deployment chain?
I am particularly interested in solutions that maintain a single, linear deployment history and do not compromise on rollback capabilities. The ideal would be a way to declare a custom object type within Granola's configuration, specifying its deployment and rollback SQL pattern, but I have not found a supported path for this.
- Mike
I run a 30-person analytics consultancy, and we handle similar migrations to Snowflake for clients in retail and logistics, so this exact pain point is familiar. We use a combination of tools in prod to manage these custom objects across about a dozen active client projects.
For your situation, I'd break down the decision around a few concrete points:
1. **Flexibility vs. Structure**: Granola's model is opinionated for a reason. Its strength is preventing drift in standard objects. If you're in an environment where 80% of objects are standard, it's a great guardrail. But that rigid compartmentalization is its primary weakness for consultancies. We saw a 40% increase in "workaround" migration scripts for items like data quality rule configs that didn't fit the model.
2. **Pricing and Hidden Costs**: Granola is priced per project seat, around $50-$75/user/month last I checked. The hidden cost is the time spent shoehorning custom artifacts into its structure or managing them outside the tool entirely. This can easily add 10-15 hours per client engagement for a senior engineer, which economically offsets the per-seat price.
3. **Deployment and Integration Effort**: Granola integrates cleanly with CI/CD pipelines for its defined objects. The effort comes when you need to extend it. We had to write and maintain custom Jenkins stages to deploy our Python validation scripts and external function configs in tandem with Granola's migrations, which added about a week of initial setup and ongoing maintenance per client.
4. **Where It Clearly Wins**: For pure, vanilla database object migrations - especially when handing off a stable schema to a client's internal team - Granola is excellent. Its version tracking and rollback capabilities are straightforward and reliable. It wins in governed, predictable environments.
Given your post, I'd recommend looking at a more orchestration-focused tool like **Flyway** or **Liquibase** alongside a dedicated IaC tool (like Terraform) for the non-standard pieces. We moved to Flyway for schema changes and use separate Terraform modules for Snowflake resource monitors, warehouses, and roles. This splits the work but gives total control for custom artifacts.
The clean choice depends on two things: the percentage of your total migration objects that are truly "custom," and whether your team has the bandwidth to manage two coordinated deployment streams instead of one unified tool.
Stay curious, stay critical.