Library · Data Practice OS
ADR-001: One platform by default, two only by workload
Decision record · Proposed · Source reviewed
Status: proposed, awaiting the organisation architecture approval.
Context
The organisation is building an AI-ready data practice and may migrate from one or more legacy warehouses (the source is not decided yet). Both Snowflake AI Data Cloud and Databricks Data Intelligence Platform are credible targets, and both now cover SQL analytics, engineering, ML and GenAI. Vendors on each side will argue that their platform does everything. Running two platforms doubles the governance surface, the skills needed and the FinOps effort. Picking one too early can leave a real workload class poorly served.
Decision drivers
- Fit to the workloads the organisation actually has, measured with Platform selection scorecard
- One governance model, so a classification tag means the same thing everywhere (Governance, security and trust)
- Skills the organisation can hire and keep in its local market
- Cost that is predictable and attributable
- No lock-in on the data itself (see ADR-002: Apache Iceberg as the interchange table format)
Options considered
| Option | For | Against |
|---|---|---|
| A. Snowflake only | Simplest to operate; strong SQL, BI serving, sharing and governance; business-user AI (Cortex Analyst, Cortex Agents and Snowflake Intelligence) | Heavy streaming, custom ML and very large unstructured workloads are possible but less natural |
| B. Databricks only | Strong engineering, streaming, ML and custom agents; open formats natively | BI serving and business-user self-service need more design; SQL-only teams face a steeper learning curve |
| C. Both, split by workload | Each workload class runs where it fits best | Two of everything: contracts, security reviews, skills, bills. Only worth it above a certain scale |
| D. Both, teams choose freely | No arguments | Duplicated data, divergent definitions, ungoverned spend. Rejected |
Decision
Score the candidate workloads with Platform selection scorecard, and choose one platform unless the scorecard shows a material workload class that the leading platform serves poorly. If both are justified, use this fixed split:
- Snowflake: governed SQL analytics, BI serving (Power BI), cross-agency sharing, and business-user AI over curated data.
- Databricks: heavy or streaming engineering, ML training and serving, custom agents, and unstructured data at scale.
Under either answer the same rules apply: dbt is the transformation layer (ADR-003: dbt as the transformation standard), Apache Iceberg is the interchange format, and each table has one catalog of record (ADR-004: One catalog of record per table).
Consequences
- A single-platform answer is a success, not a compromise. The interop decisions keep the option to add the second platform later at low cost.
- If both are adopted, the Platform team owns both paved roads and one shared governance model. Every new workload is placed by the split above, not by team preference.
- FinOps reporting must show cost per platform per data product from day one (Cost control on Snowflake, Cost control on Databricks).
Revisit when
- A workload class grows beyond what the chosen platform handles economically
- Either vendor's capability or pricing changes materially (review yearly)
- Whole-of-government or enterprise-wide purchasing arrangements change the commercial picture