Library · Data Practice OS
Choosing a platform
Reference guide · Published · Source reviewed
Short answer. Both platforms can now do almost everything. The real differences are in defaults: what is easy, what your people already know, and how you pay. Choose by workload mix and team skills, then make the choice cheap to revisit by keeping storage, transformation and semantics portable (Target reference architecture).
Where each one is strongest
| Workload | Leans Snowflake | Leans Databricks | Notes |
|---|---|---|---|
| SQL analytics and BI serving at high concurrency | Strong default: minimal tuning, mature concurrency scaling | Capable with serverless SQL warehouses | Both connect natively to Power BI |
| Business users asking questions in natural language | Cortex Analyst / Snowflake Intelligence | AI/BI Genie | Both depend on the semantic model. Pilot both on the same data |
| Batch ELT in SQL with dbt | Strong | Strong | Tie. Choose on team familiarity |
| Streaming and near-real-time (traffic sensors, incident feeds) | Snowpipe Streaming, Dynamic Tables (seconds to minutes) | Structured Streaming, Lakeflow Declarative Pipelines | Databricks is more flexible for complex stateful streaming |
| Large-scale Python and Spark engineering | Snowpark (good, improving) | Native home of Spark | Existing Spark or PySpark skills point to Databricks |
| Data science and custom ML (crash risk, asset deterioration) | Snowflake ML, notebooks, container runtime | Mature MLflow, feature engineering, GPUs | Databricks is the deeper toolset for ML teams |
| LLM functions over tables (classify, extract, summarise) | Cortex AI SQL | AI Functions | Both good. Cost model differs; benchmark on your data |
| RAG over documents | Cortex Search (very low effort) | AI Search + Agent Bricks (more control) | Snowflake is faster to start; Databricks offers more control over customisation |
| Custom agents with tools and evaluation | Cortex Agents | Agent Bricks + MLflow evaluation | Databricks has the richer agent-engineering lifecycle today. Verify, as this moves quickly |
| Sharing with other agencies and partners | Secure Data Sharing, Marketplace listings | OpenSharing (open protocol, formerly Delta Sharing) | Snowflake is the most frictionless where the consumer has Snowflake |
| Operational app backing store | Hybrid tables, Snowflake Postgres | Lakebase | Both are new. Prefer a proven OLTP database unless the app is data-centric. Snowflake Postgres (Postgres 16-18) runs in AWS Sydney and Azure Australia East but not on Google Cloud; confirm it is inside the IRAP scope before use (checked 6 Nov 2026) |
| Governance for SQL-centric estates | Horizon: simple RBAC + tags + policies | Unity Catalog | Both strong. Unity also governs files, models and functions in one place |
| Lowest operational overhead | Snowflake: almost no infrastructure to manage | More knobs (clusters, runtimes), increasingly serverless | Matters if the platform team is small |
Where a cell says strong default, the other platform can still do it. You will just work harder or pay more.
Three valid answers
A. Snowflake only. Fits when the estate is mostly SQL analytics and reporting, the team is SQL-first, sharing with other agencies matters, and AI use is mostly business-user Q&A, document search and LLM functions in pipelines. This gives the lowest operational overhead.
B. Databricks only. Fits when there is heavy engineering, streaming or data science; a Python- or Spark-skilled team; custom ML and agent engineering as a core capability; and a preference for open formats as the native storage.
C. Both, split by workload (ADR-001: One platform by default, two only by workload). Fits when there are distinct communities with distinct needs, for example enterprise reporting and sharing on Snowflake and an engineering, data science and agents group on Databricks, and the scale justifies two platform skill sets.
What "both" really costs
Running two platforms is not twice the licence. It is twice the:
- Skills: two sets of admin, security, cost and performance expertise.
- Governance: two policy engines to keep in sync, unless one Catalog is the record per table (ADR-004: One catalog of record per table).
- Security assessments: two sets of IRAP evidence, network designs and identity integrations.
- FinOps: two billing models, two sets of FinOps dashboards.
- Integration: Iceberg interop works, but it is a design to maintain, not a checkbox (Apache Iceberg as the interchange format).
Rule of thumb: choose both only if you can name at least two sizeable workloads where the second platform wins clearly and a team that will own it. Otherwise pick one and use open formats to keep the option open.
Non-negotiables for either
- Data residency: storage and processing in Australian regions, including LLM inference.
Both vendors offer AWS and Azure Australian regions. Check which AI features and models are
available in-region (Snowflake model availability). For SENSITIVE data, pin AI routing to Australia: on Snowflake set
CORTEX_ENABLED_CROSS_REGION = 'AWS_AU'explicitly, because Sydney has no native Claude models and new organisations default toANY_REGION(cross-region inference). - IRAP: Snowflake's Australian systems are assessed at PROTECTED in AWS Sydney, Azure Australia East and Google Cloud Melbourne (Snowflake IRAP, regions). Neither page names an edition or lists in-scope services. Databricks is assessed at IRAP PROTECTED in AWS ap-southeast-2, Google Cloud Sydney and the Azure Australian regions (Databricks IRAP trust page), and supports IRAP workloads through the compliance security profile, with a per-feature availability list for ap-southeast-2 (Databricks IRAP). Confirm the edition, profile and per-service scope with each vendor before committing. See the ISM.
- Identity: SSO and SCIM from the organisation's identity provider, and no local users.
- Network: private connectivity (PrivateLink or equivalent), no public endpoints for production.
- Exit: data in open formats on organisation-controlled storage, transformation in dbt, semantics in a portable layer.
Model choice is not platform choice
Anthropic Claude and other frontier models are available on both platforms (Cortex, Model Serving) and via Amazon Bedrock. Picking a model should not force a platform. Route all calls through an AI gateway (ADR-005: Governed model access through an AI gateway).
How to decide
- List the top 10 workloads for the next 24 months, with volume, latency, users and skills.
- Score each platform with Platform selection scorecard.
- Run a two-week bake-off on one real data product and one AI use case per platform. Measure effort, quality (using an evaluation set), cost per run and admin overhead.
- Record the decision as an ADR, including what would make you revisit it.
The handbooks go deeper: Snowflake at a glance, Databricks at a glance, Apache Iceberg as the interchange format.