Library · Data Practice OS
Platform selection scorecard
Template · Published · Source reviewed
Use this with Choosing a platform - Snowflake, Databricks, or both to make ADR-001: One platform by default, two only by workload concrete. Score per workload class, not for the platform overall. A platform can win BI serving and lose streaming ML.
How to use it
- List the workload classes the organisation actually has, with rough volumes (step 1 table).
- Agree the weights before anyone scores. Weights add to 100.
- Score each criterion from 0 to 5 per platform, for each workload class, with evidence: a proof-of-concept result, a vendor doc link or a reference customer. A score without evidence counts as 2.
- Weighted score = weight x score / 5. The maximum is 100 per workload class.
- Hard gates are pass/fail. A fail rules a platform out for that workload whatever its score.
- Adopt a second platform only if it beats the leader by 15 points or more on a workload class that is at least about 20% of expected spend or strategic value. Otherwise choose one platform.
Step 1: workload classes
| # | Workload class | Example in most organisations | Share of value/spend | Notes |
|---|---|---|---|---|
| W1 | Governed SQL analytics and BI serving | Registration and licensing dashboards in Power BI | ||
| W2 | Batch engineering (ELT) | Nightly loads from legacy systems | ||
| W3 | Streaming / near-real-time | Traffic sensor and incident feeds | ||
| W4 | ML training and serving | Crash-risk or demand models | ||
| W5 | Business-user AI (text-to-SQL, chat) | "How many heavy vehicle permits last month?" | ||
| W6 | Custom agents and unstructured data | Document extraction, policy RAG | ||
| W7 | Data sharing | Other agencies, NEVDIS, councils |
Step 2: hard gates (pass/fail per platform)
| Gate | Snowflake | Databricks | Evidence |
|---|---|---|---|
| Data stored and processed in an Australian region | |||
| Current IRAP assessment at the required level covers the services used (verify scope and date) | |||
| Required AI models are available in-region, or there is an approved alternative | |||
| Meets the applicable privacy law's obligations for the data involved | |||
| Integrates with the organisation identity (SSO, SCIM) and logging |
Step 3: weighted criteria
| Criterion | What "5" looks like | Weight |
|---|---|---|
| Workload fit | Native, proven for this workload in most organisations scale | 25 |
| AI capability | In-platform models, search, text-to-SQL and agents with governance and evaluation | 15 |
| Governance | Tag-driven Attribute-based access control, masking, lineage, audit; one policy model | 15 |
| Skills and operability | staff and the local market can run it; low operating toil | 15 |
| Cost model | Predictable, attributable per product, with controls (budgets, auto-suspend) | 10 |
| Interoperability | Open formats (Apache Iceberg), REST catalogs, sharing to other engines | 10 |
| Compliance depth | Beyond the gates: key management, privacy tooling, records retention | 10 |
| Total | 100 |
Adjust the weights for your organisation, but record why.
Step 4: scoring sheet (copy per workload class)
| Criterion | Weight | SF score (0-5) | SF weighted | DBX score (0-5) | DBX weighted | Evidence |
|---|---|---|---|---|---|---|
| Workload fit | 25 | |||||
| AI capability | 15 | |||||
| Governance | 15 | |||||
| Skills and operability | 15 | |||||
| Cost model | 10 | |||||
| Interoperability | 10 | |||||
| Compliance depth | 10 | |||||
| Total | 100 |
Worked example (illustrative only, not a recommendation)
The scores below are invented to show the arithmetic. Replace them with the organisation evidence.
W1 Governed SQL analytics and BI serving
| Criterion | Weight | SF | SF wtd | DBX | DBX wtd |
|---|---|---|---|---|---|
| Workload fit | 25 | 5 | 25 | 4 | 20 |
| AI capability | 15 | 4 | 12 | 4 | 12 |
| Governance | 15 | 5 | 15 | 4 | 12 |
| Skills and operability | 15 | 5 | 15 | 3 | 9 |
| Cost model | 10 | 4 | 8 | 4 | 8 |
| Interoperability | 10 | 4 | 8 | 5 | 10 |
| Compliance depth | 10 | 4 | 8 | 4 | 8 |
| Total | 100 | 91 | 79 |
W4 ML training and serving
| Criterion | Weight | SF | SF wtd | DBX | DBX wtd |
|---|---|---|---|---|---|
| Workload fit | 25 | 3 | 15 | 5 | 25 |
| AI capability | 15 | 4 | 12 | 5 | 15 |
| Governance | 15 | 4 | 12 | 5 | 15 |
| Skills and operability | 15 | 4 | 12 | 3 | 9 |
| Cost model | 10 | 3 | 6 | 4 | 8 |
| Interoperability | 10 | 4 | 8 | 5 | 10 |
| Compliance depth | 10 | 4 | 8 | 4 | 8 |
| Total | 100 | 73 | 90 |
Reading the example. Snowflake leads W1 by 12 points, under the 15-point threshold for that class. Databricks leads W4 by 17 points. If ML is 20% or more of expected value, the rule says adopt both on the default split. If ML is a minor workload, choose Snowflake and run ML on Snowflake ML, then rescore in a year.
Record the outcome
| Field | Value |
|---|---|
| Date scored | |
| Scorers (at least one engineer, one architect, one business owner) | |
| Result per workload class | |
| Decision and link to ADR | ADR-001: One platform by default, two only by workload |
| Rescore date |
Scoring rules learnt from options papers
- Gates are not offset by totals. Mark gate criteria and name the policy or person that requires each one. An option that fails a gate leaves the comparison.
- Unknowns are
?, not guesses. Publish no total while a must-have criterion is?. Mark assumption-based totals as provisional. - Run a sensitivity check. Does the order change when a contested weight moves, or when you remove the criterion that favours current work? In the platform team's layout paper, two options tied at 65/75, and removing two criteria reversed the order. The decision then rested on a stated principle, not on the scores.
- If the winner is not the recommendation, a criterion is missing. Name it and score it (often delivery risk or approval lead time).
The full method is in Options paper and fact check.