Skip to content

Library · Data Practice OS

Start here

Reference guide · Published · Source reviewed

The data practice exists to be the backbone of knowledge and decisions for your organisation: the trusted facts, shared meaning, versioned knowledge and approved rules that decisions stand on, and the record of what was decided and what happened next. The From data to decisions - the decision stack sets out those layers.

This is the working guide to the engine underneath: an AI-ready data practice on Snowflake, Databricks, or both. It covers how to modernise what we have and how to build AI-native on it. It is not a vendor brochure. Every recommendation says when to use something and when not to.

How the site is organised

Collection What it is How to read it
The path (this playbook) Ten pages, from "what AI-ready means" through the decision stack to the roadmap In order, once
Handbooks Deep references: Snowflake, Databricks, interop, engineering, AI, knowledge and decisions, migration Dip in when you need one
Decisions Architecture decision records: what we chose, why, and when to revisit Before you argue with the architecture
Templates Scorecards, canvases, checklists to copy When you start real work

On the map, every page, term and tool is a bubble. The lines are real links between them. Click a bubble to see what it leans on.

The path

  1. What AI-ready means: the definition we hold ourselves to, and how to test for it.
  2. From data to decisions - the decision stack: what the practice is for, from facts to outcomes, in seven layers.
  3. Target reference architecture: the engine's layers, and which parts are portable.
  4. Choosing a platform - Snowflake, Databricks, or both: Snowflake, Databricks, or both, decided by workload.
  5. Modernisation and migration: getting off legacy without a big-bang cutover.
  6. Governance, security and trust: classification, access, privacy and AI risk, by policy.
  7. AI-native patterns: the six patterns that cover most AI work on data.
  8. Operating model: who owns what, and how teams ship without tickets.
  9. Roadmap and maturity: what to do in the first 90 days, and after.

Ten things to believe by the end

  1. AI-readiness is a property of data products, not of a platform. Buying Cortex or Agent Bricks does not make poorly described data usable by a model. See AI-ready data.
  2. Most of the stack should be portable. Open table formats, dbt and a single semantic layer mean the platform choice can change without rewriting the practice. See ADR-002: Apache Iceberg as the interchange table format.
  3. One platform is a perfectly good answer. Choose two only when the workload mix pays for the second set of skills, governance and invoices. See ADR-001: One platform by default, two only by workload.
  4. Migrate the business rules, not the code. The value in legacy is the logic. Extract it as tests first, then conversion becomes verifiable. See Extracting business rules.
  5. Reconciliation is the definition of done for a migration, not "the pipeline runs".
  6. Governance is code. Tags, policies and grants live in git and apply automatically, or they drift. See Data as code.
  7. Evaluation is the release gate for AI. No evaluation set, no production. See Evaluating AI systems.
  8. Cost is a design input. Both platforms bill on consumption, so every workload has an owner and a budget from day one. See FinOps.
  9. Start from the decision, not the dataset. Choose what to build from a Decision inventory: the decisions with volume and consequence, and the layers each is missing. A knowledge graph, a model or a new platform is a technique for a decision, not the goal. See From data to decisions - the decision stack.
  10. Decisions are data too. Each system-supported decision leaves a Decision record of the facts, rule version, authority and outcome, so it can be explained, replayed and learned from. See Decision records and provenance.

Who this is for

What this site is not

  • Not a decision already made. The ADRs are proposed until approved through the organisation architecture governance.
  • Not a replacement for vendor documentation. Product capability, region availability and certifications change monthly. Pages say "verify" where it matters and carry a reviewed date.