Library · Data Practice OS
Roadmap and maturity
Reference guide · Published · Source reviewed
Maturity model
| Level | Name | What it looks like |
|---|---|---|
| 1 | Fragmented | Data in silos and legacy warehouses; reports are the interface; access by request; no AI in production |
| 2 | Consolidating | A modern platform is live; the first products migrated; CI exists; classification has started |
| 3 | Productised | Domains own contracted data products; a semantic layer serves BI; policy-as-code; cost attributed |
| 4 | AI-enabled | Several AI use cases in production with evaluation gates; agents use governed interfaces; supported decisions leave decision records; legacy shrinking fast |
| 5 | AI-native | AI is part of how data is produced and consumed; most new products are AI-ready by default; decision outcomes feed evaluation and rule review; legacy decommissioned |
Most public-sector estates start at 1-2. Level 4 within 24 months is ambitious but achievable if migration and AI enablement run as one programme, not two.
How readiness bands relate to these levels. The AI readiness assessment (proposed; not an agreed decision) gives each assessed AI use a band. The levels describe the practice, and a band describes one scope, so read this mapping as indicative:
| Assessment band | Indicative level |
|---|---|
| Foundational (0-39) | 1-2 |
| Emerging (40-69) | 2-3 |
| Established (70-89) | 3-4 |
| Optimising (90-100) | 4-5 |
Level 4 needs its production AI uses at Established or better with every gate passed. A band is never production approval: the readiness gates are.
Cadence. Re-score every AI use in production quarterly with AI readiness scorecard (workbook v2), and also after any material change to a source, API, ontology, policy, workflow or model provider. Report gate status, unknowns and evidence confidence alongside the score.
Roadmap
Stage 0: Decide and set foundations (first 90 days)
- Run the platform bake-off and approve ADRs 001-006 (Choosing a platform - Snowflake, Databricks, or both, Platform selection scorecard).
- Land the platform baseline: accounts and workspaces, AU-region, private networking, SSO and SCIM, Terraform, CI. Build identity and classification first, behind three signed phase gates (Security foundations, the first 90 days, from a draft plan; not an agreed decision).
- Stand up the Paved road template v0.
- Assess the legacy estate (Migration assessment) and agree wave 1.
- Draft the first Decision inventory with business owners (Decision inventory).
- Choose one lighthouse decision, then the data product and AI use case it needs. Choose by decision, not by dataset (From data to decisions - the decision stack).
- Exit: platform live in dev and test; ADRs approved; wave plan agreed; lighthouse decision chosen with its outcome signal, baseline and evaluation set written.
Stage 1: Lighthouse (to 6 months)
- Build the lighthouse product end to end: ingest → dbt → contract → semantic → BI → AI use case.
- Migration wave 1: extract rules, convert, reconcile, parallel run, cut over.
- Classification and tag-based masking live for all wave 1 data.
- AI gateway live; first AI use case in production behind its evaluation gate.
- FinOps dashboards per product.
- Decision records written for every lighthouse decision, with outcomes captured (Decision records and provenance, Outcome feedback loops).
- Exit: first legacy workload decommissioned; one AI use case in production with every readiness gate passed (AI readiness assessment); the lighthouse decision's outcome measures reported against the baseline.
Stage 2: Scale the pattern (to 12 months)
- Waves 2-3; two or three domain data product teams formed.
- Semantic layer covers the core enterprise metrics.
- Second and third AI patterns (for example RAG over policy, text-to-SQL for a business unit).
- Interop live if a second engine is in use (Apache Iceberg as the interchange format).
- Exit: maturity level 3; paved road under one day; most new work on the platform.
Stage 3: AI-enabled (to 24 months)
- Remaining migration waves; legacy decommission on schedule.
- Agents over multiple products, exposed through MCP to approved assistants.
- Continuous evaluation and monitoring for every AI system (LLM evaluation).
- Exit: maturity level 4; the legacy platform switched off.
Lighthouse candidates: decisions (to validate with the business)
Choose the lighthouse by decision. A dataset is a lighthouse only because a decision needs it.
| Decision | Why it is a good lighthouse | Data it needs | AI support | Decision outcome measure |
|---|---|---|---|---|
| Which road defects are fixed first? | Mixed structured and unstructured data, clear value, moderate sensitivity | Defect and maintenance requests, assets, crashes | Classify and route requests; recommend a ranking | Response targets met; repeat defects within six months |
| What do registration and licensing numbers tell the business unit this month? | High demand, well-understood metrics, clear owners | Registration and licensing metrics | Text-to-SQL for business units | Time to answer; answers matching the certified metric |
| What does procedure say for this customer's situation? | Little migration needed, fast value | Policy and procedure knowledge base | RAG assistant for customer-service staff | First-contact resolution; escalations; citation accuracy |
The illustrative Decision inventory has more candidates, including licence renewal.
Measure decision outcomes
Report each stage's progress in decision outcomes, not only in pipelines and products shipped. For each lighthouse and production decision, track against the baseline set in Stage 0:
| Measure | What it shows |
|---|---|
| Time to decide | Speed, from request to decision |
| Override and reversal rate | Whether recommendations are right, and where rules are wrong (Outcome feedback loops) |
| Outcome quality | The decision-specific signal: repeat defects, appeals, complaints |
| Explainability | Share of decisions with a complete Decision record |
| Cost per decision | Platform and model cost divided by decisions supported (FinOps) |
Prefer a lighthouse with some personal information, so the governance pattern gets proven early, but not the most sensitive data first.
Risks to manage
| Risk | Mitigation |
|---|---|
| Two platforms chosen before the workloads justify it | Bake-off evidence; revisit triggers in ADR-001 |
| Migration stalls in the parallel run | Exit criteria agreed up front; reconciliation automated |
| AI pilots that never reach production | Evaluation set and production path defined before the pilot starts |
| Consumption cost surprises | Budgets, alerts and owners per workload from day one (FinOps) |
| Key-person risk on legacy knowledge | Business-rule extraction early; record SME sessions |
| Vendor capability claims out of date | reviewed dates; re-verify at each stage gate |