Library · Data Practice OS

Data Practice reading list

Reading list · Published

• Snowflake documentation

  The source of truth for every Snowflake feature claim on this site.

• Snowflake AI and ML features overview

  Map of Cortex AI SQL, Search, Analyst, Agents and Snowflake ML, with current model and region availability.

• Dynamic Tables

  Declarative pipelines in Snowflake. Read before writing another stream-and-task chain.

• Snowflake Horizon

  Governance, classification, policies, lineage and Iceberg catalog capabilities in one place.

• SnowConvert AI

  Free assessment and code conversion into Snowflake. Run the assessment even if you are not sure Snowflake is the target.

• Databricks documentation

  The source of truth for every Databricks feature claim on this site.

• Unity Catalog

  Permissions, ABAC, lineage and the external-engine (Iceberg REST) story.

• Databricks AI Search (formerly Vector Search)

  How indexes sync from Delta tables and inherit permissions.

• Databricks AI Functions

  Batch LLM inference over tables from SQL.

• Lakebridge

  Databricks' assessment, transpilation and reconciliation toolkit.

• Apache Iceberg

  The table format that lets Snowflake and Databricks share one copy of data.

• Apache Polaris

  The open Iceberg REST catalog behind Snowflake Open Catalog.

• Delta Lake

  Databricks' native format, including UniForm for Iceberg readers.

• Unity Catalog (open source)

  The open-source Unity Catalog project and its multi-format, multi-engine ambitions.

• dbt documentation

  The portable transformation layer for both platforms - models, tests, contracts, docs.

• dbt model contracts

  Enforced schemas on published models - the cheapest data contract you can adopt today.

• dbt Semantic Layer

  Cross-platform metric definitions for BI and LLMs.

• Open Data Contract Standard

  A vendor-neutral YAML shape for contracts between producers and consumers.

• OpenLineage

  Emit lineage once, read it in any catalog.

• MLflow

  Tracing and evaluation for GenAI, registry for models - usable on either platform.

• Data Mesh Principles and Logical Architecture

  Zhamak Dehghani's original framing. Read for the organisational ideas; adopt them gradually.

• Team Topologies

  The vocabulary for platform teams, stream-aligned teams and enabling teams used in the operating model.

• FinOps Foundation framework

  Common language for making consumption cost visible and owned.

• Policy for the responsible use of AI in government (DTA)

  The Commonwealth baseline for accountable officials, transparency statements and AI use-case governance. A useful benchmark for any agency or enterprise.

• Example government AI policy - Queensland AI governance policy (QGEA, v2.0.0)

  Mandated in that state since August 2026. AI governance with regard to ISO 38507, and a FAIRA (or agreed equivalent) for AI solutions.

• Example AI risk assessment - FAIRA framework (QGEA, v2.0.0)

  The Foundational AI risk assessment framework and template, including FAIRA LITE for low-risk trials.

• Example classification scheme - Queensland Information Security Classification Framework (QGISCF v6.0.0)

  The OFFICIAL / SENSITIVE / PROTECTED labels and business impact levels that drive tag-based policy.

• Example privacy regulator - Office of the Information Commissioner Queensland

  One state's privacy principles and guidance on privacy impact assessments, including for AI.

• Information Security Manual (ISM)

  The control baseline behind IRAP assessments of cloud services.

• Amazon Bedrock

  Model access in Australian AWS regions when AI runs beside, not inside, the data platform.

• Model Context Protocol

  The open protocol for giving agents governed access to tools and data.

• GitLab enterprise data platform handbook

  A public, working Snowflake + dbt platform - clone per change, tiers, schema-change rules, safe-data CI checks, a curated schema for agents. A platform team mapped 18 of its practices to its own plan.

• dbt_project_evaluator

  Turns dbt best practices into countable CI checks - documentation coverage, grain tests, direct joins to sources.

• Decision Model and Notation (DMN)

  The OMG standard for decision requirements diagrams and decision tables. Read the decision table and hit policy sections before designing rule logic.

• W3C PROV overview

  The family of W3C provenance specifications - entities, activities and agents. The vocabulary behind decision records.

• SKOS Simple Knowledge Organization System reference

  The W3C standard for controlled vocabularies and thesauri - preferred labels, synonyms, broader and narrower concepts, cross-scheme matches.

• NIST AI Risk Management Framework (AI RMF 1.0)

  Govern, map, measure, manage - a widely used structure for AI risk that complements FAIRA. Its "map" function starts from the context and decision an AI system supports. Verify the current version and profiles.

• GraphRAG (Microsoft Research)

  The reference implementation of graph-based retrieval over documents. Read the indexing cost notes before proposing it.

• ISO 30401 Knowledge management systems

  The international standard for establishing and maintaining a knowledge management system. Useful for the ownership and lifecycle of the knowledge layer. Verify the current edition.

• Characterization testing (Michael Feathers)

  The source of the pinning-test idea - record what the legacy system actually does, quirks included, and keep the test even after a decision changes the rule.

• Bitemporal history (Martin Fowler)

  Separates what was true (valid time) from when we recorded it (record time). Read before deciding how trace edges carry dates.

• SEBoK - System Requirements Definition

  Systems-engineering view of requirements traceability - trace to needs, check completeness, assess the impact of change, and fix the traceability model at the start of the project.

• ISO/IEC/IEEE 29148:2018 Requirements engineering

  The international standard for requirements processes and information items, including traceable requirements. Context only - the clause text is paywalled and was not read, and a revision (DIS) is pending. Verify any specific clause claim.

• dbt meta config

  Arbitrary key-values on models, columns, data tests and unit tests, compiled into manifest.json - where rule ids live in dbt. Changing meta does not trigger state:modified, so trace reports must read the whole manifest.