AI and agents.
Notes on preparing information for AI, designing the context an agent works with and testing it under real constraints.
Start reading
33 entries
AI readiness assessment
Assess one named AI use against 54 scored controls and five readiness gates. The maturity score and the readiness decision are reported separately, and a failed gate can't be outweighed by a high score.
Context engineering for decisions
Assemble exactly the facts, rules and knowledge one decision needs, under the user's own permissions, with versions recorded - the decision-stack view of what the RAG and agent pages build.
Evaluating AI systems
Evaluation sets are the release gate. How to build them, grade them, run them in CI, and keep watching after go-live.
Draft in private, publish on purpose
A publishing workflow for reviewing unfinished writing without putting it on the public site.
ADR-005: Governed model access through an AI gateway
LLM calls go through a governed gateway on the platform holding the data, to approved models processed in Australian regions; SENSITIVE data is never routed outside Australia.
Agent memory
What agents may remember between sessions, at which scope, who may write shared memory, and where memory ends and knowledge begins - an open decision in most organisations, with the platform team's draft analysis.
Agents and MCP
When an LLM should plan and call tools, how to scope those tools, and how MCP exposes governed data to agents on either platform.
AI gateway and model access
Which models, where they run, and the single governed path every model call takes - Australian processing, no cross-region for SENSITIVE data, costs attributed.
AI readiness scorecard (workbook v2)
A formula-driven Excel workbook that scores the AI readiness assessment per control, keeps unknowns and N/A apart from zero, and takes the readiness decision from the gates rather than from the score.
AI use case intake and risk triage
An intake form that triages an AI use case for risk, privacy and data readiness, aligned to government AI risk assessments such as FAIRA and to privacy impact assessment.
AI-native patterns
The six patterns that cover most AI work on data - LLM functions in pipelines, search and RAG, text-to-SQL, agents, predictive ML, and AI-assisted engineering - with the platform options for each.
RAG and enterprise search
Grounded answers over policies, procedures and case notes - hybrid retrieval inside the governance boundary, with citations and permission-aware filters.
The semantic layer
Define metrics, joins and business vocabulary once, then serve the same definitions to Power BI, APIs and every LLM that writes SQL.
Agent memory
What an agent keeps between sessions - individual, team or organisation scope. A governance decision about scopes, write rights, review and retention, not just a storage choice.
AI agent
An LLM in a loop that plans, calls tools (SQL, search, APIs) and checks its own results to complete a task. Needs tight tool scopes, budgets and evaluation.
AI gateway
A single governed entry point for model calls - authentication, routing, rate limits, cost tracking, logging and guardrails - instead of every team holding its own API keys.
AI-ready data
Data that a model or agent can use safely without a human in the loop to explain it - described, governed, fresh, trustworthy, and reachable through a stable interface.
Context engineering
Designing what goes into a model's context for a task - which facts, knowledge, rules and memory, from where, under whose permissions and at which version - and testing that it is sufficient and no more.
Derived index
A copy of source content (chunks, embeddings) built for retrieval. Needs a reason to exist, an owner, a link back to the source and a freshness check - otherwise read in place.
Embeddings
Numeric vectors that encode the meaning of text, images or rows, so similar things sit close together.
Feature store
Managed, versioned, point-in-time-correct features shared between model training and serving.
Guardrails
Checks around a model call - input filtering, PII redaction, topic limits, output validation - that keep an AI system inside its approved use.
LLM evaluation
Measuring an AI system against a fixed set of questions and judged answers, before release and continuously after. The gate between a demo and production.
LLM functions in SQL
Calling a model from inside a query (classify, extract, summarise, translate) so unstructured data is processed in the pipeline under the same governance as the tables.
Model Context Protocol
An open protocol for exposing tools and data to AI assistants and agents in a standard way. Both Snowflake and Databricks offer managed MCP servers over governed data.
Model registry
Versioned, governed storage for trained models and their lineage, approvals and deployment stages.
Retrieval-augmented generation
Answering with an LLM grounded in documents or rows retrieved at question time, with citations. The default pattern for "ask our policies and procedures".
Semantic layer
Business definitions - metrics, dimensions, joins, synonyms - declared once and served to BI tools, APIs and LLMs, so "registrations this month" means one thing everywhere.
Text-to-SQL
An LLM turning a natural-language question into SQL. Reliable only when it is grounded in a semantic layer and evaluated against known questions.
Tool gateway
One governed endpoint for agent tool calls (MCP) - authentication, a tool catalogue, routing and logging. Different from an AI gateway, which governs model calls. An architecture can need both.
Use profile
A set of extra controls added to the common baseline for one kind of AI use (analytics, ML, RAG, training, decision support). Profiles are not cumulative; a combined workflow can need several.
Vector search
Finding items by meaning using embeddings. Usually combined with keyword search (hybrid) and filters for production retrieval.
Model Context Protocol
The open protocol for giving agents governed access to tools and data.