Skip to content

Library · Data Practice OS

AI gateway and model access

Reference guide · Published · Source reviewed

The answer: every model call in the practice goes through a governed path that authenticates the caller, enforces where the call is processed, logs it, applies Guardrails and attributes its cost. In practice that means three sanctioned routes and no personal API keys. See ADR-005: Governed model access through an AI gateway.

The three routes

Route Use for Governance comes from
Snowflake Cortex AI (AI SQL, Analyst, Search, Agents) via Cortex AI Gateway AI over data already in Snowflake Snowflake roles and model RBAC, the account's cross-region setting, gateway observability and cost views
Databricks Model Serving with Unity Gateway (Databricks AI (formerly Mosaic AI)) AI over data in Databricks; custom and fine-tuned models Unity Catalog permissions, Unity Gateway grants, rate limits, guardrails and usage tracking
Amazon Bedrock Apps and agents running outside both platforms IAM, Australian-region endpoints, CloudTrail, Bedrock Guardrails

Anthropic Claude is available on all three, so choosing a model does not force a platform choice. Prefer the route closest to the data so prompts with Personal information never leave the governance boundary.

Residency rules

  • Process in Australia by default. Pick models available in Australian regions on the chosen route.
  • SENSITIVE data is processed only in Australia. Cross-region routing is allowed only when it is pinned to Australia: Snowflake's CORTEX_ENABLED_CROSS_REGION = 'AWS_AU', Bedrock's Australian geographic inference profiles (au. model IDs, for example au.anthropic.claude-opus-5-5), and Databricks cross-geo processing left off. Global or APJ routing is for OFFICIAL, non-personal workloads only, with a recorded decision.
  • Set routing explicitly; do not inherit defaults. New Snowflake organisations created after 9 March 2026 default to ANY_REGION. Other accounts inherit a regional default that excludes government, finance and healthcare accounts. Databricks model availability depends on the workspace region and its cross-geo settings.
  • Know what the model provider retains. Confirm, per route, that prompts and outputs are not used for training and how long they are logged.
  • Model and region availability changes monthly. Verify in the Cortex, Databricks and Bedrock docs before each design decision.
-- Snowflake: Australian processing, with access to current Claude models
ALTER ACCOUNT SET CORTEX_ENABLED_CROSS_REGION = 'AWS_AU';
-- Only approved roles may call AI functions, and only approved models
REVOKE USE AI FUNCTIONS ON ACCOUNT FROM ROLE PUBLIC;
GRANT  USE AI FUNCTIONS ON ACCOUNT TO ROLE ai_builder;
-- PUBLIC inherits every model through CORTEX-MODEL-ROLE-ALL by default: remove that first
CALL SNOWFLAKE.LOCAL.REVOKE_FROM_PUBLIC_APPLICATION_ROLE('APP_ROLE', 'CORTEX-MODEL-ROLE-ALL');
ALTER ACCOUNT SET CORTEX_MODELS_ALLOWLIST = 'None';
GRANT  APPLICATION ROLE SNOWFLAKE."CORTEX-MODEL-ROLE-CLAUDE-SONNET-5-5" TO ROLE ai_builder;
-- The Cortex AI Gateway object is usable by PUBLIC by default
-- The account's Cortex AI Gateway object (named SNOWFLAKE) grants USAGE to PUBLIC by default:
-- revoke it and grant it to approved roles (syntax in the Cortex AI Gateway docs)

Use model RBAC rather than CORTEX_MODELS_ALLOWLIST. Since August 2026 the allowlist can only be set to 'None', and Snowflake will remove it later in 2026. Access is granted if either the allowlist or RBAC permits it, and ACCOUNTADMIN always reaches every model, so test restrictions with another role and USE SECONDARY ROLES NONE. Check model role names with SHOW APPLICATION ROLES LIKE 'CORTEX-MODEL%' IN APPLICATION SNOWFLAKE. See cross-region inference, model access and Cortex AI Gateway.

What the gateway enforces

  1. Identity: the end user or a named workload identity, never a shared key.
  2. Allowlist: approved models per Data classification level.
  3. Limits: per-team rate and spend caps.
  4. Guardrails: PII detection or redaction on input and output where the use case needs it, and topic restrictions for public-facing assistants.
  5. Logging: request metadata always, and payloads where the privacy assessment allows, kept for the agreed period.
  6. Cost attribution: every call tagged to a team and use case for FinOps reporting.

Choosing a model

  • Start with a strong general model (for example a current Claude Sonnet) to prove the use case passes its evaluation set.
  • Then try smaller or cheaper models against the same set, and keep the cheapest one that passes.
  • Batch LLM functions in SQL jobs (classify ten million incident narratives) are where model choice moves cost most. Test on a sample first and estimate the full run.
  • Fine-tune only when prompting plus retrieval provably fails and you have labelled data. It adds a model to own.

Worked example

The safety team wants to classify 12 years of crash narrative free text into contributing-factor categories. The narratives are SENSITIVE and include names. Route: Cortex AI SQL functions AI_CLASSIFY in the Snowflake account holding crash data, cross-region off, with an approved in-region model. A 2,000-row sample graded by analysts sets the pass bar. Projected cost for the full run is signed off before execution, and the job runs on a dedicated warehouse tagged use_case=crash-factors.

Pitfalls

  • Shadow access. Developers calling public APIs with personal keys. Make the sanctioned route easier than the workaround.
  • Assuming a model is in-region because the platform account is. Check per model.
  • Pinning nothing. Record the model version in evaluation results, and re-evaluate before upgrades.
  • Unbounded batch jobs. Always run a sample and a cost estimate first.

Related: Governance, security and trust, AI on Snowflake - Cortex, AI on Databricks, Cost control on Snowflake, Cost control on Databricks.

Lessons from platform options papers

Source: a platform team's draft options papers and platform docs (Oct-Nov 2026). All draft or proposed. None is an agreed decision; treat them as evidence, not standards.

  • Inventory embedded model calls. Managed services call models that your gateway never sees: memory extraction, knowledge-base parsing, embedding and reranking. For each use case, record the service, the model and the region, and confirm residency. The platform team could not confirm the region of the AgentCore memory-extraction model.
  • Geographic vs global cross-region inference. In-country routing options (Bedrock au. inference profiles; Snowflake CORTEX_ENABLED_CROSS_REGION = 'AWS_AU') route only between Australian regions. Global options do not. Write the residency rule to distinguish the two, and verify each with a dated check.
  • A tool gateway is not a model gateway. AgentCore Gateway governs MCP tool calls. It does not replace model-call governance (Tool gateway).