Skip to content

Library · Data Practice OS

RAG and enterprise search

Reference guide · Published · Source reviewed

The answer: for "ask our documents" use cases, use the managed retrieval service on the platform where the documents already live: Cortex Search on Snowflake, Databricks AI Search (formerly Vector Search) on Databricks. Both do hybrid (vector and keyword) retrieval, keep indexes fresh from tables, and respect platform permissions. Building your own vector database is rarely worth it.

The pattern

  1. Land documents (PDF, Word, HTML, emails) in a governed stage or volume. Tag each with source, owner, Data classification and effective dates.
  2. Parse and chunk in the pipeline using LLM functions in SQL: AI_PARSE_DOCUMENT on Snowflake (OCR and LAYOUT modes run natively in Sydney; AGENTIC mode is preview), ai_parse_document on Databricks (Databricks Runtime 17.3 or later; PDF, image, Word and PowerPoint files, so convert HTML and email first), with ai_prep_search (Beta) to chunk its output. Both checked 6 Nov 2026. Chunks become rows in a silver table, each carrying its document's metadata.
  3. Index: create a search service or vector index over the chunk table. Embeddings are managed for you.
  4. Retrieve with filters: classification ≤ user clearance, current versions only, business area.
  5. Generate with a model such as Anthropic Claude, with instructions to answer only from the retrieved passages and cite them.
  6. Evaluate against a fixed question set before release (Evaluating AI systems).
-- Snowflake: chunked policy documents -> Cortex Search service
CREATE OR REPLACE CORTEX SEARCH SERVICE policy_search
  ON chunk_text
  ATTRIBUTES business_area, classification, effective_to
  WAREHOUSE = wh_ai_xs
  TARGET_LAG = '1 hour'
  AS SELECT chunk_text, doc_title, doc_url, business_area, classification, effective_to
     FROM silver.policy_chunks;

-- Databricks: AI Search (formerly Vector Search) index synced from a Delta table (created via API/UI).
-- vector_search() is Public Preview, is not available on classic SQL warehouses, and takes named arguments.
-- It does not accept filters_json in preview: a WHERE on its output filters after retrieval.
SELECT * FROM vector_search(
  index      => 'gov.policy.policy_chunks_idx',
  query_text => 'proof of identity for a heavy vehicle licence upgrade',
  query_type => 'HYBRID',
  num_results => 8);

During the preview, Databricks' SQL vector_search() cannot take filters_json or columns, cannot query Direct Access indexes, and returns at most 100 results (vector_search function, checked 6 Nov 2026). A WHERE clause on its output filters after retrieval, so it can return fewer rows than asked for. Where a classification or clearance filter must apply before ranking, query the index through the AI Search Python SDK or REST API, which take filters (query an AI Search index).

The Snowflake example matches the CREATE CORTEX SEARCH SERVICE syntax as of 6 Nov 2026. Optional settings include PRIMARY KEY, EMBEDDING_MODEL, REFRESH_MODE, REQUEST_LOGGING and AUTO_SUSPEND, and a second form declares TEXT INDEXES and VECTOR INDEXES separately. Pin EMBEDDING_MODEL to a model available in your region. Syntax moves quickly; recheck it and the AI Search vector_search() docs.

When to use it

  • Procedures, policy manuals and legislation explainers for customer service centre staff.
  • Engineering standards and road asset inspection manuals.
  • Searching free-text fields such as incident narratives and complaint notes, combined with structured filters.

When not to

  • The question is about numbers. "How many crashes on the Bruce Highway in 2025?" is a semantic layer and Text-to-SQL question. RAG over PDFs of reports will give confident, stale answers.
  • The corpus is tiny and stable. Twenty pages fit in a prompt. Skip the index.
  • No one owns the documents. Retrieval over outdated, duplicated procedures surfaces the wrong version faster.

Permissions are the hard part

Retrieval must never return a chunk the user could not open directly.

  • Carry classification and access attributes on every chunk row, and filter on them at query time from the caller's identity, not from the prompt.
  • Keep SENSITIVE and Personal information-heavy content (customer correspondence, medical-condition notifications) in separate indexes with separate access. Do not rely on a filter alone.
  • Run a PII check on the generated answer as a second line of defence.

Worked example: licensing procedures assistant

Customer service officers spend minutes per call finding the right procedure for an interstate licence transfer. The licensing team curates about 1,200 procedure documents. A nightly pipeline parses them into roughly 40,000 chunks tagged with procedure ID, version and effective dates. A search service exposes only current versions. The assistant answers in three sentences, links the procedure, and says "I can't find a procedure for that" when retrieval confidence is low. The evaluation set holds 150 real questions from call logs, graded by senior officers. Release requires 90% correct with citation and zero answers citing superseded procedures.

Pitfalls

  • Chunking by fixed characters splits tables and numbered steps. Chunk on document structure (headings, sections) and keep the section path in the chunk.
  • No "I don't know". Instruct and test for refusal when retrieval is weak.
  • Ignoring freshness. Set the index lag to match how fast procedures change, and show the document's effective date in the answer.
  • Evaluating with demo questions. Use real, messy user questions.

Related: Agents and MCP, AI gateway and model access, AI on Snowflake - Cortex, AI on Databricks.

Lessons from platform options papers

Source: a platform team's draft options papers and platform docs (Oct-Nov 2026). All draft or proposed. None is an agreed decision; treat them as evidence, not standards.

  • Step 0: do not copy by default. Before building an index, ask two questions. Can a tool read the source in place with the user's identity? Does an existing capability already cover it? A Derived index needs a recorded reason, an owner, the source system, its last sync time and a link back to the source. The platform team holds this as a direction under evaluation, not policy (Options paper: organisational knowledge access, draft, "not ready for a decision").
  • Choose the pattern before the product. the platform team's options are: use what exists, read in place, a derived index, or a source catalogue that routes queries.
  • A classification filter is not the source's permissions. Tags on chunks do not reproduce per-item access lists in Confluence or SharePoint. Either copy the ACLs and check them at query time, or read in place. Test that identity mapping (for example by email) is unique. Bedrock Managed Knowledge Bases check Confluence permissions in real time (checked Oct 2026; verify).
  • Avoid "knowledge base" as a general term in most organisations. It already names a business-rules glossary. Say "derived index" or "authoritative source".