Library · Data Practice OS

Reconciliation, parallel run and cutover

Reference guide · Published · Source reviewed 2026-11-06

The answer: nobody switches off a legacy feed because the new one "looks right". Reconciliation is automated, runs on every load during a Parallel run, and produces a signed evidence pack. Cutover is per consumer group, reversible for a defined window, and decommissioning follows only when no reads remain. Reconciliation proves outputs on live data. A per-rule Equivalence test proves one rule on chosen inputs, including edge cases live data may not contain. You need both (Tracing rules from legacy code to modern code).

The four levels of reconciliation

Run them cheapest first, and stop drilling down once a level passes cleanly.

Level	Check	Catches
1. Counts	Row counts per table, partition and business date	Missing loads, duplicate joins, filter errors
2. Aggregates	Sums, min and max, and distinct counts of key measures by key dimensions	Calculation and rounding errors, type mapping
3. Checksums	Hash of normalised row values per key or partition	Silent column-level differences
4. Samples	Field-by-field comparison of sampled records, including edge cases	The specific rows behind a failed level 1-3 check

Lakebridge has a Reconcile module (schema, row and data checks) for Oracle, Snowflake, SQL Server, Redshift, Teradata, BigQuery and Databricks sources, but not DB2 (Reconcile). SnowConvert AI includes data migration and data validation tooling. Use them where the source is supported. Otherwise, and for DB2 or CA Gen sources, build the checks as dbt tests and macros so they live with the models.

-- Level 3: partition checksum, same SQL shape on legacy and target
SELECT business_date,
       COUNT(*)                                          AS row_count,
       SUM(fee_amount)                                   AS fee_total,
       -- normalise before hashing: trim, upper, fixed decimals, ISO dates, NULL token
       MD5(LISTAGG(MD5(CONCAT_WS('|',
             registration_id, UPPER(TRIM(vehicle_class)),
             TO_CHAR(fee_amount, 'FM999999990.00'),
             COALESCE(TO_CHAR(expiry_date, 'YYYY-MM-DD'), '~'))), '')
           WITHIN GROUP (ORDER BY registration_id))      AS partition_hash
FROM gold.fct_registration_fees
GROUP BY business_date;

Normalisation is where most false alarms come from: trailing spaces, case, empty string versus NULL, timestamp precision and time zones (the local zone may not observe daylight saving, but source systems may store UTC or local time inconsistently).

Expected differences

Some differences are intended: rules changed in Extracting business rules, bugs fixed, or data quality corrected. Record each as an expected difference with the rule ID and the approver. Reconciliation passes when every difference is either zero or expected and explained.

An expected difference that comes from a deliberate rule change needs an accepted decision record (a proposed one does not count), and the legacy pinning test is kept so the old behaviour stays documented (proposed; see "Deliberate divergence" in Tracing rules from legacy code to modern code).

The parallel run

Length. At least one full business cycle that matters, including month end and, for registrations, a financial-year boundary if it falls within the window. Two to six weeks is typical per wave.
Every run reconciled. Results go to a dashboard and an exceptions queue with an owner.
Exit criteria agreed up front, for example: 10 consecutive clean runs, zero unexplained differences, performance within SLA, and consumers signed off.

Cutover

Group consumers by Lineage: the reports, extracts and downstream systems that read each migrated product.
Repoint one group at a time. Power BI datasets, extract jobs, APIs. Keep the legacy feed running.
Rollback window of two to four weeks per group, with the legacy path still warm.
Freeze legacy changes for migrated objects, with any urgent change made in both places.
Check rule coverage per table. Before cutting over a table, check that every rule whose output lands in it (the implements edges of its writers) is verified or waived (Traceability gap report and rule trace card G8; proposed, Tracing rules from legacy code to modern code).
Communicate: what moved, what changed (expected differences), and who to call.

This is the Strangler fig in practice: the legacy shrinks consumer by consumer.

Decommissioning

Monitor legacy object reads after cutover. When reads stay at zero for 30-60 days, archive per records retention obligations (the jurisdiction's public records law usually applies, so verify retention schedules with records management), then switch off.
Before switch-off, check rule coverage: every rule the legacy program carries is claimed by modern code and verified or waived (Traceability gap report and rule trace card G9, "can I delete this program"; proposed, Tracing rules from legacy code to modern code).
Track decommissioning in the legacy-to-new object map. It covers data objects; logic is traced through rules (proposed, Tracing rules from legacy code to modern code). A migration is not finished, and the savings are not realised, until the old system is off.

Worked example

The registrations wave runs in parallel for five weeks, spanning end of October. Run one fails level 1 on cancelled registrations: the legacy view excluded them silently, which was not in any rule card. A new rule card, REG-STAT-003, is confirmed by the owner. Run three fails level 2 on fees by 0.01 on 0.2% of rows, a rounding mapping fixed in conversion. Runs 6-15 are clean. Finance's Power BI reports cut over first, then the operational extracts, then the national data exchange feed. Legacy reads hit zero after six weeks and the mart is archived.

Pitfalls

Reconciling once. One clean run proves little. Reconcile every run.
Comparing outputs only. Also compare intermediate layers for faster diagnosis.
No owner for exceptions. Differences pile up and the run stalls.
Big-bang cutover. All consumers at once removes the rollback path.
Never decommissioning. Two platforms costing money is a migration that failed slowly.