Skip to content

Migration (dw-migration)

The Migration agent plans, translates, and verifies warehouse migrations. Its core is rule-based SQL dialect translation — Oracle, Teradata, and Redshift to Snowflake, with MySQL and PostgreSQL rules as well — covering type mappings, DECODE conversion, and the dialect edge cases that become silent correctness bugs if left as warnings. Around the translator sits the part most migration tools skip: verification. Translations are checked for parity against source data, optionally judged for preserved intent, and re-translated when they fail — a self-correcting loop rather than a one-shot converter.

The agent thinks like a DBA who has been paged for a botched cutover: dry-run before execute, parity before cutover, rollback plan before forward plan. Irreversible steps are flagged and require human approval before execution — that is the default, not an option.

  • Migration assessment. assess_migration inventories objects in the source system and classifies migration complexity, including PII columns and an effort estimate.
  • SQL dialect translation. translate_sql and batch_translate_sql apply rule-based mappings for known patterns; complex SQL can fall back to your model.
  • The self-correcting loop. migrate_with_validation runs translate → (judge) → parity → re-translate until parity passes or the round limit is hit. On non-convergence it returns the best attempt with review annotations for a human — never a silent pass.
  • Intent judging. judge_translation checks whether a translation preserves the meaning of the source, not just matching rows on a sample — catching dropped DECODE defaults, changed NULL handling, and silent join-type changes.
  • Parity validation. validate_migration and run_parallel_comparison compare row counts, column stats, and sample hashes between source and target, and report divergences and match rates.
  • Dependency and blast-radius mapping. map_migration_dependencies builds the downstream-consumer graph and returns a wave-ordered migration plan (topological sort with cycle detection), flagging high-blast-radius objects.
  • A completion gate that fails closed. gate_migration_complete refuses to certify a migration done until validated and parity-checked coverage clears the threshold. An empty or under-covered scope is “not done.”
  • Fleet status. get_migration_status gives per-run progress with per-object drill-down: pending, translating, judging, validating, needs review, converged, failed.

“Assess our Teradata warehouse for migration to Snowflake — inventory, complexity, effort.”

“Translate this Oracle procedure to Snowflake and judge whether the translation preserves intent.”

“Run the self-correcting migration on these 40 views and show me anything that didn’t converge.”

“Map the dependency graph for oracle-prod-1 and give me a wave-ordered plan with blast radius.”

“Is this migration actually done? Gate it at 95% parity coverage.”

  • Snowflake — the migration target for parity checks against real data.
  • Catalog connectors — DataHub, dbt, and the rest of the connector catalog enrich dependency mapping with lineage, so blast radius includes consumers outside the source inventory.
  • The translator itself needs no connection — it works on SQL you paste in.

The agent starts in 🟡 Evaluation, and the translation rules are real logic even there: translate_sql works on your SQL from the first minute, no credential required. Parity checks and inventory run against built-in sample data until systems are connected and verified. See Verify your setup.

  • The judge fails open: if the judge model is unavailable or returns unparseable output, the verdict is pass. Treat the judge as an extra tripwire, not the last line of defense — parity validation is the harder check. The judge requires your own model key (bring-your-own-model, like everything in the swarm).
  • The completion gate fails closed — deliberately. Expect it to say “not done” until you have real coverage, including on an empty scope.
  • The self-correcting loop is bounded by maxRounds. When it doesn’t converge, you get the best attempt annotated for human review, not a certified translation.
  • Lineage enrichment for dependency mapping degrades gracefully: if no catalog is connected, the plan is built from the source inventory only, and the result says so.
  • Dialect edge cases flagged during translation are treated as blockers, not warnings — the agent will stop and tell you rather than guess.