FoundationDB Migrations

This guide documents migration behavior in FoundationDBDataStore and operational controls.

Preferred API:

• pass migration settings as migrationConfiguration = MigrationConfiguration(...)
• pass distributed lease tuning as migrationLeaseConfiguration = FoundationDBMigrationLeaseConfiguration(...)
• flat migration arguments on FoundationDBDataStore.open(...) still work for compatibility

When migration is required

On startup, the store compares stored model definitions with configured models.

No migration required (automatic):

• new model
• safe property additions
• compatible index additions (with backfill)
• relaxed validation

Migration required:

• incompatible type changes
• removals/renames without compatibility aliases
• stricter constraints that break compatibility

Handler contract

All migration hooks receive MigrationContext<FoundationDBDataStore> and return MigrationOutcome:

• Success
• Partial
• Retry
• Fatal

Available hooks:

• migrationExpandHandler
• migrationHandler (Backfill)
• migrationVerifyHandler
• migrationContractHandler

Runtime phases

Execution phases:

1. Expand
2. Backfill
3. Verify
4. Contract

Progress is persisted via MigrationState per model (phase/status/attempt/cursor/message). Restart resumes from persisted state.

Current handler hooks:

• Expand: runs migrationExpandHandler
• Backfill: runs migrationHandler
• Verify: runs migrationVerifyHandler
• Contract: runs migrationContractHandler

Dependency ordering and cycles

Model migrations are ordered by dependency graph.

• Referenced models migrate first.
• Dependency cycles are detected and rejected up front with MigrationException.

Startup budget and background continuation

Use:

• migrationConfiguration.migrationStartupBudgetMs
• migrationConfiguration.continueMigrationsInBackground = true

Behavior:

• Startup can hand migration over to background.
• Pending models remain request-blocked until completion.

Operator APIs

• pendingMigrations()
• migrationStatus(modelId)
• migrationStatuses()
• awaitMigration(modelId)
• pauseMigration(modelId)
• resumeMigration(modelId)
• cancelMigration(modelId, reason)

cancelMigration is terminal for the current store instance: the model stays blocked and you must reopen the store to resume from persisted state.

Operator semantics:

• awaitMigration(modelId) returns normally when migration completes successfully.
• awaitMigration(modelId) completes exceptionally with MigrationException if migration fails or is canceled.
• pauseMigration(modelId) only affects pending/background progress. It does not interrupt a currently running phase step.
• While paused, the model remains request-blocked.
• resumeMigration(modelId) allows the next background loop iteration to continue from the persisted MigrationState.
• After cancelMigration, the current store instance keeps the model blocked. Reopening the store resumes from the persisted phase/cursor.

Lease behavior (distributed)

Default lease is FoundationDBMigrationLease.

• Lease key per model in metadata subspace
• owner token
• TTL via migrationLeaseConfiguration.migrationLeaseTimeoutMs
• heartbeat via migrationLeaseConfiguration.migrationLeaseHeartbeatMs
• automatic takeover after TTL expiry

Use custom migrationLease only if external orchestrator semantics are needed.

Observability

Retry thresholds:

• migrationConfiguration.migrationRetryPolicy.maxAttempts
• migrationConfiguration.migrationRetryPolicy.maxRetryOutcomes

Status payload includes:

• state
• phase
• attempt
• last error
• cursor presence
• retry count
• ETA estimate

Metrics:

• migrationMetrics(modelId) / migrationMetrics()

Audit:

• default sink: migrationConfiguration.migrationAuditEventReporter (default line logger)
• optional persisted audit store: migrationConfiguration.persistMigrationAuditEvents = true
• query persisted events: migrationAuditEvents(modelId, limit)

Operational rollout pattern

1. Enable background continuation for long-running migrations.
2. Set retry thresholds for bounded failure handling.
3. Monitor runtime status/metrics.
4. Intervene with pause/resume/cancel only when required.
5. Keep persisted audit logs opt-in; use default reporter for baseline observability.

Minimal four-phase example

FoundationDBDataStore.open(
    keepAllVersions = true,
    directoryPath = listOf("maryk", "app"),
    dataModelsById = mapOf(1u to Account),
    migrationConfiguration = MigrationConfiguration(
        migrationExpandHandler = { context ->
            // Prepare compatibility before data rewrite
            MigrationOutcome.Success
        },
        migrationHandler = { context ->
            // Backfill existing rows
            MigrationOutcome.Success
        },
        migrationVerifyHandler = { context ->
            // Validate rewritten data
            MigrationOutcome.Success
        },
        migrationContractHandler = { context ->
            // Final cleanup after verification
            MigrationOutcome.Success
        }
    )
)