I Built db-migrator to Move Data Between Databases and Validate the Results

Database migrations always sound easy on paper.

Read data from one database, transform it, and write it somewhere else. Simple enough, right?

Well, not always. Things get messy when you're dealing with different data types, ID formats, nested structures, or large datasets.

And even when a migration finishes successfully, how can you be sure everything was migrated correctly?

I'm Fatih 👋 and that's one of the reasons I started building db-migrator.

Why I Built It

I wanted something that could handle database migrations without having to write custom scripts every time.

But I also wanted to see what was happening during a migration and, more importantly, check the results afterward.

That's what I've been working on with db-migrator.

What Does db-migrator Do?

db-migrator is a self-hosted tool that runs with Docker. It supports PostgreSQL, MongoDB, MySQL, MariaDB, SQL Server, TimescaleDB, and InfluxDB v2 right now. Others will be added too.

Here's what you can do with it:

Set up migrations visually, including field mappings and transformations.
Monitor progress with live metrics and execution logs.
Validate the results by comparing record counts and inspecting differences between source and target data.

A Quick Example

Let's say you're moving data from PostgreSQL to MongoDB.

You can configure both connections, select your data, and map fields to the target structure. For example, a city column could become address.city in MongoDB.

Once the migration starts, you can track its progress in real time, including the number of processed records, migration speed, and any failures.

Here's a completed PostgreSQL to MongoDB migration with 1,000 records processed and no failures. From here, you can compare the data, review the results, or roll back the migration if needed.

The comparison checks record counts, missing and extra records, and field-level differences between source and target data.

When a Migration Fails

Not every run ends that cleanly. In this run, 500 of 1,000 records failed with MongoDB duplicate key errors because their _id values already existed in the target collection.

The rollback removed only the records this run had inserted and left the existing documents untouched.

Keeping Track of Migrations

The dashboard gives you a quick overview of your migrations, including their status, progress, and validation scores. You can also keep track of upcoming scheduled migrations and active database connections, all in one place.

Try It with Docker 🐳

With Docker installed, you can start a local instance with:

docker run -d --name db-migrator -e ASPNETCORE_URLS=http://+:5000 -p 5000:5000 fatihun/db-migrator:latest

Then open http://localhost:5000 in your browser.

This is a quick way to explore the interface. For a persistent setup, see the Docker Compose example on Docker Hub, which includes a data volume and encryption key configuration.

Beyond the Basics

Job scheduling: Run recurring migrations using cron schedules.
Schema drift detection: Identify differences between live database schemas and your configured mappings.
Migration rollback: Roll back eligible completed or failed migrations using confirmed inserted-record IDs or a pre-migration snapshot, depending on the target database and configured strategy.
Webhook notifications: Receive HTTP webhook notifications about migration status changes.
Exportable reports: Download HTML and PDF reports to review and share migration and validation results.
...and more on db-migrator.com.

Documentation & More

Website and documentation: db-migrator.com
Docker image and setup instructions: fatihun/db-migrator

The official Docker image is free to run for personal and commercial use under the db-migrator Free Use License. The license terms are available on the Docker Hub page.