A new major version of your database is exciting right up until you have to run it in production. The release notes are long, the upgrade procedure has more than one step, and the question that actually matters is hard to answer from a document: what happens to my application when I do this?

MongoDB 9.0 is generally available, and it brings performance gains over 8.0, including up to 35% faster find-one queries and up to 30% faster update-one queries. Those figures are "up to" numbers that depend on hardware, workload, and data, so the best way to find out what 9.0 does for you is to try it on your own workload, and the first step to doing that is a safe upgrade path.

In this tutorial, we'll walk through a realistic upgrade of a three-member replica set from MongoDB 8.0 to 9.0 using Docker. We'll check the health of the deployment, take a backup and verify it restores, make a go/no-go decision, perform a rolling upgrade one member at a time, and then raise the feature compatibility version. Along the way, we'll look at what changes for an application and how to measure your own workload with $queryStats.

The Prerequisites

Prior to starting this tutorial, you'll need a few things in place:

  • Docker with Compose v2, and at least 2 GB of memory available to it
  • curl for downloading the sample dataset

The expectation is that you're comfortable running commands in a terminal. You don't need Node.js installed on your machine because the sample application runs inside a container.

It's worth being clear about why Docker is in the picture. We're using it for the sake of this tutorial, because it lets us create a three-member replica set on a single machine, reset it, and run through the whole process as many times as we want. Docker is not a requirement for upgrading to MongoDB 9.0. The same procedure applies to a self-managed replica set running on virtual machines or bare metal, where you'd replace the MongoDB packages or binaries on each member instead of swapping a container image. If you run on MongoDB Atlas, the platform performs the rolling upgrade for you, and we'll touch on what that looks like later.

We'll use the sample_mflix dataset that you may already know from MongoDB Atlas, but since we're not using Atlas for the main walkthrough, we'll restore it to our own deployment. Start by creating a project directory with a place for each part of the project:

mkdir mongodb-9-upgrade cd mongodb-9-upgrade mkdir data docker backup scripts mkdir -p app/src

The above commands create the project and the directories we'll fill in as we go. We'll end up with the following shape:

. ├── app/ │ ├── package.json │ └── src/ │ ├── db.js │ ├── baseline.js │ └── utcDemo.js ├── backup/ │ ├── backup.sh │ └── verify.sh ├── data/ │ └── sampledata.archive ├── docker/ │ ├── compose.yml │ └── restore.sh └── scripts/ ├── run-app.sh └── queryStats.js

The sample dataset is a single archive that contains several databases. Download it into the data directory:

curl -o data/sampledata.archive https://atlas-education.s3.amazonaws.com/sampledata.archive

The archive is roughly 400 MB, so it may take a minute. We'll only restore the sample_mflix database from it later.

What Changes in MongoDB 9.0

Before touching any servers, it helps to know what you're signing up for. The compatibility changes page is the authoritative list, and you should read it in full before upgrading a real deployment. The changes most likely to affect an application fall into three groups.

Some changes happen silently, meaning nothing errors and you get a different answer:

  • Server-side JavaScript in $function, $accumulator, $where, and mapReduce now evaluates Date operations in UTC rather than in the time zone of the mongod host.
  • The $queryStats key and keyHash for aggregate queries that set allowPartialResults are serialized differently, so anything that matches on keyHash for those queries may see new values. The queryShapeHash doesn't change.

Some changes can cause new errors:

  • A single operation is limited to the greater of 1 GB or 20% of server memory. Queries that previously used unbounded memory can fail with ExceededMemoryLimit, or with QueryExceededMemoryLimitNoDiskUseAllowed if disk use isn't allowed.
  • All ingested BSON is validated by default, so invalid BSON that older versions accepted now returns an error.
  • An accumulator with an empty field name in $group now returns an error.
  • A query can fail with QueryKilledError if an indexed field becomes multikey while the query runs.
  • A change stream opened with a readPreference of primary or secondary returns error code 11602 from getMore after an election. Compliant drivers resume on their own, but manual getMore loops need to resume with resumeAfter.

And some changes are new tooling worth knowing about:

  • $queryStats is on by default, samples 1% of operations, and now covers update, insert, and delete.
  • $function, $accumulator, and $where are no longer deprecated, and they run on a sandboxed WebAssembly-based engine.

We'll demonstrate the time zone change later because it's the one most likely to go unnoticed.

Create a Three-Member Replica Set on MongoDB 8.0

A realistic upgrade starts from a realistic deployment. In production, you'd run a replica set with at least three members, and the upgrade procedure for a replica set is a rolling one that takes advantage of that. We'll recreate the same shape locally.

Create a docker/compose.yml file with the following:

name: mongo9blog x-mongo: &mongo command: ["--replSet", "rs0", "--bind_ip_all", "--wiredTigerCacheSizeGB", "0.25"] environment: TZ: America/New_York services: mongo1: <<: *mongo image: ${MONGO1_IMAGE:-mongodb/mongodb-community-server:8.0.34-ubuntu2204} volumes: - mongo1-data:/data/db mongo2: <<: *mongo image: ${MONGO2_IMAGE:-mongodb/mongodb-community-server:8.0.34-ubuntu2204} volumes: - mongo2-data:/data/db mongo3: <<: *mongo image: ${MONGO3_IMAGE:-mongodb/mongodb-community-server:8.0.34-ubuntu2204} volumes: - mongo3-data:/data/db volumes: mongo1-data: mongo2-data: mongo3-data:

There are a few things happening in this file. The x-mongo block at the top holds the settings that all three members share, and each service pulls them in with <<: *mongo.

x-mongo: &mongo command: ["--replSet", "rs0", "--bind_ip_all", "--wiredTigerCacheSizeGB", "0.25"] environment: TZ: America/New_York

The --replSet rs0 flag makes each mongod a replica set member. The cache size is capped at a quarter of a gigabyte so that three members fit comfortably on a laptop alongside everything else you're running. The TZ value runs the servers in the New York time zone, which will matter when we look at the server-side JavaScript change. Production servers are very often set to UTC, and if yours are, you won't see that particular difference, but the demo is still a good way to understand it.

The more important part for the upgrade is how each member gets its image:

 mongo1: <<: *mongo image: ${MONGO1_IMAGE:-mongodb/mongodb-community-server:8.0.34-ubuntu2204} volumes: - mongo1-data:/data/db

Each member has its own image variable, which defaults to MongoDB 8.0.34. That means we can start everything on 8.0 now, and later upgrade a single member by setting just that member's variable. Each member also has its own named volume, so the data survives when a container is replaced with a different image.

Start the three members:

docker compose -f docker/compose.yml up -d

The first run pulls the image, which takes a moment. Even after the containers are running, each mongod needs a few seconds to start accepting connections, so wait a few seconds before running the next command. If you see a connection refused error, wait a moment and run it again. The members also don't know about each other yet, so we need to initiate the replica set:

docker compose -f docker/compose.yml exec mongo1 mongosh --quiet --eval ' rs.initiate({ _id: "rs0", members: [ { _id: 0, host: "mongo1:27017", priority: 2 }, { _id: 1, host: "mongo2:27017" }, { _id: 2, host: "mongo3:27017" } ] }) '

The priority: 2 on the first member makes mongo1 the preferred primary, which keeps the walkthrough predictable. The hostnames only resolve inside the Docker network, which is why the sample application will run in a container later.

Give the replica set a few seconds to elect a primary, then load the sample data. Create docker/restore.sh with the following:

#!/bin/sh set -eu cd "$(dirname "$0")/.." docker run --rm \ --network mongo9blog_default \ -v "$PWD/data:/data-in:ro" \ mongo:8.0 \ mongorestore \ --uri "mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=rs0" \ --archive=/data-in/sampledata.archive \ --nsInclude 'sample_mflix.*'

The script runs mongorestore from a throwaway container on the same network, so you don't have to install the MongoDB Database Tools. The --nsInclude flag restores only the sample_mflix database from the archive. Run it with:

sh docker/restore.sh

When it finishes, the last line should tell you everything was restored:

67661 document(s) restored successfully. 0 document(s) failed to restore.

That's 21,349 movies, 41,079 comments, and the rest of the sample_mflix collections, and our starting point is ready.

Create an Application to Test the Upgrade

An upgrade that only proves the server starts isn't much of a test, so we'll use a small application that talks to the replica set before, during, and after the upgrade. It's written in Node.js, but the specifics matter less than the habit of running something real against the deployment at each stage.

Create app/package.json with the following:

{ "name": "mongodb-9-upgrade-demo", "version": "0.1.0", "private": true, "type": "module", "scripts": { "baseline": "node src/baseline.js", "utc-demo": "node src/utcDemo.js" }, "dependencies": { "mongodb": "^7.8.0" } }

The two scripts correspond to the two files we're about to write. Next, create app/src/db.js, which is a small helper that connects, runs a function, and always closes the client:

import { MongoClient } from "mongodb"; const uri = process.env.MONGODB_URI ?? "mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=rs0"; export async function withDb(fn) { const client = new MongoClient(uri); try { await client.connect(); return await fn(client.db("sample_mflix"), client); } finally { await client.close(); } }

The connection string lists all three members and names the replica set. That's what lets the driver find whichever member is primary, and it's also what lets it keep working while members restart during a rolling upgrade.

The first script, app/src/baseline.js, reports what the application sees and does a read and a write:

import { withDb } from "./db.js"; await withDb(async (db) => { const admin = db.admin(); const { version } = await admin.serverInfo(); const fcv = await admin.command({ getParameter: 1, featureCompatibilityVersion: 1 }); console.log("version:", version); console.log("fcv:", fcv.featureCompatibilityVersion.version); const movies = db.collection("movies"); console.log("movies:", await movies.countDocuments()); const one = await movies.findOne({ title: "The Matrix" }, { projection: { title: 1, year: 1 } }); console.log("find-one:", one); const update = await movies.updateOne({ title: "The Matrix" }, { $inc: { demoViews: 1 } }); console.log("update-one:", { matched: update.matchedCount, modified: update.modifiedCount }); });

The script prints the server version and feature compatibility version, counts the movies, runs a findOne, and runs an updateOne. Because it does both a read and a write, it shows that the application keeps working through each stage. It is not a benchmark.

To run it, we need a Node.js container on the same network as the replica set. Create scripts/run-app.sh with the following:

#!/bin/sh set -eu cd "$(dirname "$0")/.." docker run --rm \ --network mongo9blog_default \ -v "$PWD/app:/app" \ -w /app \ -e MONGODB_URI="mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=rs0" \ node:22 \ sh -c "npm install --silent --no-audit --no-fund && npm run --silent $1"

The script mounts the app directory into a node:22 container, installs the dependencies, and runs whichever npm script you pass in. Run the baseline:

sh scripts/run-app.sh baseline

You should see output similar to the following:

version: 8.0.34 fcv: 8.0 movies: 21349 find-one: { _id: new ObjectId('573a139bf29313caabcf3d23'), year: 1999, title: 'The Matrix' } update-one: { matched: 1, modified: 1 }

This is our "before" picture: MongoDB 8.0.34, feature compatibility version 8.0, and an application that reads and writes without any trouble.

Add a Check for a Behavior Change

A clean upgrade tells you that the server runs, not that your application behaves the same. One change in 9.0 is easy to miss because nothing errors: server-side JavaScript now evaluates Date operations in UTC instead of the time zone of the mongod host. We'll capture a "before" result now so we can compare it after the upgrade.

Create app/src/utcDemo.js with the following:

import { withDb } from "./db.js"; await withDb(async (db) => { const comments = db.collection("comments"); const total = await comments.countDocuments(); const matchesNewYorkHour = await comments.countDocuments({ $expr: { $eq: [ { $function: { body: "function (d) { return d.getHours(); }", args: ["$date"], lang: "js", }, }, { $hour: { date: "$date", timezone: "America/New_York" } }, ], }, }); console.log(`getHours() equals the New York hour: ${matchesNewYorkHour} of ${total}`); });

The query counts the comments where the hour returned by JavaScript's getHours() equals the hour calculated by $hour with an explicit America/New_York time zone. Our servers run in the New York time zone, so on 8.0 these should always agree. Run it against the 8.0 replica set:

sh scripts/run-app.sh utc-demo

Every comment agrees:

getHours() equals the New York hour: 41079 of 41079

Keep that number in mind. We'll run the same query again after the upgrade.

Check the Health of the Deployment Before You Upgrade

Before changing anything, confirm that what you have is healthy. The upgrade documentation calls out that no member should be in the ROLLBACK or RECOVERING state, and you also want to know how far each secondary is behind the primary.

Run the following to get a summary of the replica set:

docker compose -f docker/compose.yml exec mongo1 mongosh --quiet --eval ' const status = rs.status(); const primary = status.members.find((m) => m.stateStr === "PRIMARY"); printjson({ members: status.members.map((m) => ({ name: m.name, state: m.stateStr, health: m.health, lagSecs: (primary.optimeDate - m.optimeDate) / 1000 })), version: db.version(), fcv: db.adminCommand({ getParameter: 1, featureCompatibilityVersion: 1 }).featureCompatibilityVersion.version }) '

The command lists each member with its state, health, and replication lag in seconds, plus the server version and feature compatibility version. A healthy set looks like this:

{ members: [ { name: 'mongo1:27017', state: 'PRIMARY', health: 1, lagSecs: 0 }, { name: 'mongo2:27017', state: 'SECONDARY', health: 1, lagSecs: 0 }, { name: 'mongo3:27017', state: 'SECONDARY', health: 1, lagSecs: 0 } ], version: '8.0.34', fcv: '8.0' }

All three members are healthy with no lag, and the feature compatibility version is 8.0. If you see a member in RECOVERING or ROLLBACK, or lag that isn't shrinking, stop here and fix that first.

Two more checks belong in a real pre-flight and can't be shown with a sample dataset. First, you can only upgrade to 9.0 from 8.0 or 8.3, so if you're on 7.0 or earlier, upgrade to 8.0 first. If you're on 8.3, its feature compatibility version must be 8.3. Second, confirm that your drivers are compatible with 9.0. The upgrade documentation warns that incompatible drivers can produce unexpected or undefined behavior. These are the first driver versions that are fully compatible with MongoDB 9.0, according to the client library compatibility tables:

  • Node.js: 7.6
  • PyMongo: 4.18
  • Java (sync and reactive streams), Kotlin, and Scala: 5.12
  • .NET/C#: 3.10
  • Go: 2.9
  • Rust: 3.9
  • Ruby: 2.26
  • PHP library: 2.5
  • C: 2.5
  • C++: 4.6

Older versions listed as partially compatible will work with 9.0, but they don't support every new 9.0 feature, and the tables recommend moving to the versions above. The sample application in this tutorial uses version 7.8 of the Node.js driver, so it meets the requirement. Check the compatibility page for any driver or framework that isn't listed here.

Take a Backup and Prove That It Restores

If something goes wrong during an upgrade, a backup is the one safety net that doesn't depend on the upgrade itself working. It's also the step people skip, so we'll make it a real step and verify it.

For this tutorial, we'll use mongodump, which is a logical backup that most readers can run. Create backup/backup.sh with the following:

#!/bin/sh set -eu cd "$(dirname "$0")/.." FILE="backup-$(date +%Y%m%d-%H%M%S).archive.gz" docker run --rm \ --network mongo9blog_default \ -v "$PWD/backup:/backup" \ mongo:8.0 \ mongodump \ --uri "mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=rs0" \ --oplog \ --gzip \ --archive="/backup/$FILE" echo "Wrote backup/$FILE"

The script is run from your own machine, not from inside a container. The container names in the connection string, mongo1, mongo2, and mongo3, only resolve inside the Docker network, so the script uses docker run to start a short-lived mongo:8.0 container on that network (mongo9blog_default) and runs mongodump there. The -v flag mounts your local backup directory into that container, which is how the archive ends up on your machine. The --rm flag removes the container when the dump finishes, so there's nothing to clean up.

The --oplog flag is the important one for a replica set. While the dump runs, writes keep arriving, and --oplog captures those changes so the archive reflects a single point in time. The --gzip flag compresses the archive, and the timestamp in the file name keeps backups from overwriting each other. From the root of the project, in a terminal on your host machine, run it:

sh backup/backup.sh

On a small dataset, this finishes quickly and ends with the name of the file it wrote:

Wrote backup/backup-20261103-075154.archive.gz

Our archive is about 73 MB. Having a file isn't the same as having a backup, though, because you don't know a backup works until you've restored it. Create backup/verify.sh, which restores an archive into a throwaway MongoDB container and compares document counts against the live replica set:

#!/bin/sh set -eu cd "$(dirname "$0")/.." ARCHIVE="$(basename "$1")" SOURCE="mongodb://mongo1:27017,mongo2:27017,mongo3:27017/?replicaSet=rs0" COUNT='JSON.stringify({movies: db.getSiblingDB("sample_mflix").movies.countDocuments(), comments: db.getSiblingDB("sample_mflix").comments.countDocuments()})' docker rm -f mongo9blog-scratch >/dev/null 2>&1 || true docker run -d --name mongo9blog-scratch --network mongo9blog_default mongo:8.0 >/dev/null until docker exec mongo9blog-scratch mongosh --quiet --eval 'db.runCommand({ping: 1}).ok' >/dev/null 2>&1; do sleep 1 done docker run --rm \ --network mongo9blog_default \ -v "$PWD/backup:/backup:ro" \ mongo:8.0 \ mongorestore \ --uri "mongodb://mongo9blog-scratch:27017" \ --oplogReplay \ --gzip \ --archive="/backup/$ARCHIVE" \ --quiet LIVE="$(docker run --rm --network mongo9blog_default mongo:8.0 mongosh "$SOURCE" --quiet --eval "$COUNT")" RESTORED="$(docker exec mongo9blog-scratch mongosh --quiet --eval "$COUNT")" docker rm -f mongo9blog-scratch >/dev/null echo "live: $LIVE" echo "restored: $RESTORED" if [ "$LIVE" = "$RESTORED" ]; then echo "Backup verified." else echo "Counts do not match." >&2 exit 1 fi

That's a longer script, so it helps to look at the parts. The first part starts a scratch mongod and waits until it answers a ping:

docker rm -f mongo9blog-scratch >/dev/null 2>&1 || true docker run -d --name mongo9blog-scratch --network mongo9blog_default mongo:8.0 >/dev/null until docker exec mongo9blog-scratch mongosh --quiet --eval 'db.runCommand({ping: 1}).ok' >/dev/null 2>&1; do sleep 1 done

The scratch container is completely separate from the replica set, so restoring into it can't affect your data. The second part restores the archive into it with --oplogReplay, which applies the oplog entries that --oplog captured:

docker run --rm \ --network mongo9blog_default \ -v "$PWD/backup:/backup:ro" \ mongo:8.0 \ mongorestore \ --uri "mongodb://mongo9blog-scratch:27017" \ --oplogReplay \ --gzip \ --archive="/backup/$ARCHIVE" \ --quiet

The final part compares counts between the live replica set and the restored copy, and exits with an error if they differ. Run it against the backup you just took, using your own file name:

sh backup/verify.sh backup/backup-20261103-075154.archive.gz

If the backup is good, the counts match and the script says so:

live: {"movies":21349,"comments":41079} restored: {"movies":21349,"comments":41079} Backup verified.

Comparing two document counts is a deliberately small check. In a real environment, you'd also restore into something that resembles production and run a representative query or two against the restored copy.

Atlas handles backups differently. On a paid cluster, snapshots run automatically once backups are enabled. Before you upgrade, confirm that backups are enabled and that your most recent snapshot is recent. Then take an on-demand snapshot so you have a restore point from just before the change, which matters here because Atlas can't downgrade a major version. And just like on Docker, a snapshot you haven't restored is a snapshot you haven't tested, so restore one to a separate cluster and check the result.

If your cluster doesn't include cloud backups, take the backup the same way we did here, by connecting to the cluster with mongodump and using the connection string from Atlas.

Decide Whether to Go Ahead

At this point, you have everything you need to make a decision, and it's worth making that decision deliberately instead of just continuing because the next step is on the page. Before moving forward, make sure the following are true:

  • The replica set is healthy, with no member in ROLLBACK or RECOVERING and no growing lag.
  • You have a backup, and you've restored it and checked the result.
  • You've read the compatibility changes and none of them break your application, or you have a plan for the ones that do.
  • Your drivers support 9.0.
  • You're on 8.0 or 8.3, and you can afford to run for a while with mixed versions in the replica set.

If any of those aren't true, stop here. Nothing has changed yet, and nothing needs to be undone. The remaining steps are the ones that touch your servers.

Upgrade the Secondaries First

A rolling upgrade replaces one member at a time so the replica set stays available throughout. You upgrade the secondaries first, then step down the primary and upgrade it last. The application keeps working while this happens because the replica set always has a majority of members online.

We'll start with mongo2. Setting the image variable for just that member and bringing it up again replaces its container with one running MongoDB 9.0.4, while keeping its data volume:

MONGO2_IMAGE=mongodb/mongodb-community-server:9.0.4-ubuntu2204 \ docker compose -f docker/compose.yml up -d mongo2

Compose recreates only mongo2. The member shuts down, starts again with the 9.0 binaries against the same data files, and rejoins the replica set. It takes a few seconds to start, so if you see a connection error, wait a moment and try again. Start by asking mongo2 itself which version it's running:

docker compose -f docker/compose.yml exec mongo2 mongosh --quiet --eval 'db.version()'

Once it answers, you should see the new version:

9.0.4

Another member's view of the replica set can briefly show a stale state right after a restart, so don't rely on it alone. Once mongo2 answers, ask the primary for each member's state and how far behind it is:

docker compose -f docker/compose.yml exec mongo1 mongosh --quiet --eval ' const status = rs.status(); const primary = status.members.find((m) => m.stateStr === "PRIMARY"); status.members.map((m) => m.name + " " + m.stateStr + " lag: " + (primary.optimeDate - m.optimeDate) / 1000 + "s").join("\n") '

Wait until mongo2 is a SECONDARY with no lag:

mongo1:27017 PRIMARY lag: 0s mongo2:27017 SECONDARY lag: 0s mongo3:27017 SECONDARY lag: 0s

The replica set is now running mixed versions. The primary and mongo3 are on 8.0.34, and mongo2 is on 9.0.4, which is normal and expected during a rolling upgrade. The feature compatibility version is still 8.0 on all members, so the data is still readable by 8.0.

Upgrade mongo3 the same way. Each docker compose command only knows the variables you pass it, so keep passing the one for mongo2. We name only mongo3 here, so mongo2 isn't touched either way, but the habit protects you if you ever run up -d without a service name, which would put mongo2 back on the 8.0 image:

MONGO2_IMAGE=mongodb/mongodb-community-server:9.0.4-ubuntu2204 \ MONGO3_IMAGE=mongodb/mongodb-community-server:9.0.4-ubuntu2204 \ docker compose -f docker/compose.yml up -d mongo3

Once mongo3 reports SECONDARY again, run the application to confirm that it still works while the primary is the only member left on 8.0:

sh scripts/run-app.sh baseline

The output still shows version...