MongoDB Schema Migration – How to Change Schema in Production

migrationmongodbschemaupgrade

I have a mean-stack website www.myweb.io. I need to upgrade the code and the database. In the previous database, I have a schema:

var UserSchema = new mongoose.Schema({
    username: {type: String, lowercse: true, unique: true},
    ...
})

I decided to change a lot UserSchema, especially to delete username.

I just uploaded the code from dev to prod. Adding the first user worked fine, whereas adding another user gave me:

events.js:182
      throw er; // Unhandled 'error' event
      ^
MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: news.users.$username_1  dup key: { : null }
    at Function.MongoError.create (/opt/funfun/node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/lib/error.js:31:11)

I guess it is because both of the two new users don't have username. But why does the system still remember username is the key? In MongoDB, we don't have to change schema, right?

Additionally, I backed up the previous database of production, and copied to development, and restored it. Oddly, unlike production, creating two new users did not give error in development.

Does anyone know if I have to change schema in production? and how?

Best Answer

In MongoDB, we don't have to change schema, right?

MongoDB does not require you to predeclare schema, but there is still some onus for an administrator to remove unneeded indexes or data constraints. The Mongoose Node.js framework you are using includes a declarative client-side schema, but the MongoDB server is unaware of any changes to field or index usage.

By default Mongoose creates new indexes on startup (depending on the config.autoIndex option of your schema), but does not have any tracking to remove or alter index definitions once created. Since you had previously created a unique index on username, the index will need to be manually dropped if you no longer require that constraint.

The duplicate key error message indicates the index name and offending key value:

E11000 duplicate key error index: news.users.$username_1 dup key: { : null }

Any documents missing the field in a unique index will have a value of null indexed. You cannot have two documents in the same collection missing a field in a unique index unless that index is configured as a sparse or partial index.

You can manually drop the problematic index using the mongo shell or an admin tool. For example: db.getSiblingDB('news').users.dropIndex({username:1})

Additionally, I backed up the previous database of production, and copied to development, and restored it. Oddly, unlike production, creating two new users did not give error in development.

The difference in outcome will depend on whether the target database had the unique index previously created.

Related Question