Mongoose PII Plugin

TL;DR

Store your data like your MongoDB database is getting stolen tomorrow, without sacrificing Mongoose comfort.

The slightly longer intro

Best practices for data storage dictate that:

1. Passwords should be securely hashed; the typical state of the art right now being BCrypt with a securely-random IV and 10+ rounds (e.g. 2¹⁰+ iterations) in production.
2. PII should be securely ciphered; typically we'd use AES256.

These help avoid access to cleartext passwords and compromission of PII (Personally Identifiable Information, such as e-mails, names, Social Security Numbers, Driver’s License information, Passport numbers…) by database theft or unauthorized direct access.

This is all good, but we want to retain the comfort of authenticating, in our code, with cleartext password values that were typed in a form or sent in the API call; we also want to be able to query based on PII fields using cleartext values, or to update them with cleartext values.

In short, we want secure storage without having to worry about it.

This plugin does exactly that.

In this document

1. Installing
2. API
3. Caveats
4. Contributing
5. License and copyright

Installing

If you’re using npm:

npm install mongoose-pii
# or npm install --save mongoose-pii if you're running npm < 5.x

With yarn:

yarn add mongoose-pii

Quick start

First, prep your schemas

For every schema that has PII, passwords, or both:

1. open the file that define your schema
2. Require the plugin
3. Register it as a schema plugin, providing relevant field lists and, for ciphering PII, the ciphering key.

Here’s what it could look like:

// 2. Require the plugin
const { markFieldsAsPII } = require('mongoose-pii')

const userSchema = new Schema({
  address: String,
  email: { type: String, required: true, index: true },
  firstName: String,
  lastName: String,
  password: { type: String, required: true },
  role: String,
})

// 3. Register the plugin
userSchema.plugin(markFieldsAsPII, {
  fields: ['address', 'email', 'firstName', 'lastName'],
  key: process.env.MONGOOSE_PII_KEY,
  passwordFields: 'password',
})

const User = mongoose.model('User', userSchema)

That’s it! Now…

• Your PII fields will be automatically ciphered at save and deciphered at load (so in-memory, they’re cleartext), and you can use cleartext values for queries and updates on them.
• Your PII fields will be automatically ciphered in query arguments for finders (e.g. findOne()) and updaters (e.g. updateMany(), findOneAndUpdate()).
• Your password fields will be automatically hashed in a secure manner at save. This is a one-way hash, so you’ll never have access to the cleartext again, which is as it should be. To authenticate, use the plugin-provided authenticate() static method:

const user = await User.authenticate({
  email: 'foo@bar.com',
  password: 'secret',
})

In its default mode, this resolves to either null, or the first matching User document.

Second, convert your existing data

You’re likely to have a ton of existing, unprotected data in your collections already. However, the moment you register the plugin with your Mongoose schemas, loading data starts to break down because it expected hashed passwords for authentication and ciphered PII in the database!

It would be way too detrimental to loading performance to check for the ciphered state of data in the raw loaded document (not to mention heuristics are not universal there), so instead, we provide a helper API for you to convert your existing collections once you registered the plugin with the proper options.

See the convertDataForModel() API below for details.

Check out our examples!

Find more usage examples in the examples directory.

API

Click on the API names (or press Return when they have focus) to toggle API documentation for them.

const { markFieldsAsPII } = require('mongoose-pii')
const { Schema } = require('mongoose')

const schema = new Schema({
  // …
})

schema.plugin(markFieldsAsPII, {
  fields: ['address', 'city', 'email', 'ssn', 'lastName'],
  key: process.env.MONGOOSE_PII_KEY,
})

schema.plugin(markFieldsAsPII, { passwordFields: 'password' })

schema.plugin(markFieldsAsPII, {
  fields: 'address city email ssn lastName',
  key: process.env.MONGOOSE_PII_KEY,
  passwordFields: 'password',
})

async function logIn(req, res) {
  try {
    const { email, password } = req.body
    const user = await User.authenticate({ email, password })
    if (!user) {
      req.flash('warning', 'No user matches these credentials')
      res.render('sessions/new')
      return
    }

    req.logIn(user)
    req.flash('success', `Welcome back, ${user.firstName}!`)
    res.redirect(paths.userDashboard)
  } catch (err) {
    req.flash('error', `Authentication failed: ${err.message}`)
    res.redirect(paths.logIn)
  }
}

Helper functions

These functions are used internally by the plugin but we thought you’d like to have them around. They’re accessible as named exports from the module, just like the plugin.

if (await checkPassword('secret', user.password)) {
  req.flash('warning', 'Your password is a disgrace to privacy')
}

const key = 'I say: kickass keys rule supreme'
cipher(key, 'I wish all APIs were this nice')
// => 'urWDOjnc6EeMv3ASdrerGAn9YIZw3gjO7lve2EzBQ7Qz7uq4b8UsEBRsOCUPfHitA='

const key = 'I say: kickass keys rule supreme'
decipher(key, 'urWDOjnc6EeMv3ASdrerGAn9YIZw3gjO7lve2EzBQ7Qz7uq4b8UsEBRsOCUPfHitA=')
// => 'I wish all APIs were this nice'
decipher(key, 'ZdZK5sk5P6BGfQJX9qqvFgBUFhR/OXZtv27LaPeCk7kuGrglgq2BS+jSZU1H34GJs=')
// => 'I wish all APIs were this nice' -- this used a non-derived IV

async function demo(newPass, cb) {
  try {
    cb(null, await hashPassword(newPass))
  } catch (err) {
    cb(err)
  }
}

function demo(newPass, cb) {
  try {
    cb(null, hashPassword(newPass, { sync: true }))
  } catch(err) {
    cb(err)
  }
}

Data migration utility

const YourModel = require('./path-to-your-model')
const { convertDataForModel } = require('mongoose-pii/convert')

convertDataForModel(YourModel)
  .then((convertedCount) => console.log(`Converted ${convertedCount} documents`))
  .catch((error) => console.error('Failed during the conversion:', error))

const EventEmitter = require('events')
const YourModel = require('./path-to-your-model')
const { convertDataForModel } = require('mongoose-pii/convert')

const emitter = new EventEmitter()
// This is fired for every successfully-converted Document (1-n)
emitter.on('docs', (convertedCount) => { /* … */ })
// This is fired every time the (rounded-down) process completion percentage changes (1-100)
emitter.on('progress', (updatedPercentage) => { /* … */ })

convertDataForModel(YourModel, emitter)
  .then((convertedCount) => console.log(`Converted ${convertedCount} documents`))
  .catch((error) => console.error('Failed during the conversion:', error))

Caveats

There are a few things to keep in mind when using this plugin.

You need to cipher deleteMany() queries yourself

Mongoose does not yet provide a deleteMany hook, which means we’re not auto-ciphering queries used with deleteMany(). If you’re using queries on ciphered fields with it, you currently need to cipher values yourself, using your ciphering key and the helper cipher() function we provide (see above), staying in deriveIV mode.

We have to mutate many objects you pass Mongoose methods

Mongoose’s plugin API does not let us return updated objects for documents, queries or update descriptors: all we have to work with are the “original” objects, and we have to mutate these.

This means you should be super-careful to not inadvertently reuse an object you pass Mongoose that contains ciphered or hashed-password fields, as such objects will likely be mangled by the plugin; you’d end up double-ciphering stuff, possibly yell at double-deciphering attempts, too.

Rotating keys isn’t easy right now

Security best practices would mandate that you rotate ciphering keys as time passes, to further reduce the risk of compromission. However, using a new key would:

• invalidate deciphering of existing PII in the database
• incorrectly cipher fields in queries, causing empty results or mismatches

The usual workaround for this is to work with a keyring: a small array of keys, most-recent first, where we use the most-recent for writes and all keys for queries. This is fine for cookie signature scenarios, but it seems to us that this could quickly aggravate queries on the database, and presents challenges in query descriptor mutations or composition to turn otherwise single-value matches into working OR clauses. If anyone can whip up a good benchmark on this, perf-wise, and a working PR with tests, we’d love it!

Be wary of your maximum field sizes

In MongoDB, maximum field sizes aren’t very useful… Still, sometimes you put some maximum length in there, usually based on well-known data formats, such as SSN’s, driver’s license numbers, phone numbers, etc. Many such fields are PII indeed.

Do remember that ciphering these fields results in 22 characters of IV prefix plus at least 33% more characters than the original data, due to ciphering and Base64 encoding. Adjust your maximum lengths accordingly, if any.

Avoid case transforms

We store all our ciphered and hashed data in Base64 format, which is case-sensitive. It's nice to normalize such data as e-mail addresses to lowercase, but this will break deciphering in a very big way, as this essentially corrupts the ciphertext. Make sure you don't have such transforms set on your ciphered or hashed fields.

We need Node 8.6+, unless you Babelize us

We use modern ECMAScript, including REST/Spread properties (“Object spread”). Although it became an official part of the language in ES2018, it’s been available in Node since v8.6.0. Node 8 is currently (October 2018) the Maintenance LTS version, with Node 10 being the Active LTS, and Node 11 out already. Our package.json contains an engines.node field requiring >= 8.6, in order for npm to display a warning should you install it on a lower version.

Still, if you absolutely must use a version below it (which means you’re on a version that was, or is imminently going to be, End-Of-Lifed: not a wise choice), you can configure Babel to transpile our source, too.

We’re soon going to dual-publish (using both our native and transpiled source in the module’s package), but still, you should keep your Node runtimes up-to-date, at least with the latest LTS. There’s a lot to gain with this approach.

Contributing

You want to help? That’s awesome! Check out the details of our contribution process (it’s fairly standard).

This project is run under the Contributor Covenant: make sure you read its dispositions and agree with it before you start contributing.

License and copyright

This library is © 2018 Delicious Insights and is MIT licensed. See LICENSE.md for details.