schema-shot

Framework-agnostic snapshot testing using "schema by example" for highly dynamic data

If you like snap-shot (snapshot testing for any JS framework), but have data that is hard to pin down, maybe this package will be useful. Instead of storing literal data snapshot, it stores json-schema derived from a the snapshot object seen first time (using validate-by-example to derive it). Next time an object arrives, it will be validated against the schema. Any missing property, or new one will trigger an exception.

Read the blog post

Example

Imagine we are fetching most popular item from an API service. Obviously it changes often, so we cannot just store it as a snapshot for direct comparison. We could massage the data and derive invariant snapshots, but that is boilerplate code!

Instead, use schema-shot!

npm install --save-dev schema-shot

In your test

// spec.js
const schemaShot = require('schema-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  schemaShot(top)
})

Suppose first time it runs, the API returns top = {id: '45a12e'} - an object with just its id property. The __snapshots__/spec.js.schema-shot file will be saved with

exports['returns most popular item 1'] = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "required": true
    }
  },
  "additionalProperties": false
}

Now imagine the same day later running again. The API returns something else, but the object still has same shape, just a different id {id: 8812f0}. This object passes the schema validation step.

A week later, the new API version gets deployed. Now it returns the top item, but instead of id property, it returns uuid property. The test will fail!

$ npm test
Error: schema difference
  data has additional properties
  data.id: is required

Returned value is schema object

You can inspect or even snap-shot the schema object.

const snapshot = require('snap-shot')
const schemaShot = require('schema-shot')
it('returns schema object', () => {
  const o = {name: 'my name'}
  const schema = schemaShot(o)
  snapshot(schema)
})

Both objects in this case will be

{
  '$schema': 'http://json-schema.org/draft-04/schema#',
  type: 'object',
  properties: {
    name: { type: 'string', required: true }
  },
  additionalProperties: false
}

Inferred property formats

A more specific property formats are inferred, see validate-by-example#inferring-formats and can be overridden.

const person = {
  name: 'Joe Smith',
  email: '[email protected]'
}
// let it infer that person.email has "email" format
schemaShot(person)
// or we can set it ourselves
schemaShot(person, {email: 'email'})
/*
  schema properties
  {
    name: {type: 'string', required: true},
    email: {type: 'string', required: true, format: 'email'}
  }
*/

Dry run and showing the updates

If you just want to see what a new schema would be, without saving it, run the tests with DRY=1 npm test option.

If you want to see the schema and save it, run the tests with SHOW=1 npm test

Arrays

When training with arrays

• All items in the array must pass the schema derived from the first object

Difference with snapshot testing

The schema shots do not store the direct information. A good example is a test using snap-shot and fake-todos in test/todos-spec.js. The snapshot test always fails on the second run, because a returned todo is different. The JSON schema on the other hand stays consistent, as long as fake-todos returns objects with same shape.

const snapshot = require('snap-shot')
const schemaShot = require('schema-shot')
const generate = require('fake-todos')
describe('fake-todos', () => {
  it.skip('returns a different todo', () => {
    const todos = generate(1)
    snapshot(todos[0])
    /*
      Fails of course, because every todo is different
        1) fake-todos returns a different todo:
           Error: snapshot difference
      {
        id: "4e040570-..." => "129a55b4-..."
        what: "do adults" => "skip chess"
      }
    */
  })
  it('returns a todo', () => {
    const todos = generate(1)
    schemaShot(todos[0])
  })
})

Small print

Author: Gleb Bahmutov <[email protected]> © 2017

• @bahmutov
• glebbahmutov.com
• blog

License: MIT - do anything with the code, but don't blame me if it does not work.

Support: if you find any problems with this module, email / tweet / open issue on Github

MIT License

Copyright (c) 2017 Gleb Bahmutov <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.