cache-me-outside
v1.0.0
Published
Caching tool for quicker builds in CI systems
Readme
Cache me outside
Caching tool for quicker builds in CI systems
Usage
1. Configure the files you want to cache
/* code from ./install-from-cache-example.js */
const path = require('path')
const cacheMeOutside = require('../lib') // require('cache-me-outside')
/* cache destination folder */
// const cacheFolder = path.join('/opt/build/cache', 'storage')
const cacheFolder = path.join(__dirname, '.cache')
/* Array of folders to cache */
const contentsToCache = [
{
contents: path.join(__dirname, '../node_modules'),
handleCacheUpdate: 'npm install',
shouldCacheUpdate: async ({ cacheManifest, actions }) => {
console.log('cacheManifest', cacheManifest)
console.log(actions)
/* Your custom invalidation logic */
/*
- Dates cache last updated
- Make remote api request
- Diff dependencies files like package.json
- Whatever you want
*/
// This example uses changes to package.json to invalid cached 'node_modules' folder
const packageJson = path.join(__dirname, '../package.json')
const packageJsonChanged = await actions.diff(packageJson)
console.log('packageJsonChanged', packageJsonChanged)
const updateCache = packageJsonChanged
return updateCache // Boolean
},
},
// ... add more folders if you want
]
// Run cacheMeOutside
cacheMeOutside(cacheFolder, contentsToCache).then((cacheInfo) => {
console.log('Success! You are ready to rock')
cacheInfo.forEach((info) => {
console.log(info.cacheDir)
})
})2. Add to your build step
Now that we have configured what we want to cache, we need to add this to our build step.
Inside package.json, or whatever build tool you are using, run the catch-me-outside script before your build step.
{
"scripts": {
"prebuild": "node ./install-from-cache-example.js",
"build": "react-scripts build"
}
}Inside of CI system: npm run build will run through the diagram below.
- Check for the cached files
- If no cache, run the
handleCacheUpdatecommands or function (ienpm install) - Then save the contents to the cache directory for the next run.
If any of shouldCacheUpdate return true, the cached files are invalidated and handleCacheUpdate is ran again.
If you omit shouldCacheUpdate, the hash of the folder contents will be used, so if any file changes within the contents you are caching, the handleCacheUpdate will run.
API
/* code from install-from-cache-multiple.js */
const path = require('path')
const cacheMeOutside = require('../lib') // require('cache-me-outside')
/* Netlify cache folder */
//let cacheFolder = path.join('/opt/build/cache', 'my-cache-folder')
let cacheFolder = path.join(__dirname, '.cache')
/* Array of folders to cache */
const contentsToCache = [
{
/**
* Directory of files to cache
* @type {String}
*/
contents: path.join(__dirname, '../node_modules'),
/**
* Command or Function to run on `shouldCacheUpdate = true`
* @type {String|Function}
*/
handleCacheUpdate: 'npm install && echo "this runs when cache is invalid"',
/**
* Sets whether or not cache should get updated
* @param {object} api
* @param {object} api.cacheManifest - contains useful info for custom invalidation
* @param {object} api.actions - contains helpful functions for diffing
* @return {Boolean} Returns true or false
*/
shouldCacheUpdate: async ({ cacheManifest, actions }) => {
// This example uses changes to package.json to invalid cached 'node_modules' folder
const packageJson = path.join(__dirname, '../package.json')
const packageJsonChanged = await actions.diff(packageJson)
// You can check multiple files or run custom logic
return packageJsonChanged
},
},
{
contents: path.join(__dirname, '../other/node_modules'),
shouldCacheUpdate: function() {
/* your custom cache invalidation logic here */
return false
},
handleCacheUpdate: 'yarn install'
},
{
contents: path.join(__dirname, '../serverless-test/.serverless'),
handleCacheUpdate: () => {
console.log('run my custom stuff here')
}
// if `shouldCacheUpdate` omitted will use contents folder hash
},
]
/*
// local cache folder for testing
cacheFolder = path.resolve('./cache')
/**/
// Run cacheMeOutside
cacheMeOutside(cacheFolder, contentsToCache).then((cacheInfo) => {
console.log('Success! You are ready to rock')
cacheInfo.forEach((info) => {
console.log(info.cacheDir)
})
})When the cache is saved it generates a cache.json manifest file. This is passed into shouldCacheUpdate if you want to use it to invalidate your cache.
{
"createdOn": 1534296638475,
"createdOnDate": "Wed, 15 Aug 2018 01:30:38 GMT",
"modifiedOn": 1534300695541,
"cacheDir": "/Users/davidwells/Netlify/cache-magic/cache/Netlify/cache-magic/serverless-test",
"cacheDirContents": "/Users/davidwells/Netlify/cache-magic/cache/Netlify/cache-magic/serverless-test/.serverless",
"contents": {
"src": "/Users/davidwells/Netlify/cache-magic/serverless-test/.serverless",
"hash": "0496d16c0a8b1d43ca2d3c77ca48a8e237fdb625",
"files": {
"stuff.txt": "11b80f260a5eea9e867a23ab7f96aff77080ff90"
}
}
}How it works
┌───────────────────────────────────────┐
│ │
│ npm run build │
│ │
│ "node ./cache-me-script.js" │
│ │
└───────────────────────────────────────┘
│
Does cache exist?
│
│
├───────────Yes───────────┐
│ │
│ │
┌────────────No───────────┘ │
│ ▼
│ ┌───────────────────────────┐
│ │ │
│ │ Check if Cache is valid │
│ │ via `shouldCacheUpdate` │
│ │ │
│ │ │
▼ └───────────────────────────┘
┌───────────────────────────┐ │
│ │ Is cache valid?
│ Run `handleCacheUpdate` │ │
│ command or function │◀────────────Not valid───────────────┴─────────────Valid─────┐
│ │ │
│ │ │
└───────────────────────────┘ │
│ ▼
│ ┌────────────────────────────────────────────┐
▼ │ │
┌────────────────────────────┐ │ Cache is good and no files that would │
│ │ │ trigger an update have changed. │
│ Copy contents to cache │ │ │
│ directory for next run │ │ Copy cache contents to source destination │
│ │ │ │
│ │ └────────────────────────────────────────────┘
└────────────────────────────┘