async-by-page

When you need to operate on a list of things, but you can only get 1 page at a time.

This might already exist, but it's something I've had to do a few times recently so it's time to make a lib.

Usage

const byPage = require('async-by-page')

byPage(fetchNextPage, (item, next) => {
  // Let's say we only want to operate on items
  // that have a certain flag set. We can just call `next()`
  // with no args to skip.
  if (item.isSubscribedToNewsletter) { return next() }

  // For the others, we can perform an action (like sending an email)
  // and then call `next()` when it's done.
  sendEmail(item, next)
})

// Here we define out "fetch next page" function, that is called from `async-by-page`.
// It takes a single callback argument. If the callback is called with an error,
// the iteration terminates.
//
// The main thing to know with this function is that it is stateful.
// It should keep track of its own state, and return the next page of results
// each time it is called.

let page = 1
function fetchNextBatch (cb) {
  getLeadsBatch(page, (err, res) => {
    page += 1
    return cb(err, res)
  })
}

Can I use it? Is it stable?

Not really :) I'm going to be trying a few different things first, and adding a couple of extra features (eg. option for running in parallel VS series). But your feedback is still welcome. I'd also like to write an implementation with pull-streams because I think it would be a very useful comparison (and probs much cleaner too!).

License

MIT