npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

live-source

v0.4.4

Published

Provide an object view of web resources, updated when they change

Downloads

2

Readme

live-source

NPM version

Provide an object (a "doc") which shows the state of a web resource, updated and emitting 'change' events when the web resource changes. This core module just uses polling, but related modules act as plugins which can handle various kinds of pub/sub in the cases where they are implemented.


Ordinary usage

Here is example1.js. It polls a resource that returns the unix time, changing every second. It misses some values because it's polling at the default rate of 1.5s. See example1a for faster polling, although that's not usually recommended.

const livesource = require('live-source')

const url = 'https://time.hawkeworks.com/?units=1'
const doc = livesource.open(url)
let count = 0
doc.on('change', ({text}) => {
  console.log(++count, text)
  if (count >= 5) doc.close()
})
/* 
=> 
1 '1561761213'
2 '1561761215'
3 '1561761216'
4 '1561761218'
5 '1561761219'
*/

Adding formats with custom parsers

Custom formats can be added with addFormat, which takes an {init, run} object, with two functions:

  • init(doc) sets properties of doc at constructor time, for what you want the initial values to be for the properties this format produces. The init value should usually be something the source would never have, so that a change event fires at the first load, while also being harmless.

  • run(doc) is called after doc.text has been set by fetch(). It should read properties of the doc, like doc.text, and set other properties appropriately. If any format's run() function sets doc.changed to true or false, that determines whether the essential value of the doc is considered changed, and thus whether a 'change' event fires.

Example2 uses two formats, the first turning doc.text into a parsed integer, and the second turning that parsed integer into a Date object, assuming it's a unix timestamp.

const livesource = require('live-source')

const int = {
  init: doc => {
    doc.intValue = 0
  },
  run: doc => {
    const old = doc.intValue
    doc.intValue = parseInt(doc.text)
    return old !== doc.intValue
  }
}

const unixtime = {
  init: doc => {
    doc.time = new Date(0)
  },
  run: doc => {
    const old = doc.time
    doc.time = new Date(doc.intValue * 1000)
    return old.getTime() !== doc.time.getTime()
  }
}

const url = 'https://time.hawkeworks.com/?units=1'
livesource.addFormat(int)
livesource.addFormat(unixtime)
const doc = livesource.open(url)
let count = 0
doc.on('change', ({text, intValue, time}) => {
  console.log(++count, {text, intValue, time})
  if (count >= 3) doc.close()
})
/* 
=>
1 { text: '1561759480',
  intValue: 1561759480,
  time: 2019-06-28T22:04:40.000Z }
2 { text: '1561759482',
  intValue: 1561759480,
  time: 2019-06-28T22:04:40.000Z }
3 { text: '1561759483',
  intValue: 1561759480,
  time: 2019-06-28T22:04:40.000Z }
*/

Adding openers

The livesource module is an instance of "Opener", and has the open(url) function you've seen above. You can give it sub-openers for handling specific URLs, based on looking at their text. Sub-openers have to decide decide synchronously if they'll handle each URL, because open returns synchronously.

An example of sub-openers is in live-source-gdoc, which does special handling for Google Doc URLs.

todo: allow these handlers to take over after the first fetch, since that might be when we discover the page has a websub hub.

Additional modules

  • live-source-proxy is useful for lightweight clients, aggregating polling into hubs, or clients in a browser.