@vostersc/crawler
v2.0.9
Published
This package is a wrapper around Puppeteer. The package provides faster out of the box use by taking care of things like proxy urls, ips, user agents, and others.
Downloads
44
Readme
Crawler
This package is a wrapper around Puppeteer. The package provides faster out of the box use by taking care of things like proxy urls, ips, user agents, and others.
Quick Start
const C = new Crawler();
const config = [
{selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=peach']},
{selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=avocado']},
{selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=steak']}
];
const anon = false;
const logger = false;
const keepOpen = true;
C.performAction(config, anon, logger, keepOpen); //ignore the await
setTimeout(() => {
console.log('status', C.getStatus());
}, 30000);