hawk-webcrawler
v0.8.5
Published
Simple, fast webcrawler
Downloads
10
Readme
Hawk
The Hawk is a web crawler tool developed for high performance, supports pages mapping and updating information already collected.
Install Hawk
$ sudo apt-get install cpulimit
$ sudo npm install -g pm2
$ git clone https://github.com/andrehrf/hawk.git
$ npm install
Start application
$ cd hawk
$ pm2 start ecosystem.json
http://localhost:5800