SCP_scrape

this package goes to the wikidot website and scrapes the main content and should only take the html

install

new project

npm init

now

npm install scp_scrape 

existing project

npm install scp_scrape 

run

var scrape = require('scp_scrape');
scrape(1,3)

result

Data/{number}.html

so the folder will look something like this:

Data/

SCP-002.html

SCP-003.html

SCP-004.html

if there is a failure is will be printed but it will keep going so watch out!!!!!

I would fork the code if you are going to use it because you probably want more precise logging.

Note: i havent yet looked at SCP-001 i just know that there are more than one on the same number so i just if you put in anything lower than 2 it will start at 2.

Note_2: the requester is not so fast so it wil take days to go through every single scp and you will get some errors so this is not perfect.