gotsentimental

GoT Twitter Sentiment Analysis

As seen on www.got.show

Installing

$ npm install gotsentimental --save

Dependencies:

• recent node.js + npm
• MongoDB

Usage

You need to create a Twitter API key for the crawler.

See example/app for an advanced example.

Example:

const gotsent = require('gotsentimental');

// adjust config
gotsent.cfg.extend({
    "mongodb": {
        "uri": "mongodb://example/gotsentimental"
    },
    "twitter": {
        "access_token": "xxx",
        "access_token_secret": "xxx",
        "consumer_key": "xxx",
        "consumer_secret": "xxx"
    }
});

// initialize
gotsent.init();

// update DB - this might take a few hours
gotsent.update().then(function(res) {
    // print some update stats
    console.log(res);

    // get top5 most popular characters
    gotsent.mostPopular(5).then(function(res) {
        res.forEach(function(character) {
            console.log(character.name);
        });
    }, console.error);

    // gracefully shut down
    gotsent.shutdown();
}, function(err) {
    console.error(err);
    gotsent.shutdown();
});

API

Generated CSV files

The crawler generates static CSV files in the directory set in the config ("csvpath"). These files have to be made available, e.g. by using express.static(__dirname + '/csv');. See example/app for an example.

The following files are generated and required by chart.js:

Episodes:

/csv/episodes.csv

Data per Character:

/csv/Character_Name.csv               (complete overview, grouped per day)
/csv/Character_Name/2016-03.csv       (monthly overview, grouped per hour)

Types

Character

| Name | Type | Description | | --- | --- | --- | | name | string | name of the character | | slug | string | human-readale URL-identifier for the character | | total | number | total number of tweets in database | | positive | number | total number of positive tweets in database | | negative | number | total number of negative tweets in database | | heat | number | how controverse is the character | | popularity | number | how much is the character is discussed | | updated | Date | date when the document was last updated |

Methods and Attributes

• .cfg : Object
• .cfg.extend(json)
• .css : string
• .js : string
• .init()
• .shutdown()
• .update([full]) ⇒ Promise.<Object>
• .updateCharacter(name, [full]) ⇒ Promise.<Object>
• .startUpdateLoop()
• .stopUpdateLoop() ⇒ Promise
• .character(name) ⇒ Promise.<Character>
• .mostPopular([n]) ⇒ Promise.<Array.<Character>>
• .mostHated([n]) ⇒ Promise.<Array.<Character>>
• .mostDiscussed([n]) ⇒ Promise.<Array.<Character>>
• .stats() ⇒ Promise.<Object>

gotsentimental.cfg : Object

Object containing the package configuration. The config can be changed by directly overwriting attributes or using .cfg.extend(json). The default values are stored in defaults.json. Every value can be overwritten.

gotsentimental.cfg.extend(json)

Merges the given Object into the config by overwriting attributes. Arrays are concatenated.

| Param | Type | Description | | --- | --- | --- | | json | Object | Config Object |

gotsentimental.init()

Initilaize the package. Opens the MongoDB connection and initializes the Twitter client.

gotsentimental.shutdown()

Close any open resources like the database connection.

gotsentimental.update([full]) ⇒ Promise.<Object>

Update data by crawling for new tweets and generating new CSV files.

| Param | Type | Default | Description | | --- | --- | --- | --- | | [full] | boolean | false | full rebuild or incremental update |

Returns: Promise.<Object> - A promise to the update results.

gotsentimental.updateCharacter(name, [full]) ⇒ Promise.<Object>

Update data for given character by crawling for new tweets and generating new CSV files.

| Param | Type | Default | Description | | --- | --- | --- | --- | | name | string | | Name of the character | | [full] | boolean | false | full rebuild or incremental update |

Returns: Promise.<Object> - A promise to the update results.

gotsentimental.startUpdateLoop()

Start the update loop. Waits the amount of secunds set in the config after completing one iteration before starting the next incremental update.

gotsentimental.stopUpdateLoop() ⇒ Promise

Waits for the current update to complete, if one is running.

Returns: Promise - A promise which resolves when the loop is stopped.

gotsentimental.character(name) ⇒ Promise.<Character>

Get a character by name.

Returns: Promise.<Character> - A promise to the character.

| Param | Type | Description | | --- | --- | --- | | name | string | Name of the character |

gotsentimental.mostPopular([n]) ⇒ Promise.<Array.<Character>>

Get the most popular characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

| Param | Type | Default | Description | | --- | --- | --- | --- | | [n] | number | 10 | Number of Characters to return |

gotsentimental.mostHated([n]) ⇒ Promise.<Array.<Character>>

Get the most hated characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

| Param | Type | Default | Description | | --- | --- | --- | --- | | [n] | number | 10 | Number of characters to return |

gotsentimental.mostDiscussed([n]) ⇒ Promise.<Array.<Character>>

Get the most discussed characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

| Param | Type | Default | Description | | --- | --- | --- | --- | | [n] | number | 10 | Number of Characters to return |

gotsentimental.css : string

Absolute path to the Chart CSS file. It should be served with e.g. express' sendFile.

gotsentimental.js : string

Absolute path to the Chart JS file. It should be served with e.g. express' sendFile.

gotsentimental.stats() ⇒ Promise.<Object>

Get stats about tweets in database. The returned Object has the following attributes:

• total (total number of tweets),
• positive (total number of positive tweets),
• negative (total number of negative tweets).

Returns: Promise.<Object> - A promise to the stats Object

Testing

Install Gulp:

npm install -g gulp

npm test

Hook up npm and git

To run npm test automatically before every git commit, install a git pre-commit hook:

npm run hookup

git aborts the commit if the tests fail. You can (but shouldn't) bypass it with git commit --no-verify ....