MWBot

Description

MWBot is a Node.js module for interacting with the MediaWiki API created and originally developed by @Fannon.

The library makes use of the Promise pattern and behind the scenes uses the native Node.js fetch API together with tough-cookie for cookie-based session management.

The design goal is to be as flexible as possible, with the ability to overwrite options and behaviour at any point. The library also lets you freely choose the abstraction/convenience level on which you want to work. You can use convenience functions that bundles (with concurrency) multiple API requests into one function, but you can also handcraft your own custom MediaWiki API and pure HTTP requests.

Requirements: Node.js ≥ 20, MediaWiki ≥ 1.39.

Documentation

Since this library is based on promise, it can be used either via the promise notation, or using async/await.

Typical Example (Promise .then/.catch)

const MWBot = require('mwbot');

let bot = new MWBot();

bot.loginGetEditToken({
    apiUrl: settings.apiUrl,
    username: settings.username,
    password: settings.password
}).then(() => {
    return bot.edit('Test Page', '=Some more Wikitext=', 'Test Upload');
}).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

Typical Example (async/await)

async function main() {
    const bot = new MWBot({
        apiUrl: 'https://www.wikidata.org/w/api.php'
    });
    await bot.loginGetEditToken({
        username: …
        password: …
    });
    …
    for (const itemId of …) {
        await bot.request(…);
    }
}
// In the root scope of a script, you must use the promise notation, as await is not allowed there.
main().catch(console.error);

For more examples, read the documentation or have a look at the /test directory

Constructor and Settings

constructor(customOptions, customRequestOptions)

Constructs a new MWBot instance.

let bot = new MWBot();

Construct with custom options:

let bot = new MWBot({
    verbose: true
});

Construct with custom request options (e.g. timeout):

let bot = new MWBot({}, {
    timeout: 160000
});

setOptions(customOptions)

Overwrite/extend the default bot options

bot.setOptions({
   verbose: false,  
   silent: false,
   defaultSummary: 'MWBot',
   concurrency: 1,
   apiUrl: false,
   sparqlEndpoint: 'https://query.wikidata.org/bigdata/namespace/wdq/sparql'
});

setGlobalRequestOptions(customRequestOptions)

Overwrite/extend the default request options. This may be important for more advanced usecases, e.g. setting a custom user agent or adding authentication headers.

User-Agent: mwbot sends mwbot/<version> by default. If you are running against Wikimedia wikis, the Wikimedia User-Agent policy requires an identifying user agent. Use the recommended format: BotName/version (https://example.org/; [email protected])

bot.setGlobalRequestOptions({
    headers: {
        'User-Agent': 'MyBot/1.0.0 (https://example.org/; [email protected])'
    },
    timeout: 120000 // 120 seconds
})

Login and Session Management

Login with user and password. This will be necessary for most bot actions. A successful login will add the login token to the bot state.

.login(loginOptions)

bot.login({
  apiUrl: "http://localhost:8080/wiki01/api.php",
  username: "testuser",
  password: "testpassword"
}).then((response) => {
    // Logged In
}).catch((err) => {
    // Could not login
});

.getEditToken()

Fetches a CSRF edit token required for write operations (edit, create, delete, move, etc.). After a successful call the token is available as both bot.editToken and bot.state.csrftoken.

bot.getEditToken().then(() => {
    // token is now in bot.editToken
    return bot.edit('Test Page', 'New content', 'My summary');
}).catch((err) => {
    // Error: Could not get edit token
});

loginGetEditToken(loginOptions)

Combines .login() and .getEditToken() into one operation. This is the recommended flow for bots that need to write to the wiki.

setApiUrl(apiUrl)

For read-only operations no login is needed — just set the API URL and call read() directly.

bot.setApiUrl('https://en.wikipedia.org/w/api.php');

Or equivalently via the constructor:

const bot = new MWBot({ apiUrl: 'https://en.wikipedia.org/w/api.php' });

Example — read a page without logging in:

const bot = new MWBot({ apiUrl: 'https://en.wikipedia.org/w/api.php' });
const response = await bot.read('Main Page');
console.log(response.query.pages['1'].revisions[0].slots.main['*']);

CRUD Operations

create(title, content, summary, customRequestOptions)

Creates a wiki page. If the page already exists, it will fail

• See https://www.mediawiki.org/wiki/API:Edit

bot.create('Test Page', 'Test Content', 'Test Summary').then((response) => {
    // Success
}).catch((err) => {
    // General error, or: page already exists
});

// Or simpler in async/await
try {
  const response = await bot.create('Test Page', 'Test Content', 'Test Summary');
} catch (err) {
  // General error, or: page already exists
}

read(title, customRequestOptions)

Reads the content of a wiki page. To fetch more than one page, separate the page names with |

• See https://www.mediawiki.org/wiki/API:Query

bot.read('Test Page|MediaWiki:Sidebar', {timeout: 8000}).then((response) => {
    // Success
    // The MediaWiki API Result is somewhat unwieldy:
    console.log(response.query.pages['1']['revisions'][0].slots.main['*']);
}).catch((err) => {
    // Error
});

readWithProps(title, props, redirect, customRequestOptions)

Reads the content and meta-data of one (or many) wikipages based on specific parameters. To fetch more than one page, separate the page names with |. To define multiple props separate them also with |.

• See https://www.mediawiki.org/wiki/API:Query

bot.readWithProps('Test Page|MediaWiki:Sidebar', 'user|userid|content', true, {timeout: 8000}).then((response) => {
    // Success
    console.log(response.query.pages['1']['revisions'][0].slots.main['*']);
}).catch((err) => {
    // Error
});

update(title, content, summary, customRequestOptions)

Updates a wiki page. If the page doesn't exist, it will fail.

• See https://www.mediawiki.org/wiki/API:Edit

bot.update('Test Page', 'Test Content', 'Test Summary').then((response) => {
    // Success
}).catch((err) => {
    // Error
});

edit(title, content, summary, customRequestOptions)

Edits a wiki page. If the page does not exist yet, it will be created.

• See https://www.mediawiki.org/wiki/API:Edit

bot.edit('Test Page', 'Test Content', 'Test Summary').then((response) => {
    // Success
}).catch((err) => {
    // Error
});

upload(title, pathToFile, comment, customParams, customRequestOptions)

Upload a file to the wiki. If the file exists, it will be skipped. Make sure your wiki is configured correctly for file uploads

bot.upload(false, __dirname + '/mocking/example1.png')}).then((response) => {
  // Success
}).catch((err) => {
  // Error
});

uploadOverwrite(title, pathToFile, comment, customParams, customRequestOptions)

Like upload(), but will overwrite files on the server

Convenience Operations

batch(jobs, summary, concurrency, customRequestOptions)

This function allows to work more conveniently with the MediaWiki API. It combines all CRUD operations and additionally manages concurrency, logging, error handling, etc.

let batchJobs = {
    create: {
        'TestPage1': 'TestContent1',
        'TestPage2': 'TestContent2'
    },
    update: {
        'TestPage1': 'TestContent1-Update'
    },
    delete: [
        'TestPage2'
    ],
    edit: {
        'TestPage2': 'TestContent2',
        'TestPage3': Math.random()
    },
    upload: {
        'Image1.png': '/path/to/Image1.png'
    }
};

bot.batch(batchJobs, 'Batch Upload Summary').then((response) => {
    // Success
}).catch((err) => {
    // Error
});

Alternatively, an array.array notation can be used. The first array item is the operation name, the second declares the page name. All following array items are used as function parameters.

bot.loginGetEditToken(loginCredentials.valid).then(() => {
    return bot.batch([
        [
            'create',
            'TestPage1',
            'TestContent1',
            'Batch Upload Reason'
        ],
        [
            'update',
            'TestPage1',
            'TestContent1-Update',
            'Batch Upload Reason'
        ],
        [
            'delete',
            'TestPage1',
            'Batch Upload Reason'
        ]
    ], false, 1);

}).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

sparqlQuery(query, endpointUrl, customRequestOptions)

Query Triplestores / SPARQL Endpoints like those from wikidata and dbpedia.

let endPoint = 'https://query.wikidata.org/bigdata/namespace/wdq/sparql';
let query = `
    PREFIX wd: <http://www.wikidata.org/entity/>
    PREFIX wdt: <http://www.wikidata.org/prop/direct/>
    PREFIX wikibase: <http://wikiba.se/ontology#>

    SELECT ?catLabel WHERE {
        ?cat  wdt:P31 wd:Q146 .

        SERVICE wikibase:label {
            bd:serviceParam wikibase:language "en" .
        }
    }
`;

bot.sparqlQuery(query, endPoint).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

askQuery(query, customRequestOptions)

let apiUrl = 'https://www.semantic-mediawiki.org/w/api.php';
let query = `
    [[Category:City]]
    [[Located in::Germany]] 
    |?Population 
    |?Area#km² = Size in km²
`;

bot.askQuery(query, apiUrl).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

Basic Requests

In case that the standard CRUD requests are not sufficient, it is possible to craft custom requests:

request(params, customRequestOptions)

This request assumes you're acting against a MediaWiki API. It allows you to easily craft custom MediaWiki API Request. It also does basic error handling, and uses the login data if given.

bot.request({
    action: 'edit',
    title: 'Main_Page',
    text: '=Some Wikitext 2=',
    summary: 'Test Edit',
    token: bot.editToken
}).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

rawRequest(requestOptions)

Executes a raw HTTP request via the native Node.js fetch API. Use this if you need full flexibility or want to make generic HTTP requests.

bot.rawRequest({
    method: 'GET',
    uri: 'https://jsonplaceholder.typicode.com/comments',
    qs: {
        postId: 1
    }
}).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

Helper Functions

MWBot.logStatus(status, currentCounter, totalCounter, operation, pageName, reason)

Static function that prints "pretty" upload status log messages. Is used internally to print .batch() status messages.

MWBot.logStatus('[+] ', counter, total, 'USER', user.userName);

MWBot.Promise

Exposes the native Node.js Promise constructor.

MWBot.map

Exposes p-map for concurrent batch requests.

MWBot.mapSeries

Exposes a sequential mapper for batch requests (equivalent to p-map with concurrency 1).

Tips and Tricks

Learn how to use the Promise pattern! It handles concurrency, parallel and sequential requests, and more.

mwbot exposes MWBot.map (p-map) and MWBot.mapSeries for batch operations:

• For concurrent batch requests: use MWBot.map(items, fn, { concurrency: N })
• For strictly sequential batch requests: use MWBot.mapSeries(items, fn)

Complete Examples

Fetch content of a page, change it and upload the changed page

let bot = new MWBot();
bot.loginGetEditToken({
    apiUrl: "http://localhost:8080/wiki01/api.php",
    username: "testuser",
    password: "testpassword"
}).then(() => {
    return bot.read('Main Page');
}).then((response) => {
    let pageContent = response.query.pages['1']['revisions'][0].slots.main['*'];
    pageContent += ' Appendix';
    return bot.update('Main Page', pageContent);
}).then((response) => {
    // Success
}).catch((err) => {
    // Error
});

Concurrent execution of batch jobs with MWBot.map (p-map)

This example takes a list of pages and executes a purge action on it. It also demonstrates how to (re)use the static MWBot.logStatus helper function.

• See https://github.com/sindresorhus/p-map

let bot = new MWBot();

let pages = [
    'Main Page',
    'Test Page'
];

let pagesTotal = pages.length || 0;
let pageCounter = 0;

bot.loginGetEditToken({
    apiUrl: "http://localhost:8080/wiki01/api.php",
    username: "testuser",
    password: "testpassword"
}).then(() => {

    return MWBot.map(pages, (page) => {

        pageCounter += 1;

        return bot.request({
            action: 'purge',
            titles: page,
            forcelinkupdate: true
        }).then((response) => {

            // Use MWBot.logStatus helper function
            if (response.error) {
                MWBot.logStatus('[E] ', pageCounter, pagesTotal, 'PURGE', response.purge[0].title);
            } else {
                MWBot.logStatus('[=] ', pageCounter, pagesTotal, 'PURGE', response.purge[0].title);
            }
        }).catch((err) => {
            MWBot.logStatus('[E] ', pageCounter, pagesTotal, 'PURGE', page);
            log(err);
        });

    }, {
        concurrency: 2
    }).then((response) => {
        // Success
    }).catch((err) => {
        // Error
    });

}).catch((err) => {
    // Login Error
});

Testing

To run CI locally, use make ci or e.g. NODE_VERSION=20 MW_VERSION=1.43 make ci.

To develop against a running Wiki, run

# start wiki
$ make up
# run the node container
$ make bash
# run tests inside the container
> npm test
> ...
# finally exit
> exit
# and shutdown the wiki again 
$ make down