geonames-data-import

Cross-platform console application written in TypeScript and Node.js to automate downloading/unzipping of GeoNames worldwide geographical database dumps and importing them to database of choice (MySQL is supported at the moment).

Demo

(some large files were omitted)

Usage

• Install Node.js and Docker (if you plan to run MySQL as a container).

• Clone git repository:

git clone https://github.com/mezzario/geonames-data-import.git

• From app's folder install node modules:

npm i

• Edit src/config.ts to adjust configuration, if needed (see below).

• Configure your MySQL credentials in .env file:

MYSQL_USER=root
MYSQL_PASS=your_password

• Start MySQL instance (wait for it to load):

npm run mysql

• Run application in another terminal window:

npm start

To automatically run additional SQL queries after import, please refer to file:

assets/db/<db-type>/post-import.sql

Configuration

Edit src/config.ts to adjust app's configuration:

const config: Config = {
  // base URL do download files from
  baseUrl: "http://download.geonames.org/export",

  // list of file paths to download (relative to 'baseUrl')
  dataFilePaths: [...],

  // directory where files will be downloaded
  localDownloadDir: "download",

  // local directory with files missing on remote server
  localDataFilesDir: "assets/data",

  // db queries to create, import or drop data
  localDbAssetsDir: "assets/db",

  // currently selected db type to import data to
  dbType: "mysql",

  // settings for MySQL db
  mysql: {
    // connection params
    connection: {
      host: "127.0.0.1",
      port: "3306",
      user: process.env.MYSQL_USER || "root",
      password: process.env.MYSQL_PASS || "root",
    },

    // db name to import data to
    databaseName: "geonames",
  },

  // set to true to overwrite already downloaded files
  forceDownloading: false,

  // action to perform if DB already exists:
  // - ClearDbAction.None: error will be raised if db exists
  // - ClearDbAction.Drop: db will be dropped before import
  // - ClearDbAction.Truncate: all db tables will be truncated
  actionIfDbExists: ClearDbAction.None,
}

Development

• Build the project:

npm run build

• Watch for changes and rebuild automatically:

npm run dev

Credits

• GeoNames-MySQL-DataImport for basic SQL scripts;
• node-status-bar for great status bar on long-running operations.