What is it?

A "simple" tool to get your i18n translations out of DB, and put them into i18n JSON file(s).

Or vice versa.

Designed to be used as a part of CI/CD pipeline.

Codebase is availabe at this Repo.

Feel free to drop a feature request, proposal, or bug report.

How to use?

Via npx (cli)

1. Create config for the tool

   • see Settings chapter below for details

2. Create env file, if needed

   • see Settings chapter below for details

3. Run the command

npx -p @incrediblez/i18n-fetcher run --env=./somewhere/.env --config=./somewhere/config.json

Voila!

Via npm module

npm i --save @incrediblez/i18-fetcher

And then just include the runner.ts in your project, and use as you wish.

Settings

To run the package, you will need at least the config file.

Names and locations of the env file, and config.json file can be wherever, as long as Node process, that runs the package can reach these.

Custom locations/names can be supplied via command line arguments.

Env file

Is optional, can be used to provide "secrets" for static and persisted "configs".

Using common .env file format.

You can use it to override settings of Database connection.

You can either specify whole set of DB connection settings in config.json file, in section source -> config, or supply it via .env file.

Structure of an env file:

  host?: String
  user?: String
  password?: String
  database?: String
  port?: String

More detailed info can be found at common.ts -> type Env definition.

Minimum required info is (regardless of where it is present, in env or in config):

• host
• user
• password
• database

Creating config from the TS Type:

The definition of the Config type can be found at:

common.ts-> Config

Just follow the definition, and you'll be fine :)

Creating config using schema:

You can also create the config by just creating empty config.json file, and add reference to the JSON Schema file, which will then help you out by providing intellisence for filling-in the config.

Just add your config, wherever you need it, and refer our config.schema.json schema file from it. F.e. like this:

{
  "$schema": "./config.schema.json"
}

And then - just fill-in all the required attibutes.

Notes:

• If you would like to use the Schema without installing anything explpicitly on your PC - you can refer tot he schema file via public URL of that file in our repo.

  Something like this:

  {
    "$schema": "https://bitbucket.org/incrediblez/i18-fetcher/raw/SPECIFIC_COMMIT_SHA/config.schema.json"
  }

  • To get the actual URL (with SPECIFIC_COMMIT_SHA variable included) - you can navigate to i18-fetcher/src/main/config.schema.json in our repo, select desired commit on the left, and on the right menu select "Open raw" option. This will generate the link to the actual schema file.

• Most IDEs are supporting json schemas OOB, so - in majority of cases, you can enforce proper schema validation just by adding a refference to the Schema file. In case your IDE does not automatically highlights the validation errors - just look for a plugin, with JSON support. That should do the trick.

• In this package - schema file is compilled straight from the TS Type, which ensures integrity across whole package, and allows you to use Schema validation, or checking the actual config class from the common.ts-> Config type. Whichever is easier for you.

Config caveats:

• While filling-in the config, validator will point out that section config have to have all the parameters, required to access the DB.

  • But - you can also provide those settings in .env file, making sure they are treated as "secrets" (see above chapter about env).

  • The runtime of the package will merge .env into that section automatically, before performing actual queries on DB.

• In case you are relying on JSON Schema, and your config file is NOT in the root of the package (which is totally OK) - modify "$schema"ref to point to the config.schema.json location.

Synthetic language translations files

Package have quite oftenly needed, but not (yet) available anywhere else feature of creating "Synthetic" language translations file.

Basically it introduces "fake" language into your translation files, which, when selected in any system that relies on these translations, will populate all translation placeholders with their correlated translation keys.

In some cases it is very hard to get these keys-per-placeholders correlations out of vendors, or even your own codebase (if it grows too big).

This approach allows to introduce easy remedy for that, which can be used to speed up translations by content specialists, or debug issues with translations easily.

This feature works only on Export (creating translation files from Datasource), not vice versa.

To activate it - just provide synthetic_column_name field in your configuration. Value - any language code.

{
  "$schema": "./config.schema.json",
  "source": {
    "table_name": "my_supercool_translations",
    "setup": {
      "translation_key_column": "translation_key",
      "translation_columns": {
        "en": "labels_en",
        "fi": "labels_fi",
        ...
      },
      "synthetic_column_name": "eo" <-- THIS. Generates synthetic language translation file, with translation_keys, and calls it 'eo' (Esperanto).
    },
  }
  ...
}

Import of i18n files into DB

In case you would like to put your current translations into DB - you can do that with this tool too.

It can also update existing table, with changes from your translation files (will respect info that is already in the Table, if there is any).

Configuration (and env file) requirements are the same, and will work same way (accessing the DB configured there, including specified Table, and use LANGUAGE to COLUMNS correlations, marked there.).

The only difference is that - this mode does all that in "reverse" order.

1. Read i18n files into "common" translations object
2. Read DB data
3. Match against DB (find missing, new, or changed translations)
4. Merge those changes into DB, one-by-one.

To activate that mode, just add instruction=import flag to your command.

Like this:

npx -p @incrediblez/i18-fetcher run --env=./somewhere/.env --config=./somewhere/config.json --instruction=import

Limitations

• This mode works only with "simply"-named files.

  • Setting destination.name_template will be ignored in this mode.
  • In case you have multifile setup (one file per language), these files have to be named like LANG.json. E.g.: ./somewhere/en.json, ./somewhere/fi.json, ./somewhere/etc.json.

• This mode expects all translations to be provided in "flat" keys.

  Like this:

  {
    "en": { "translation.key.a": "en.a", "translation.key.b": "en.b", "translation.key.c": "en.c", "translation.key.d": "en.d" },
  }

  NOT like this:

  {
    "en": {
        "translation": { "key": { "a": "en.a", "b": "en.b", "c": "en.c", "d": "en.d" } }
      }
  }

  This is a subject of change, when I have time to finish this transformation handler...

• If one of your translation languages is missing translation in files, but others do not - this mode will clear out translations only for those specific languages in DB. It will not remove whole translation.

• If you would like to remove specific translation key from DB - make sure it is removed on all languages.

Development of a package

This section is designed for contributors.

Dev-related Schema updates

To update the config schema, in case you have modified something there, or added some jsdoc comments - just run:

node ./node_modules/ts-json-schema-generator/bin/ts-json-schema-generator.js --path './common.ts' --type='Config' > config.schema.json

It will update the schema with latest changes. Including comments.

Run package locally

npm run start -- --env=./.env --config=./config.json

Or, if you have your env at ./.env, and your config at ./config.json - just:

npm run start

Caveats

• Currently tool supports only MySQL2 as a source.

• Currently tool supports only one mode, when translations table contains tralsnations in columns:

  | translation_id | translation_en | translation_etc | | -------------- | -------------- | --------------- | | homepage.hello | Hello there! | ?!??!?!?!?!?!?! |

We do have plans (and initial support) for other use-cases, but as currently we are the only ones who are using the tool - we scope development to our accute needs.

Further development of the tool will depend on Community requests, and our needs :)

Please feel free to reach out to us for requests of additional features.