Download Assets

A tool for downloading and setting up data assets for building software projects.

This tool allows to specify files that should be downloaded and uncompressed or copied to specific locations within a (software) project.

The idea is, to avoid the need to manually download and uncompress / move files from public resources.

Features:

• cache downloaded files
  • detect changed file contents via checksums (sha256)
• handle "redirect" pages for public Google Drive links (i.e. handle "could not scan for viruses" warning for larger files)
• allow (automatic) conversion of text file encodings (using iconv-lite)
• for archive files:
  decompress ZIP files to specified target location
• for binary non-archive files:
  avoid file duplication by using hard links instead of "manually copying" files

Prerequisites

• Node.js version >= 16.x
  (older versions may work when disabling download progress --disable-progress)

Installation

Install from [npm][]:

npm install dl-assets

Usage

For installing from assets.json file

For using an existing assets file, use the command (by default loads assets.json):

# from package.json
download-assets -m

# with npx:
npx download-assets -m

# show available options:
npx download-assets --help

For creating assets.json file

For creating an assets file from existing files, directories, or download links, use

# from package.json
create-assets -m [file or directory or link]

# with npx:
npx create-assets -m [file or directory or link]

# show available options:
npx create-assets --help

NOTE: for assets entries created from local files or directories the url field needs to be manually added!

Example assets file (e.g. assets.json):

{
  "assets": [
    {
      "name": "hunspell-dictionary.zip",
      "url": "https://sourceforge.net/projects/aoo-extensions/files/1075/15/dict-de_de-frami_2017-01-12.oxt",
      "target": "resources/dictionary",
      "sha256": "af515015e306c634373b5382808a39691200de3439811d2f1502e9b91c23b46a",
      "type": "dictionary",
      "convertEncoding": { "source": "ISO8859-1", "target": "UTF-8" }
    }
  ]
}

Contributing

For contributing code:
See CONTRIBUTING.md.

License

MIT License, see LICENSE