@msar/school-website

A component of msar: Archive content from the School Website

Archive content from the School Website. Output is organized as named JSON index files for each downloaded content type, with the files themselves stored in the same directory mirroring the CDN paths. Relative file paths to content are inserted into the index files alongside the original URLs.

Install

This workflow is a subcommand of the msar tool, which can be installed using npm (or your preferred equivalent):

npm install -g msar

It depends on Node.js which provides the npm package manager tool when installed.

Usage:

  msar schoolWebsite -h --u=<username> --p=<password> --o=<outputPath> --announcements --audio --news --photoAlbums --videos --headless --devtools --quit
  --pretty --commands --silent --logging --ignoreErrors --logRequests --url=<https://example.myschoolapp.com> --sso=<sso> --mfa=<mfa>
  --viewportWidth=<viewportWidth> --viewportHeight=<viewportHeight> --opAccount=<example.1password.com> --opItem=<1Password unique identifier> --opToken=<token
  value> --logFilePath=<logFilePath> --stdoutLevel=<all|trace|debug|info|warning|error|fatal|off> --fileLevel=<all|trace|debug|info|warning|error|fatal|off>
  --concurrency=<concurrency> --rate=<rate> [...]

Arguments

-h --help

Show this usage information

School Website options

Archive content from the School Website. Output is organized as named JSON index files for each downloaded content type, with the files themselves stored in the same directory mirroring the CDN paths. Relative file paths to content are inserted into the index files alongside the original URLs.

--url=<https://example.myschoolapp.com>

URL of MySchoolApp instance (required if capturing --audio or --videos})

--announcements

Download announcements (Default: true, use --no-announcements to disable)

--audio

Download audio items, requires --url) (Default: true, use --no-audio to disable)

--news

Download news items (Default: true, use --no-news to disable)

--photoAlbums

Download photo albums (Default: true, use --no-photoAlbums to disable)

--videos

Download videos, requires --url) (Default: true, use --no-videos to disable)

Puppeteer options

--headless

Run Puppeteer's Chrome instance headless (Default: false)

--devtools

Open Chrome DevTools with the window

--quit

Quit Puppeteer's Chrome instance on successful completion (Default: true, use --no-quit to disable)

-u<username> --username=<username>

MySchoolApp username

-p<password> --password=<password>

MySchoolApp password

--sso=<sso>

MySchoolApp SSO configuration (currently only accepts "entra-id", will use the value in environment variable PUPPETEER_SSO if present)

--mfa=<mfa>

MySchoolApp MFA configuration (currently only accepts "authenticator", will use the value in environment variable PUPPETEER_MFA if present)

--viewportWidth=<n>

Default: 0

--viewportHeight=<n>

Default: 0

Sky API options

The OAuth 2.0 client_id is set from the environment variable SKY_CLIENT_ID, if present. See https://developer.blackbaud.com/apps/ for more information.

The OAuth 2.0 client_secret is set from the environment variable SKY_CLIENT_SECRET, if present.

The OAuth 2.0 scope is set from the environment variable SKY_SCOPE, if present.

The OAuth 2.0 redirect_uri, which must at least redirect to localhost, is set from the environment variable SKY_REDIRECT_URI, if present. (e.g. "http://localhost:3000/redirect")

The Sky API subscription_key is read from the SKY_SUBSCRIPTION_KEY environment variable, if present. See https://developer.blackbaud.com/subscriptions/ for more information.

The OAuth 2.0 refresh_token is read from the SKY_REFRESH_TOKEN environment variable, if present.

Output options

-o<outputPath> --outputPath=<outputPath>

Path to output directory or file to save command output, will use the value in environment variable OUTPUT_PATH if present

--pretty

Pretty print output to file (if --outputPath option is used)

1Password environment integration

If 1Password secret references are stored in the environment, a 1Password service account token is required to access the secret values.

--opAccount=<example.1password.com>

1Password account to use (if signed into multiple); will use environment variable OP_ACCOUNT if present

--opItem=<1Password unique identifier>

Name or ID of the 1Password API Credential item storing the 1Password service account token; will use environment variable OP_ITEM if present. Requires the 1Password CLI tool (https://developer.1password.com/docs/cli)

--opToken=<token value>

1Password service account token; will use environment variable OP_TOKEN if present

--commands

Include shell commands in log (Default: true, use --no-commands to disable)

--silent

Hide command output (Default: false)

--logging

Log commands and output at level debug (Default: true, use --no-logging to disable)

Logging options

--logFilePath=<logFilePath>

Path to log file (optional)

--stdoutLevel=<all|trace|debug|info|warning|error|fatal|off>

Log level to console stdout (Default: "info")

--fileLevel=<all|trace|debug|info|warning|error|fatal|off>

Log level to log file if --logFilePath provided (Default: "all")

--concurrency=<n>

The number of concurrent threads to run (Default: 1)

--rate=<n>

The number of server requests allowed per second

Workflow behavior options

--ignoreErrors

Continue run even if errors are encountered (Default: true, use --no-ignoreErrors to disable)

--logRequests

Log fetch requests and responses for analysis and debugging (Default: false)