io-plugin-http

HTTP and HTTPS source/sink plugin for pluggable-io-framework, loaded via dynamic-plugin-framework

Key Features

• One plugin registering two provider factories, for the http and https protocols, sharing one fetch-based implementation with the js payload kind.
• A location is a full URL plus optional request options: method for writes (PUT by default, or POST), basic auth (username/password), a bearerToken, and extra headers. Credentials are marked secret in the location schema.
• Every location is a single entry; HTTP has no containers or patterns.
• Reads are GET requests. Readable handles are RangeReadable, using a Range header on a fresh request, so the framework can read in parts and seekable works.
• Writes stream the request body of one PUT or POST.
• getProperties uses HEAD (size, last modified, content type, etag) and delete uses DELETE.
• Failures are classified for the framework's retries: network errors, timeouts, throttling and server errors are transient; other client errors are permanent.

Usage

Install the plugin where a pluggable-io-framework host discovers plugins, then address entries by URL (registry is a discovered ProviderRegistry):

const { source, dest, options } = await registry.createProvidersForTransfer(
  "https://example.com/data/report.csv",
  "file:///tmp/report.csv",
);
await copy(source.provider, source.target, dest.provider, dest.target, options);

Further Details

• Configuration
• Development
• API Documentation

License

MIT © Flowscripter