strapi-provider-upload-multi-cloud-storage
v1.1.0
Published
Production-oriented Strapi 5 upload provider with unified AWS S3, Google Cloud Storage and Azure Blob Storage drivers, signed URLs, atomic collision-safe naming and HLS-aware storage.
Maintainers
Readme
strapi-provider-upload-multi-cloud-storage
A Strapi 5 upload provider with a unified interface over AWS S3, Google Cloud Storage and Azure Blob Storage.
Features
- One provider, three drivers behind a common
StorageDriverinterface. - Buffer and streaming uploads; object metadata; correct MIME + Cache-Control.
- Public/CDN URLs or signed (private) URLs — independent of filename rules.
- Atomic, collision-safe filename allocation (conditional writes; never
exists()-then-upload()). - Identity-first auth (IAM role / ADC / Managed Identity); static credentials optional.
- Startup config validation (zod) with actionable errors; no silent fallback between clouds.
- HLS zip packages: upload one
.zipand get the master playlist URL back (see below). - Exposes a shared storage core at
strapi-provider-upload-multi-cloud-storage/storagefor programmatic use.
HLS zip uploads
Upload a .zip of an HLS package (master playlist, variant playlists, segments) through the Media Library. The provider:
- extracts the zip in memory, without storing the zip itself;
- uploads every file under
<rootPath>/<folder>/<zip-name>-<random>/, keeping the zip's internal relative paths so the playlists' relative URIs still resolve (a single wrapping top-level folder is dropped); - sets the media entry's
urlto the master playlist and itsmimetoapplication/vnd.apple.mpegurl.
How the master playlist is chosen. If the zip has one .m3u8, that playlist is the master. With several, the playlist containing #EXT-X-STREAM-INF wins. Remaining ties go to the shallowest path, then by name: master.m3u8 > index.m3u8 > playlist.m3u8 > alphabetical.
Behavior and limits:
- A zip with no
.m3u8is stored as an ordinary file, as before. - Package objects are always public, even with
signedUrl.enabled, because players cannot sign relative segment URIs. The exception is.keyencryption files: they stay private, and a warning is logged. - Deleting the media entry deletes every object in the package.
__MACOSX/,.DS_Store,Thumbs.dband dotfiles are skipped. A path-traversal entry (../x) rejects the whole zip.- Only HLS file types are stored from a zip: playlists (
.m3u8,.m3u), segments (.ts,.m4s,.mp4,.m4v,.m4a,.aac,.mp3,.ac3,.ec3), init and CMAF files,.key, subtitles (.vtt,.webvtt), and trick-play images (.jpg,.jpeg,.png,.webp). Anything else (HTML, SVG, scripts, executables, nested zips) is skipped and logged. A zip therefore cannot publish active content or bypass the host app'sallowedTypes/deniedTypes. - With
hls.validatePlaylists(on by default), every URI in every playlist must be a relative path that stays inside the package. That covers plain URI lines andURI="…"attributes such as#EXT-X-KEY. Absolute or scheme URIs (https:,//host,javascript:,data:), root-relative paths and..escapes (including percent-encoded%2e%2e) reject the upload. References to files missing from the zip only log a warning. - Limits:
hls.maxFiles: at most this many files (default 2000).hls.maxFileSizeBytes: the largest single file, and also the largest zip accepted (default 1 GiB).hls.maxTotalBytes: the largest total uncompressed size (default 1 GiB).hls.maxCompressionRatio: the zip-bomb guard (default 100:1). Real HLS media is already compressed and sits near 1:1.
- Deleting a package validates every recorded key against the package root before anything is removed.
- Set
hls.enabled: falseto store zips untouched. - The host app's
upload.config.security.allowedTypesmust allowapplication/zip.
Security
src/tests/security.test.ts is a regression suite for OWASP file-upload risks. It covers zip-slip and path traversal, zip bombs and resource limits, unrestricted file types, playlist URI injection, file-name injection (XSS, SQL, CRLF, template), tampered delete records, private/signed access, and secret redaction in logs and errors. Things to configure on your side:
- Serve the bucket or CDN from its own domain (never the Strapi/admin origin), with
X-Content-Type-Options: nosniff. - If a WAF sits in front of Strapi, its request-body limit must allow your largest HLS zip.
Provider contract
init(providerOptions) returns { upload, uploadStream, delete, checkFileSize, getSignedUrl, isPrivate }.
Authored as module.exports = {...} (CommonJS) — not export default, which would compile to module.exports.default and break provider.init().
Installation
npm install strapi-provider-upload-multi-cloud-storageConfiguration
You need two files in your Strapi app:
config/plugins.tsselects the provider and its options.config/middlewares.tssets the request size limit, so large uploads such as HLS zips aren't rejected before they reach the provider, and the Content-Security-Policy, so the admin can preview media from the bucket.
config/plugins.ts
import type { Core } from '@strapi/strapi';
type Env = Core.Config.Shared.ConfigParams['env'];
/** MIME types the Media Library accepts (checked against the sniffed type). */
const allowedTypes = [
'image/*',
'video/*',
'audio/*',
'application/pdf',
'application/vnd.apple.mpegurl', // HLS playlists
'application/x-mpegurl',
'application/zip', // HLS zip packages
'application/x-zip-compressed',
'text/plain',
'text/csv',
];
/** Always rejected, even if a wildcard above would match. */
const deniedTypes = [
'image/svg+xml',
'text/html',
'application/vnd.microsoft.portable-executable',
'application/x-msdownload',
'application/x-msdos-program',
'application/x-executable',
'application/x-dosexec',
'application/x-sh',
'text/x-shellscript',
'application/x-mach-binary',
];
/** Turn one folder name into a safe key segment. */
const slug = (name: string): string =>
(name || '')
.normalize('NFKC')
.replace(/[\u0000-\u001f\u007f]/g, '')
.replace(/\s+/g, '-')
.replace(/[^\p{L}\p{N}._-]+/gu, '-')
.replace(/-{2,}/g, '-')
.replace(/^[-.]+/, '')
.slice(0, 100);
/**
* Optional: store files under the Media Library folder NAMES
* ("uploads/banners/home/…") instead of numeric ids ("uploads/1/5/…").
* Return '' to fall back to the numeric path.
*/
const folderPrefixResolver = async (folderPath: string): Promise<string> => {
const s = (globalThis as { strapi?: any }).strapi;
if (!s?.db || !folderPath || folderPath === '/') return '';
try {
const query = s.db.query('plugin::upload.folder');
const names: string[] = [];
let current = await query.findOne({ where: { path: folderPath }, populate: { parent: true } });
for (let guard = 0; current && guard < 50; guard++) {
const seg = slug(current.name);
if (seg) names.unshift(seg);
if (!current.parent?.id) break;
current = await query.findOne({ where: { id: current.parent.id }, populate: { parent: true } });
}
return names.join('/');
} catch {
return '';
}
};
const cloudOptions = (env: Env) => ({
s3: () => ({
bucket: env('S3_BUCKET'),
region: env('S3_REGION'),
accessKeyId: env('S3_ACCESS_KEY_ID'), // omit both keys to use the IAM role
secretAccessKey: env('S3_SECRET_ACCESS_KEY'),
sessionToken: env('S3_SESSION_TOKEN'),
endpoint: env('S3_ENDPOINT'), // MinIO / R2 / other S3-compatible stores
forcePathStyle: env.bool('S3_FORCE_PATH_STYLE', false),
baseUrl: env('S3_BASE_URL'), // public / CDN URL
rootPath: env('S3_ROOT_PATH', 'uploads'),
acl: env('S3_ACL'), // e.g. "public-read"; omit for "Bucket owner enforced"
}),
gcs: () => ({
bucket: env('GCS_BUCKET'),
projectId: env('GCS_PROJECT_ID'),
clientEmail: env('GCS_CLIENT_EMAIL'), // omit to use Application Default Credentials
privateKey: env('GCS_PRIVATE_KEY', '').replace(/\\n/g, '\n') || undefined,
keyFilename: env('GCS_KEY_FILENAME'),
baseUrl: env('GCS_BASE_URL'),
rootPath: env('GCS_ROOT_PATH', 'uploads'),
apiEndpoint: env('GCS_API_ENDPOINT'), // fake-gcs-server emulator
predefinedAcl: env('GCS_PREDEFINED_ACL'), // e.g. "publicRead"; omit for Uniform access
}),
azure: () => ({
containerName: env('AZURE_CONTAINER'),
accountName: env('AZURE_ACCOUNT_NAME'), // alone = Managed Identity / Entra ID
accountKey: env('AZURE_ACCOUNT_KEY'),
connectionString: env('AZURE_CONNECTION_STRING'),
serviceUrl: env('AZURE_SERVICE_URL'), // Azurite emulator
baseUrl: env('AZURE_BASE_URL'),
rootPath: env('AZURE_ROOT_PATH', 'uploads'),
}),
});
const providerOptions = (env: Env) => {
const provider = env('STORAGE_PROVIDER', 's3') as 's3' | 'gcs' | 'azure';
const selected = cloudOptions(env)[provider];
return {
provider,
signedUrl: {
enabled: env.bool('STORAGE_SIGNED_URLS', false),
expiresInSeconds: env.int('STORAGE_SIGNED_URL_TTL', 900),
},
fileNaming: {
replaceSpacesWithHyphens: env.bool('STORAGE_REPLACE_SPACES', true),
appendHashWhenSigned: env.bool('STORAGE_APPEND_HASH_WHEN_SIGNED', true),
hashLength: env.int('STORAGE_HASH_LENGTH', 8),
collisionStrategy: env('STORAGE_COLLISION_STRATEGY', 'increment'),
},
folders: {
preserveHierarchy: env.bool('STORAGE_FOLDER_PRESERVE_HIERARCHY', true),
},
folderPrefixResolver,
hls: {
enabled: env.bool('HLS_ENABLED', true),
maxFiles: env.int('HLS_MAX_FILES', 2000),
maxFileSizeBytes: env.int('HLS_MAX_FILE_SIZE_BYTES', 1024 * 1024 * 1024),
maxTotalBytes: env.int('HLS_MAX_TOTAL_BYTES', 1024 * 1024 * 1024),
maxCompressionRatio: env.int('HLS_MAX_COMPRESSION_RATIO', 100),
concurrency: env.int('HLS_CONCURRENCY', 8),
validatePlaylists: env.bool('HLS_VALIDATE_PLAYLISTS', true),
},
publicCacheControl: env('STORAGE_PUBLIC_CACHE_CONTROL', 'public, max-age=31536000, immutable'),
privateCacheControl: env('STORAGE_PRIVATE_CACHE_CONTROL', 'private, no-store'),
...(selected ? { [provider]: selected() } : {}),
};
};
export default ({ env }: Core.Config.Shared.ConfigParams): Core.Config.Plugin => ({
upload: {
config: {
provider: 'strapi-provider-upload-multi-cloud-storage',
providerOptions: providerOptions(env),
actionOptions: { upload: {}, uploadStream: {}, delete: {} },
// Largest accepted upload in bytes. Keep config/middlewares.ts in step.
sizeLimit: env.int('UPLOAD_SIZE_LIMIT', 500 * 1024 * 1024),
security: { allowedTypes, deniedTypes },
},
},
});config/middlewares.ts
import type { Core } from '@strapi/strapi';
const hostOf = (url?: string): string | undefined => {
if (!url) return undefined;
try {
return new URL(url).host;
} catch {
return undefined;
}
};
/** Hosts the media is served from, for the admin's CSP (img-src / media-src). */
const mediaHosts = (): string[] => {
const env = process.env;
const byProvider: Record<string, (string | undefined)[]> = {
s3: [
hostOf(env.S3_BASE_URL),
hostOf(env.S3_ENDPOINT),
env.S3_BUCKET ? `${env.S3_BUCKET}.s3.${env.S3_REGION || 'us-east-1'}.amazonaws.com` : undefined,
],
gcs: [hostOf(env.GCS_BASE_URL), 'storage.googleapis.com'],
azure: [
hostOf(env.AZURE_BASE_URL),
hostOf(env.AZURE_SERVICE_URL),
env.AZURE_ACCOUNT_NAME ? `${env.AZURE_ACCOUNT_NAME}.blob.core.windows.net` : undefined,
],
};
return (byProvider[(env.STORAGE_PROVIDER || 's3').toLowerCase()] ?? []).filter(
(h): h is string => Boolean(h),
);
};
/** Same value as upload.sizeLimit in config/plugins.ts. */
const uploadSizeLimit = Number(process.env.UPLOAD_SIZE_LIMIT) || 500 * 1024 * 1024;
const config: Core.Config.Middlewares = [
'strapi::logger',
'strapi::errors',
{
name: 'strapi::security',
config: {
contentSecurityPolicy: {
useDefaults: true,
directives: {
'connect-src': ["'self'", 'https:'],
'img-src': ["'self'", 'data:', 'blob:', 'market-assets.strapi.io', ...mediaHosts()],
'media-src': ["'self'", 'data:', 'blob:', 'market-assets.strapi.io', ...mediaHosts()],
upgradeInsecureRequests: null,
},
},
},
},
'strapi::cors',
'strapi::poweredBy',
'strapi::query',
{
name: 'strapi::body',
config: {
jsonLimit: '10mb',
formLimit: '10mb',
textLimit: '10mb',
formidable: {
// Without this, uploads above Strapi's 200 MB default fail with
// "exceeds the maximum request size allowed by the server".
maxFileSize: uploadSizeLimit,
},
},
},
'strapi::session',
'strapi::favicon',
'strapi::public',
];
export default config;Environment variables
STORAGE_PROVIDER=gcs # s3 | gcs | azure
STORAGE_SIGNED_URLS=false
STORAGE_SIGNED_URL_TTL=900 # seconds, max 604800 (7 days)
UPLOAD_SIZE_LIMIT=524288000 # 500 MB; used by both config files
# AWS S3 / S3-compatible
S3_BUCKET=
S3_REGION=
S3_ACCESS_KEY_ID= # leave empty to use the IAM role
S3_SECRET_ACCESS_KEY=
S3_BASE_URL= # CDN URL, e.g. https://cdn.example.com
S3_ROOT_PATH=uploads
# Google Cloud Storage
GCS_BUCKET=
GCS_PROJECT_ID=
GCS_CLIENT_EMAIL= # leave empty to use Application Default Credentials
GCS_PRIVATE_KEY=
GCS_BASE_URL=
GCS_ROOT_PATH=uploads
# Azure Blob Storage
AZURE_CONTAINER=
AZURE_ACCOUNT_NAME=
AZURE_ACCOUNT_KEY= # or AZURE_CONNECTION_STRING; or neither for Managed Identity
AZURE_BASE_URL=
AZURE_ROOT_PATH=uploadsEmpty variables are treated as "not set".
Option reference (providerOptions)
| Option | Default | Description |
|---|---|---|
| provider | — (required) | 's3', 'gcs' or 'azure'. The matching block below is required. |
| signedUrl.enabled | false | Store objects privately and sign URLs on request. HLS packages stay public. |
| signedUrl.expiresInSeconds | 900 | Signed URL lifetime, 1 to 604800. |
| fileNaming.replaceSpacesWithHyphens | true | my file.png becomes my-file.png. |
| fileNaming.appendHashWhenSigned | true | Signed mode adds a random suffix: name-1a2b3c4d.png. |
| fileNaming.hashLength | 8 | Suffix length, 4 to 64. |
| fileNaming.collisionStrategy | 'increment' | On a name clash: 'increment' (name-1.png) or 'error'. |
| folders.preserveHierarchy | true | Prefix keys with the Media Library folder. |
| folderPrefixResolver | — | (folderPath) => string, maps a folder path to a readable prefix (example above). |
| hls.enabled | true | Extract HLS zips. false stores zips as ordinary files. |
| hls.maxFiles | 2000 | Most files allowed in one zip. |
| hls.maxFileSizeBytes | 1073741824 | Largest single file, and largest zip (1 GiB). |
| hls.maxTotalBytes | 1073741824 | Largest total uncompressed size (1 GiB). |
| hls.maxCompressionRatio | 100 | Zip-bomb guard (uncompressed : zip size). |
| hls.concurrency | 8 | Parallel uploads per package, 1 to 64. |
| hls.validatePlaylists | true | Require #EXTM3U on the master and only in-package relative playlist URIs. |
| publicCacheControl | public, max-age=31536000, immutable | Cache-Control for public objects. |
| privateCacheControl | private, no-store | Cache-Control for private objects and .key files. |
| s3 | — | bucket (required), region, accessKeyId, secretAccessKey, sessionToken, endpoint, forcePathStyle, baseUrl, rootPath, acl |
| gcs | — | bucket (required), projectId, clientEmail, privateKey, keyFilename, baseUrl, rootPath, apiEndpoint, predefinedAcl |
| azure | — | containerName (required), accountName, accountKey, connectionString, serviceUrl, baseUrl, rootPath |
folders.syncWithCloud, hls.preserveRelativePaths and hls.packageHashWhenSigned are accepted but currently have no effect.
Invalid options fail at startup with a message naming each bad key.
Bucket setup for HLS playback
Your website plays HLS from the bucket domain, so the bucket needs a CORS rule allowing GET and HEAD from your site's origin. Public access must come from a bucket policy, IAM policy or ACL. On GCS, for example:
[{ "origin": ["https://www.example.com"], "method": ["GET", "HEAD"], "responseHeader": ["Content-Type", "Range"], "maxAgeSeconds": 3600 }]Apply it with gcloud storage buckets update gs://BUCKET --cors-file=cors.json.
Scripts
npm run build # tsc → dist (CJS + d.ts)
npm test # vitest (builds first)
npm run typecheck # tsc --noEmitMIT.
