goodfellow-plugin-s3

S3 storage for Goodfellow sites: the site's media kept in a bucket of any S3-compatible service, such as Amazon S3, Cloudflare R2, Backblaze B2, DigitalOcean Spaces or MinIO, instead of the site's repository. Files can be much larger, up to about 5 GB each, such as long videos, and nothing goes into the repository's history.

Content still refers to files as /media/…, so pages don't change when the site's media moves. Goodfellow lists the files the bucket keeps in content/media-storage.json, and builds show them from the bucket's public address.

It needs Goodfellow 0.6 or later, with the starters' setup for plugins' tasks: .github/workflows/plugin-task.yml on GitHub, the plugin-task job in .gitlab-ci.yml on GitLab, or the plugin-task pipeline in bitbucket-pipelines.yml on Bitbucket, which create-goodfellow writes for sites stored there. Sites made before 0.6 copy them from the starter.

How it works

• Keys: the bucket's keys are kept in the site's CI secrets, S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY (GitHub Actions secrets, GitLab CI/CD variables or Bitbucket's secured Pipelines variables), never in the admin panel, an editor's browser or the site's files, beside S3_BUCKET, the bucket and folder they're for.
• Uploading: the admin panel's Media screen hands each batch of files to the plugin, which runs in a sandboxed frame of its own. The plugin asks the site's CI to sign the uploads, with its sign task, which gets only those secrets and takes about a minute. The task answers with an address for each file that works for an hour and only for putting that file, sealed so only that admin panel can read it, and the frame sends the files straight to the bucket. Then the admin panel publishes the list of files.
• Where the screen can send files: the plugin's frame may reach any address under the services' own domains (*.amazonaws.com, *.r2.cloudflarestorage.com and the others), and a config's own services' addresses, since a frame's rules can't name only the site's bucket. It sends files only to addresses the signer gave and it checked, so that's how far a fault in the frame's own code could send them.
• Full-stack sites can keep the keys in their server's environment instead, which signs uploads straight away (see below).
• Moving the site's media: once the plugin is set up, the Media screen offers to move the files already in the site's own files to the bucket, and to move them back (files up to 25 MB), in one publish each.
• Settings: the service, the bucket, its region (or Cloudflare account) and the files' public address are in content/s3-storage.json, which holds no keys.
• What it signs: only files in the bucket and folder S3_BUCKET names, at the media library's addresses, and only the kinds of file the media library takes (photos, documents, sound and video), each served as the type its name says, whatever the browser asks for, and at the size the admin panel said, up to about 5 GB. Editors can change the settings file, so the secret, which they can't, is what keeps the keys to the site's own files: settings naming another bucket or folder get nothing signed. The signer also works out the service's address itself, from the service and its region or account, as the settings screen does, since some services read an address such as https://other-bucket.s3.amazonaws.com as another bucket whatever the path says: settings giving any other address get nothing signed. A service added in the config (see below) has its address in the secret instead. An SVG opened on its own is downloaded rather than shown, since an SVG can hold a script; pages show it as usual. The plugin's frame checks that each signed address goes to the file it asked for before sending anything.
• Deleting: removing a file from the Media screen deletes it from the bucket. On a full-stack site's server only owners can have deletes signed, or uploads over a file the site already keeps there, which would replace it (the server reads the list of files it was built with, so one added since isn't covered); but through the CI anyone who can edit the site can do both, for any file in the site's folder, since the CI can't see editors' limits. Turn on the bucket's versioning where the service has it (see below), so a deleted file can be brought back.

Installing

At a checked admin address or with a token broker, an owner installs it from Site Settings → Plugins. Otherwise a developer adds it to the site:

npm install goodfellow-plugin-s3

// goodfellow.config.tsx
import s3 from "goodfellow-plugin-s3";

export default defineConfig({
  // …
  plugins: [s3, ...installedPlugins],
});

Owners choose the service from a list: Amazon S3, Cloudflare R2, Backblaze B2, DigitalOcean Spaces, Wasabi, Akamai (Linode) Object Storage and Google Cloud Storage, and give only what it needs to find the bucket, such as its region. The admin panel can reach only those. For another service, such as your own MinIO, a developer adds it in the config, and it joins the list:

import { s3 } from "goodfellow-plugin-s3";

plugins: [
  s3({
    services: [
      {
        id: "minio",
        name: "Our MinIO server",
        endpoint: "https://minio.example.org",
        // Optional: the region it signs for (us-east-1 if not given), and
        // where its buckets' files are public, with {bucket} for the name.
        region: "us-east-1",
        publicUrl: "https://media.example.org/{bucket}",
      },
    ],
  }),
],

Its S3_BUCKET secret then starts with the service's address, such as https://minio.example.org/parish-media/site/, which the settings screen shows: the signer can't tell a service it doesn't know from the settings file, which editors can change. The address must start https:// (or http:// for localhost or 127.0.0.1), with nothing after the host; the settings screen doesn't offer a service whose address doesn't.

Setting up the bucket

The plugin's screen, Site Settings → S3 storage, walks through these steps for the service chosen, with the words its dashboard uses.

1. Create a bucket for the site, and choose its service, bucket and region on the plugin's screen.

2. Make its files public, so visitors' browsers can show them. The screen says how for the service chosen, and the address they're shown from:

   • Amazon S3, Backblaze B2, Wasabi and Google Cloud Storage give each bucket an address of their own, which works once the bucket lets anyone read its files (on Amazon S3, turn off Block public access and add a bucket policy that allows s3:GetObject to everyone; on Backblaze B2, make the bucket public; on Google Cloud Storage, give allUsers the Storage Object Viewer role).
   • DigitalOcean Spaces and Akamai (Linode) Object Storage need nothing: the plugin makes each file it uploads readable by anyone.
   • Cloudflare R2 has no address until you turn one on: in the bucket's settings, turn on its r2.dev address or connect a domain, and copy the address it shows into Public address.

   The public address can't be on the site's own domain, where a file could run as the site's pages do; a subdomain such as media.example.org is fine.

   Use an address of my own is for files shown from somewhere else, such as a domain of your own or a service that copies them closer to visitors (a CDN, such as CloudFront in front of Amazon S3). Leave it off if you're not sure.

3. Allow the admin panel's requests in the bucket's CORS settings. The plugin's frame is sandboxed, so its requests come from no origin of their own, and the rule has to allow any (the screen has this to copy, under Let the admin panel upload to the bucket):

   [
     {
       "AllowedOrigins": ["*"],
       "AllowedMethods": ["PUT", "DELETE"],
       "AllowedHeaders": ["content-type", "content-disposition", "x-amz-acl"],
       "MaxAgeSeconds": 3600
     }
   ]

   Requests are still signed, so only someone the site's CI signed for can change the bucket. Pages show files with images and players, which need no CORS rule.

4. Make keys for the site that can only put and delete files in this bucket: on Cloudflare R2, an API token with Object Read & Write for the one bucket; on Amazon S3, an IAM user with a policy like this one:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:PutObject", "s3:DeleteObject"],
         "Resource": "arn:aws:s3:::your-bucket/*"
       }
     ]
   }

   Turn on versioning where the service has it (Amazon S3, Backblaze B2, Google Cloud Storage and Wasabi, among others), so a file deleted from the Media screen, by mistake or by someone who shouldn't have, can be brought back: a delete then only hides the file, and keys like the ones above, without s3:DeleteObjectVersion, can't remove its old versions. A lifecycle rule can remove old versions after a while, so they don't cost more than they need to. Where the service has no versioning, keep a copy of the bucket elsewhere.

5. Add the keys to the site's secrets as S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY, with S3_BUCKET: the bucket's name, followed by a slash and the folder where the settings have one (parish-media, or parish-media/site/), with a service of the config's own's address first (https://minio.example.org/parish-media/site/); the screen shows it. The screen shows only the steps for this site, with a link to the page where they're added: on GitHub, the repository's Settings → Secrets and variables → Actions → New repository secret; on GitLab, Settings → CI/CD → Variables, masked and protected, since tasks run on the main branch; on Bitbucket, Repository settings → Pipelines → Repository variables, secured, with Pipelines turned on; on a full-stack site, the server's environment; in goodfellow dev and next dev, the site's .env.local (see below). The plugin's task gets only these three. On GitHub nothing else in the site's CI gets them; GitLab and Bitbucket give every job in the site's pipelines the variables, so keep only what the site needs there.

6. Save the settings. New uploads go to the bucket from then on. Try the bucket puts a small file and removes it, which checks the settings, the keys and the CORS settings together.

7. On the Media screen, Move … files to S3 storage moves the files the site already has.

Full-stack sites

A full-stack Next.js site can sign uploads on its own server, straight away, rather than through its CI. Put S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY and S3_BUCKET in the server's environment and pass the plugin's server module to the API route:

// app/api/goodfellow/[...path]/route.ts
import s3Server from "goodfellow-plugin-s3/server";

export const { GET, POST } = goodfellowApi({
  // …
  plugins: { "goodfellow-plugin-s3": s3Server },
});

It reads the bucket's settings from the server's copy of the site's files, so a change to them takes effect once the server has been rebuilt. Without the keys, the plugin signs through the CI as on any other site.

The server signs uploads for every editor signed in, and deletes for the site's owners only: editors can be limited to some of the site, and the server can't see those limits, so only owners can delete files kept in the bucket.

Trying it out locally

goodfellow dev and next dev run plugins' tasks on your computer. They read the secrets from the site's .env.local each time, which the starters' .gitignore keeps out of the site's files:

# .env.local
S3_ACCESS_KEY_ID=…
S3_SECRET_ACCESS_KEY=…
S3_BUCKET=…

Removing it

Move the media back on the Media screen first: the Plugins screen won't remove the plugin while the bucket keeps files the site shows. Files larger than 25 MB can't go into the site's own files, so pages need other files in their place before then.

Developing

npm install
npm test
npm run build

npm run build also writes goodfellow-plugin.json, what the plugin asks to do, which Goodfellow's Plugins screen shows before it's installed; commit it with any change to the plugin's permissions.

Every change that reaches the published plugin needs a changeset: run npx changeset, choose patch, minor or major, and describe the change for the plugin's users. It writes a file in .changeset/, which goes in the same pull request.

1. Run npm run release:version, which turns the changesets into a new version in package.json and an entry in CHANGELOG.md, and puts the same version in the plugin's description in src/index.ts, which a test keeps the same. Commit what it changes. Every release needs a new version: npm never lets one be published twice.

2. Once that's on main, tag it with v and the version, and push the tag, which starts publish.yml:

   git switch main && git pull
   git tag v0.1.2 && git push origin v0.1.2

   The workflow stops unless the tag matches package.json's version and is on main. A GitHub release for the tag, for its notes, is up to you: it doesn't start anything.

3. publish.yml checks and builds the plugin again in a job that can only read the repository, then, from a second job that runs none of the repository's code, stages the file it packed on npm with provenance, which the Plugins screen shows owners: which repository and commit it was built from. A staged version isn't published: nobody can install it until a maintainer approves it. The run's page lists the digest of the file it staged.

4. Approve it from your computer, with npm 11.15 or later (npm install --global npm@11), signed in to npm (npm login): npm stage list goodfellow-plugin-s3 lists what's waiting, with each one's id and digest. Compare the digest with the run's, then run npm stage approve <id>, which asks for your two-factor code and publishes it. To throw a staged version away instead, run npm stage reject <id>.

The workflow signs in to npm with trusted publishing, so the repository holds no npm token, and it may only stage versions, so nothing reaches npm without a maintainer's two-factor authentication. Set it up once, with npm 11.15 or later:

npm trust github goodfellow-plugin-s3 --file publish.yml --repo goodfellow-cms/goodfellow-plugin-s3 --allow-stage-publish --yes

Without --allow-publish, npm refuses npm publish from the workflow. A package has one trusted publisher: if it already has one that may publish, find its id with npm trust list goodfellow-plugin-s3 and remove it first with npm trust revoke --id <id> goodfellow-plugin-s3. npm drops a trusted publisher that hasn't staged anything within 2 days of being added, so add it shortly before a release. Then choose Require two-factor authentication and disallow tokens under the package's Settings → Publishing access.

Pushing a v tag is what starts a release, so keep that to maintainers, though nothing reaches npm until one approves it: under the repository's Settings → Rules → Rulesets, choose New ruleset → New tag ruleset, set Enforcement status to Active, add the target v* under Target tags, turn on Restrict creations, Restrict updates and Restrict deletions, and add the Repository admin and Maintain roles to the Bypass list. If a tag's push didn't start the workflow, such as one pushed before it ran on tags, run Publish from the Actions tab and choose the tag under Use workflow from; that works only for tags whose commit has this workflow. The Plugins screen offers a release once it's three days old.

peerDependencies names the Goodfellow releases it works with, as a range on @goodfellow-cms/core (>=0.6.0 <1.0.0). Goodfellow's Plugins screen offers a release only to sites on a Goodfellow release in that range, so narrow it in a new version if a Goodfellow release breaks the plugin. A caret range such as ^0.6.0 would mean 0.6 alone.

Reviews

Pull requests are reviewed by CodeRabbit and by Claude. Claude's review (claude-review.yml) runs Claude Code's /code-review once CI has passed on a pull request that's ready for review, on the Claude subscription behind the CLAUDE_CODE_OAUTH_TOKEN secret (made with claude setup-token). It reads the change at the commit it was chosen for, so a push while it waits or runs doesn't change what it reviews. What it finds is posted as comments on the lines (or on the file, where a line isn't part of the change), with one comment saying how the review went, what it used, and any tool it was refused (or, where GitHub dropped a waiting review because a newer one joined the queue, that it was dropped and how to ask again), which lists any finding about a line outside the change, or one GitHub didn't take as a comment, and says that the no-claude-review label passes the status where one doesn't apply.

A comment whose first line is /claude-review, in any case, asks again (/claude-review high at that level, for that review only), as does adding the claude-review label, which the review takes off afterwards, when the person asking can push here; adding claude-review:low, :high, or :xhigh asks at that level, taking off any other level label, and the level then holds for the pull request's later reviews, medium otherwise. The no-claude-review label skips the review CI starts. It never reviews pull requests from forks, whose runs get no secrets, and CI doesn't start one for a push by a bot, which Claude Code's action refuses; /claude-review still asks.

review-status.yml says how each pull request's reviews stand as two statuses, neither of them required: "CodeRabbit reviewed", which fails until CodeRabbit has reviewed the latest commit (comment @coderabbitai review to ask) and then until its comments are resolved, and "Claude reviewed", which carries Claude's latest review to later commits and fails while that review couldn't run or didn't finish or listed findings in its comment, and then until Claude's comments are resolved.

License

MIT