@learnshopifydev/cli

The local grader for the Shopify App Development with AI path on learnshopify.dev.

Some checkpoints on that path are about the state of your Shopify store, and those are verified in the browser. Others are about the state of your code and your toolchain: does the config parse, does the Function build, does shopify app function run produce the right output. Only something running on your machine can answer those, which is what this is.

npx @learnshopifydev/cli login     # pair this machine with your account
npx @learnshopifydev/cli doctor    # check your environment
npx @learnshopifydev/cli check     # run the checks for the lesson you are on

Commands

| Command | What it does | | --- | --- | | login | Prints a pairing code and opens your browser. Confirm it there and this machine is paired. | | doctor | Reports your Node, your Shopify CLI, whether you are in an app directory, and whether you are paired. Run this first when anything behaves oddly. | | check [lesson] | Runs the checks for a lesson. With no argument, asks the server which lesson you are on. | | logout | Forgets the credentials on this machine. | | server [url] | Shows or sets which learnshopify.dev to talk to. Students never need this; it exists for running the site locally. |

Flags

| Flag | Effect | | --- | --- | | --path <dir> | Your app directory. Defaults to the current directory. | | --json | Machine-readable output. Useful for piping into something else. | | --offline | Skips every network call. Results are queued and sync later. | | --no-submit | Runs the checks without recording the result. |

Exit codes

| Code | Meaning | | --- | --- | | 0 | Every check passed. | | 1 | A check failed. Your work is not there yet. | | 2 | A check could not run. Something is wrong with the environment, not with you. |

Three codes rather than two, because "wrong" and "could not tell" are different answers and a script piping this somewhere should be able to distinguish them.

Design notes

Three outcomes, never two

Every check returns pass, fail, or error.

error exists because the alternative is lying to you. A missing Shopify CLI, an old Node, a network that went away, a bug in one of our checks: all of those are "we could not look", and reporting any of them as a failure would tell you your correct work is wrong. That is the fastest way to lose the student who tried hardest, so we do not do it.

What this can and cannot promise

The promise is: if you have not done the work, you will not see a pass. Not through a missing file, not through empty output, not through a crashed command, not through a stale artifact, and not through a bug in one of our checks.

What it is not is tamper-proof. This runs on your machine, so its output is yours to change. Every local grader ever built has that property, and adding ceremony around it would only make the tool more annoying without making it more honest. The guarantee here is about correctness, not about defeating someone who has decided to cheat themselves.

How a check is built

Two halves, and the split is the whole design:

type Check = {
  id: string;
  label: string;
  requires?: { node?: string; shopifyCli?: string };
  collect(ctx): Promise<Evidence>;   // impure: runs commands, reads files
  assert(evidence): Verdict;         // PURE: evidence in, verdict out
};

Because assert is pure, it can run in CI against recorded evidence with no Shopify account, no dev store, and no network. That is what makes the testing discipline below possible at all.

Testing

Four layers, all in CI:

1. Runner unit tests. The mapping from what happened to what you are told: non-zero exit, timeout, missing binary, unmet version requirement, a check that throws. Each has exactly one correct outcome and a test asserting it.
2. Per-check fixture tests. Every check ships one passing fixture and at least two distinct failing ones: the "did nothing" case and a plausible near miss. CI fails a check that has no failing fixture. One fixture catches a check that always passes; it does not catch a check that only looks at whether a file exists.
3. Golden-app end-to-end. Committed reference apps under fixtures/apps/, one correct and several broken in specific ways. The real collect runs against real directories, which catches the case fixtures cannot: a collector that stopped gathering the right evidence while assert kept happily agreeing with it.
4. The no-false-pass guard. Generic "nothing happened" evidence is thrown at every check, and none of them may return pass. This is the test that found two genuine weaknesses on the day it was written.

pnpm test

Version skew

The Shopify CLI moves under us, and it auto-upgrades since 4.0, so the tool you have today may not be the tool you had yesterday.

• Missing, or a version we cannot parse: error, with the install command. Never a failure.
• Below a check's minimum: error, naming what you have, what is needed, and how to upgrade. Being on an old CLI is not a mistake in your code.
• Above the version we have tested: runs, warns, and tags the submission with your version so we see the skew in aggregate before you have to report it.

Our own version is gated the same way. A build below the server's minimumSupported refuses to run rather than produce verdicts we know are wrong.

Where credentials live

~/.config/learnshopify/config.json, mode 0600, honouring XDG_CONFIG_HOME. Deliberately not in your app directory: a dotfile next to your code eventually gets committed.

Pairing rather than a pasted token, for the same reason. A credential that travels through a clipboard ends up in a shell history or a screenshot.

Local development

pnpm install
pnpm --filter @learnshopifydev/cli build
pnpm --filter @learnshopifydev/cli test

To use it the way a student would, but against a local server, link it once and point it once:

cd packages/cli && npm link      # puts `learnshopify` on your PATH
learnshopify server http://localhost:3000
learnshopify login

npm unlink -g @learnshopifydev/cli undoes the link. While it is pointed at a localhost server the CLI prints learnshopify ... in its own remedies rather than npx @learnshopifydev/cli ..., so following its advice does not send you to fetch a different build from npm.

The checks live in src/checks/index.ts and the lessons they grade live in src/lessons/7-app-dev-with-ai/ in the main app. The two are kept in step by src/grading/grading.contract.test.ts, which fails if a lesson and its checks disagree about anything: the lesson slug, the exercise id, or which lessons have local checks at all.

License

Copyright learnshopify.dev. All rights reserved.

You may install and run this package for its intended purpose: completing the Shopify App Development path on learnshopify.dev. You may read the source, and we would rather you did, since a grader you cannot inspect is a grader you have to take on faith.

You may not redistribute it, publish a derivative under another name, or use it as the basis of a competing product.