api-false-success

Of 73 public APIs measured, 29 return HTTP 200 for a request that cannot succeed. Only 5 of those are indefensible; 14 are legitimate documented contracts. This package records what each one actually returned, and how to tell success from failure per API.

Corrected in v0.4.0 after adversarial review. Earlier versions implied all 29 were faults. That was wrong. A 200 carrying an empty or partial body is not automatically a defect — GraphQL's HTTP specification explicitly permits execution errors and partial data inside a success response. The measurement was right; the framing was not.

The five that are indefensible

api.sunrise-sunset.org, asked for sunrise at latitude 999, longitude 999 — coordinates that cannot exist on a sphere:

{"results":{"sunrise":"12:00:00 AM","sunset":"12:00:00 AM",
            "solar_noon":"5:30:20 PM","day_length":"00:00:00"}}

HTTP 200. It does not fail — it answers. Well-formed, confident, invented. Only day_length: 00:00:00 hints, and only to someone already suspicious.

| API | Asked | Returned | |---|---|---| | sunrise-sunset | sunrise at latitude 999 | invented times | | randomuser | results=-5 — a negative number of people | actual generated humans | | thecolorapi | hex=ZZZZZZ — not hexadecimal | a structured colour, input echoed as valid | | zenquotes | a nonexistent endpoint | the error text inside the quote field — an app showing "today's quote" displays it as inspiration | | deckofcards | draw 999 cards from 52 | success:false and a populated cards array — flag and payload contradict |

Three classifications

| | Meaning | |---|---| | PROTOCOL_DEFECT (5) | No reliable discriminator, or invented data. You cannot tell it failed | | APPLICATION_CONTRACT (14) | A conventional, codeable pattern with a clear discriminator — [], an error object, success:false. Defensible | | AMBIGUOUS (10) | A discriminator exists but is nested, unusual, or easily missed — e.g. worldbank buries its error inside an array, so body.error checks miss it |

The measurement is a durable fact. The classification is judgement and may be argued with.

Use

const afs = require('api-false-success');

afs.detect('sunrise-sunset');
// { severity:'fabricates', detect:"results.day_length == '00:00:00' …" }

afs.byClassification('PROTOCOL_DEFECT');  // the five
afs.falseSuccess();                       // all 29 needing application-specific checks
afs.get('worldbank').false_success.classification_basis;

Also carries, per API: which rate-limit headers actually appear (13 of 14 in the first batch exposed none), pagination signals, error shape, and wrong-method status — which varied across six behaviours (405, 404, 403, 301, 200, connection failure). Six APIs return HTML, XML or bare text from a JSON endpoint, so the parser throws rather than branching.

Honest limits

• Convenience sample. 73 APIs chosen by hand, partly because they looked likely to be interesting. Rates describe the APIs measured — not APIs in general.
• One date, not a contract. APIs change. Re-measure before relying.
• Absence of a flag means not observed, not safe.
• No reproducibility manifest ships yet. Raw observations exist but are not published per-entry.

Method

For each API: one request that is valid, and one that is structurally valid but semantically impossible — latitude 999, the year 3000, 999 cards from a deck of 52. Appending a nonsense path only tests the router; every API passes that. Roughly three requests per API, unauthenticated, no load.

A measured counter-example is worth more than an opinion. Corrections welcome.

CC-BY-4.0.