piston-judger

English | 繁體中文

English

A wrapper for the Piston code execution engine with an integrated judger, built for grading submitted code.

Its distinguishing feature is fault attribution: every verdict says whether the failure belonged to the submission or to the system. A grader that cannot tell those apart will eventually charge a student for a busy machine.

Credits / Acknowledgements

This project uses the Judge Server from Piston by engineer-man. Piston is a high-performance general purpose code execution engine.

The API design for the piston client object is inspired by node-piston by dthree.

Installation

npm install piston-judger

Usage

import { pistonJudger, StatusCode, FaultAttribution } from "piston-judger";

const judger = pistonJudger({ server: "http://localhost:2000" });

const outcome = await judger.execute("c", sourceCode, {
  stdin: "10 20",
  run_timeout: 5000,
  compile_timeout: 30000,
});

const verdict = judger.judge(outcome, {
  expectedOutput: "30\n",
  compareMode: "loose",
});

if (verdict.attribution === FaultAttribution.SYSTEM) {
  // Not the student's fault. Retry, or surface it to an operator —
  // but never record it as a grade.
  return retryLater();
}

recordGrade(verdict.status); // AC | WA | TLE | MLE | OLE | RE | CE

judge() accepts everything execute() can return, including the failure arm produced when the backend is unreachable. You never need to pre-check the outcome before judging it.

Fault attribution

Read attribution first, status second. Attribution is orthogonal to the verdict, and it cannot be forgotten the way a new status code can.

| Condition | status | attribution | |-----------|----------|---------------| | Transport / backend unreachable | SE | system | | Compile stage internal error | SE | system | | Compile stage timed out | CTLE | system | | Compile stage compiler diagnostic | CE | submission | | Run stage internal error | SE | system | | Run stage timed out | TLE | submission | | Run stage output allowance exceeded | OLE | submission | | Run stage memory allowance exceeded | MLE | submission | | Run stage runtime failure | RE | submission | | Output mismatch | WA | submission | | Output match | AC | submission |

Why a compile timeout is the system's fault

A run-stage timeout is the submission's fault by definition — the allowance is the problem being set. A compile-stage timeout is genuinely ambiguous: it can be pathological template instantiation, or it can be a machine with nothing left to give. These are indistinguishable from a single result, so this library does not guess. It attributes them to the system, because being wrong that way costs a retry and being wrong the other way costs a student their marks.

Retry policy stays with you: you know the machine load and the retry count, and this library does not.

Attribution never reads submission output

stdout, stderr and output are written by the submitted program and are never inputs to an attribution decision. Only backend-authored fields (code, signal, status, message, resource usage) are. A submission that prints Killed does not receive a memory-limit verdict.

Limits

Every limit that can change a verdict is either supplied by you or reported back:

import { DEFAULT_LIMITS } from "piston-judger";

DEFAULT_LIMITS.compile_timeout; // 10000

const outcome = await judger.execute("c", source);
if (outcome.ok) {
  outcome.result.limits; // the allowances actually applied
}

Runtime catalogue

Cached with a time-to-live (default 5 minutes, override with runtimesTtlMs). A failed refresh throws PistonUnavailableError rather than silently serving a stale copy — a stale catalogue fails later and further from its cause.

What this library does not do

• It does not protect against backend saturation. It reports attribution correctly; it does not limit concurrency. Bounding concurrency is the caller's job.
• It does not reduce compilation load. One execution is one compilation; Piston has no compile cache.
• It does not retry. Retrying a system-attributed result is a policy decision that needs machine state this library cannot see.

Migrating from 0.1.x

| 0.1.x | 0.2.0 | What to do | |-------|-------|-----------| | execute() returns Result (any) | returns ExecutionOutcome, a discriminated union | Narrow on outcome.ok; delete any casts | | Pre-checking success === false before judging | judge() accepts the failure arm | Delete the pre-check | | Compile timeout ⇒ CE | ⇒ CTLE with attribution: "system" | Do not score it; retry per your policy | | Submission printing Killed ⇒ MLE | no effect | Nothing — defect removal | | ExtendedExecutionResult | folded into ExecutionResult | Update type references | | details.runInfo / details.compileInfo | details.run / details.compile, raw and unmodified | Update field access | | timeLimit / cpuTimeLimit judge options | removed (never used) | Remove them | | Compile allowance silently defaulted | reported in limits; DEFAULT_LIMITS exported | Set it explicitly |

If you switch exhaustively on StatusCode, add a CTLE arm — or better, branch on attribution first, which stays correct as verdicts are added.

繁體中文

這是一個針對 Piston 程式碼執行引擎的封裝，內建判題器 (Judger)，專為程式碼評分而設計。

它最核心的特性是失敗歸因 (fault attribution)：每一個判定都會說明這次失敗屬於學生的程式還是系統。一個無法區分兩者的評分器，遲早會把「機器太忙」記在學生頭上。

來源說明 / 銘謝

本專案使用由 engineer-man 開發的 Piston 作為 Judge Server (判題伺服器)。 Piston 是一個高效能的通用程式碼執行引擎。

piston client 物件的 API 設計參考自 dthree 的 node-piston。

安裝方式

npm install piston-judger

使用方式

import { pistonJudger, StatusCode, FaultAttribution } from "piston-judger";

const judger = pistonJudger({ server: "http://localhost:2000" });

const outcome = await judger.execute("c", sourceCode, {
  stdin: "10 20",
  run_timeout: 5000,
  compile_timeout: 30000,
});

const verdict = judger.judge(outcome, {
  expectedOutput: "30\n",
  compareMode: "loose",
});

if (verdict.attribution === FaultAttribution.SYSTEM) {
  // 不是學生的錯。可以重試或通報管理者，但絕不能記為成績。
  return retryLater();
}

recordGrade(verdict.status); // AC | WA | TLE | MLE | OLE | RE | CE

judge() 接受 execute() 可能回傳的任何值，包含後端無法連線時產生的失敗分支。你不需要在判題前自行做前置檢查。

失敗歸因

先讀 attribution，再讀 status。 歸因與判定碼是正交的；判定碼會增加，而增加時漏掉一個就會靜默誤判，但歸因欄位不會被忘記。

| 狀況 | status | attribution | |------|----------|---------------| | 傳輸層／後端無法連線 | SE | system | | 編譯階段後端內部錯誤 | SE | system | | 編譯階段逾時 | CTLE | system | | 編譯階段編譯器診斷訊息 | CE | submission | | 執行階段後端內部錯誤 | SE | system | | 執行階段逾時 | TLE | submission | | 執行階段超出輸出限制 | OLE | submission | | 執行階段超出記憶體限制 | MLE | submission | | 執行階段執行期失敗 | RE | submission | | 輸出不符 | WA | submission | | 輸出相符 | AC | submission |

為什麼編譯逾時算系統的錯

執行階段逾時在定義上就是學生的問題——時限本身就是題目的一部分。但編譯階段逾時是真正模稜兩可的：它可能是病態的 template 展開，也可能只是機器已經沒有餘力了。從單一筆結果無法分辨這兩者，所以這個函式庫不猜。它歸因為系統，因為這個方向猜錯的代價是浪費一次重試，而另一個方向猜錯的代價是學生丟分。

重試政策留給呼叫端：你知道機器負載與重試次數，而這個函式庫不知道。

歸因永遠不讀學生的輸出

stdout、stderr、output 是學生程式寫的，因此絕不作為歸因判斷的依據。只有後端撰寫的欄位（code、signal、status、message、資源用量）才會被採用。學生程式印出 Killed 不會讓自己被判 MLE。

限制值

任何會影響判定的限制值，不是由你提供，就是會回報給你：

import { DEFAULT_LIMITS } from "piston-judger";

DEFAULT_LIMITS.compile_timeout; // 10000

const outcome = await judger.execute("c", source);
if (outcome.ok) {
  outcome.result.limits; // 實際套用的限制值
}

Runtime 清單

以 TTL 快取（預設 5 分鐘，可用 runtimesTtlMs 覆寫）。重新取得失敗時會拋出 PistonUnavailableError，而不會靜默沿用過期副本——過期的清單只會讓錯誤在更晚、離成因更遠的地方以「不支援的語言」形式爆開。

這個函式庫不做的事

• 不防止後端過載。 它只保證歸因正確，不限制併發。限制併發是呼叫端的責任。
• 不減少編譯負載。 一次執行就是一次編譯；Piston 沒有編譯快取。
• 不重試。 系統性失敗要不要重試是政策決定，需要這個函式庫看不到的機器狀態。

從 0.1.x 遷移

| 0.1.x | 0.2.0 | 該怎麼改 | |-------|-------|---------| | execute() 回傳 Result（any） | 回傳 ExecutionOutcome 可辨識聯集 | 用 outcome.ok 收窄；移除 any 轉型 | | 判題前先檢查 success === false | judge() 直接接受失敗分支 | 刪掉那段前置檢查 | | 編譯逾時 ⇒ CE | ⇒ CTLE，attribution: "system" | 不要計分；依你的政策重試 | | 學生印出 Killed ⇒ MLE | 無影響 | 不用改——這是缺陷修復 | | ExtendedExecutionResult | 併入 ExecutionResult | 更新型別引用 | | details.runInfo / details.compileInfo | details.run / details.compile，原始未加工 | 更新欄位存取 | | timeLimit / cpuTimeLimit 判題選項 | 移除（從未被使用） | 刪掉 | | 編譯時限靜默採用預設 | 回報於 limits；DEFAULT_LIMITS 已匯出 | 明確設定它 |

如果你對 StatusCode 做窮舉分支，請加上 CTLE；更好的做法是先分支 attribution，這樣未來新增判定碼時仍然正確。