npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

easy-jev-e2e

v0.1.6

Published

Run natural-language test case trees with Jev and Playwright

Readme

easy-jev-e2e

Run natural-language E2E cases directly from a testCaseTree with TypeSafe Jev and Playwright. The primary output is a per-case result, not generated test code.

Install

From another local project, install the package directory or the tarball produced by pnpm pack:

pnpm add file:/absolute/path/to/packages/natural-test playwright
pnpm exec playwright install chromium

After the package is published, pnpm add easy-jev-e2e playwright can be used instead. The package is not published by this repository change.

Node.js 20 or newer and Playwright 1.63 or newer are required. Configure TYPESAFE_API_KEY on the server or pass apiKey to runTestCaseTree. Do not call this package from browser-side code.

Use

import { runTestCaseTree } from 'easy-jev-e2e';

const result = await runTestCaseTree({
  url: 'http://localhost:3000/login',
  tree: {
    nodes: [
      { id: 'root', parentId: null, kind: 'root', title: '登录' },
      {
        id: 'login-error', parentId: 'root', kind: 'case', type: 'e2e',
        title: '错误密码提示',
        steps: ['在邮箱输入框填写用户邮箱', '在密码输入框填写错误密码', '点击登录按钮'],
        inputData: '{"邮箱":"[email protected]","密码":"wrong-password"}',
        expectedResult: '页面显示密码错误提示',
      },
    ],
  },
  traceDir: './test-artifacts',
});

console.log(result.cases[0].outcome); // passed | failed | blocked

The input shape is a structural subset of hartk's TestCaseTree; other projects can use it without importing hartk. Only kind: 'case' and type: 'e2e' nodes run. Each case gets a fresh Playwright browser context. storageState or beforeCase can establish authentication. The URL origin is the only allowed origin by default; set allowedOrigins if a case intentionally leaves it.

Jev selects an operation and a target from the currently observed controls. It cannot provide selectors, JavaScript, or input text. Input values come from inputData, quoted step text, or the caller's resolveInput callback. Unknown values block the case. A separate Jev judgment checks whether the final visible page state supports expectedResult; uncertain evidence produces blocked, not passed. The default action and verification probability thresholds are provisional; evaluate them on representative cases before relying on them as a release gate.

DecisionProvider is injectable, so projects can use another TypeSafe transport or test the runner without network calls. traceDir saves a Playwright trace for each case. The result includes stable caseId, outcome, action log, reason, observed page state, verification probabilities, and trace path.

When the broad decision is blocked (including a target choice of none) or a fill decision falls below minActionConfidence, Jev rechecks editable fields with focused yes/no questions. These retain observed refs, labels, ancestor context and the sanitized snapshot, including for unnamed fields. Requests use batches of up to 40 fields; recovery requires one strong match (at least 0.85) and every other field below 0.5 across all batches. The runner still applies its configured confidence gate and rechecks DOM identity. Explicit assignments such as 填写“批量操作数量上限”为 10 resolve to 10; the quoted field name is not used as the value.

Blocked or low-confidence click decisions also receive focused per-control checks, including when the broad target choice is none. A unique strong match is required across all batches under the same 0.85/0.5 policy. Matching evidence includes Chinese button labels without decorative spacing (提 交 → 提交), ancestor context and recent action/exploration history; execution retains the original label and ref. Submit-to-validate steps can click an enabled submit button while other required fields are empty. This recovery selects the requested interaction; the separate final verification still judges the expected result. Completed clicks are excluded from this fallback to avoid duplicate submissions.

Bounded exploration for missing targets

When the next target is missing or the action decision is uncertain, the runner explores and observes again before returning blocked. Each step has a separate maxExplorationsPerStep budget (default 8, set 0 to disable recovery); explorations do not spend maxActionsPerStep. After every exploration it waits explorationWaitMs (default 500 ms, configurable from 0 to 5,000 ms) for rendering/loading, captures a fresh ARIA snapshot, and retries the same step. A target revealed by the final allowed exploration can still be acted on. The budget counts attempts, including unchanged observations and recoverable exploration errors, rather than elapsed time. Existing API and click timeouts still apply.

Jev's exploration decision selects only offered capabilities: WAIT, EXPAND (collapsed ARIA disclosures and native details), OPEN_MENU (ARIA menu triggers and clickable comboboxes), SCROLL (the page or up to ten visible vertical scroll containers, including open shadow roots), and NEXT_PAGE (enabled controls with rel=next or a recognized English/Chinese next-page name). Disclosure and pagination choices must relate to the current step. Clicks retain observed DOM identity and recheck the target before execution. An attempted control or scroll position is not offered again for the same observation, preventing unchanged-content loops. Labels that do not identify a next-page action or widgets lacking disclosure semantics may require application accessibility improvements.

Custom decision providers can implement optional explore({ step, observation, candidates, recentExplorations, remainingBudget, testCase }), returning { candidateId, probability, confidence? }. Without it, recovery waits first and then tries available scrolling; it does not guess which disclosure/menu/page control to click. Missing, invalid, or low-confidence exploration choices fall back to a bounded wait (legacy providers without confidence use probability). If another wait is selected after an unchanged wait, the runner tries an offered, unvisited downward content position when available. Automatic scrolling prioritizes main content and the region containing focus; it never scrolls navigation/menu regions or reverses direction automatically. Providers can still explicitly select a relevant menu or upward scroll. choose also receives recentExplorations and minActionConfidence, so revealed UI state can count toward completing the current step and recovery can use the configured gate.

Results include explorations with the step, operation, target, original URL, whether the subsequent observation changed, and execution details/errors. On exhaustion, reason reports the consumed budget, recent attempts, and known observation gaps (truncation, omitted frames, or remaining closed disclosures). It does not equate missing evidence with target absence. Budget exhaustion applies to missing-target recovery; invalid normal actions, unsupported input, navigation outside allowed origins, and provider failures can still block independently. Scroll candidates include semantic ancestor context, main/navigation scope, focus containment, direction and geometry, so the provider can distinguish the current form from a sidebar. Visited scroll positions remain excluded across unrelated snapshot changes while geometry is unchanged. Already observed controls outside the viewport can be selected directly: Playwright scrolls them into view during click/fill. Candidate windows page through the captured main-document tree before UI exploration; iframe content remains unsupported.

ARIA observations and references

The observation uses Playwright's public page.ariaSnapshotJSON({ mode: 'ai' }) API. snapshot.tree is a semantic accessibility tree with roles, accessible names (including aria-labelledby), parent/child relationships, native refs, and states such as checked, selected, disabled, and expanded. Both action selection and outcome verification receive this tree. The final case result also retains the sanitized snapshot for inspection.

For example, two identically named buttons can be distinguished by their surrounding rows:

{
  "format": "aria",
  "tree": [{ "role": "table", "children": [
    { "role": "row", "ref": "e4", "children": [
      { "role": "cell", "name": "Alice" },
      { "role": "button", "name": "Delete", "ref": "e7" }
    ] },
    { "role": "row", "ref": "e8", "children": [
      { "role": "cell", "name": "Bob" },
      { "role": "button", "name": "Delete", "ref": "e11" }
    ] }
  ] }],
  "truncated": false,
  "omittedFrames": false
}

elements remains the list of executable candidates for custom decision providers. Its id matches a tree ref, and context identifies ancestors by role, name, and ref. Its index is only the observation order, not a selector. Refs belong to the active observation window. Decisions may include observationId and windowIndex; the runner rejects mismatches. The default Jev provider includes both fields. Before acting, the runner checks current semantics and ancestor context, then uses the exact DOM element captured during observation. Removed/replaced nodes, changed row or list-item contents, and stale observations block execution instead of resolving to a new positional match.

Each candidate window retains at most 500 nodes, a 20,000-character tree budget, and 100 executable controls. Ancestor paths are repeated in later windows and count toward those budgets. Rows and list items also carry already observed static context (up to 32 fragments and 2,000 characters per record), counted in the same window budgets; editable values and executable refs are not copied into that context. Names and text fragments are capped at 500 and 2,000 characters. A capture visits at most 10,000 nodes with at most 64 ancestors per path; these limits bound local traversal, not Playwright's initial full ARIA capture. The initial observation includes a preview of up to 30 enabled options per native select; additional options are available through the option-window API. Tree or candidate omissions set truncated; frame content is excluded and sets omittedFrames. Missing content is not evidence that an expected element is absent. Input, textarea, and contenteditable contents are represented by an empty string or [filled]; actual entered values are not evidence available to Jev. The compatibility text field is derived from the sanitized tree. Page-authored names/text and Playwright trace files are not general-purpose secret-redacted artifacts.

Candidate windows and coverage

observation.coverage reports observationId, zero-based windowIndex, hasMore, complete, and omission reasons. complete describes this single window, so the last window of a large page is still incomplete even when hasMore is false. Reasons distinguish windowing, text clipping, option clipping, omitted frames, transient references, and traversal/context/exploration limits. disclosureCount counts retained closed disclosures from visited windows.

Jev can return NEXT_WINDOW to read more evidence without changing the UI. For compatibility, BLOCKED and low-confidence decisions also advance when another window is available. none in a target Choice means no match in the offered window; it does not establish absence elsewhere. maxCandidateWindowsPerStep allows 20 additional windows per step by default; 0 disables continuation. This budget is independent of action and UI exploration budgets, and stale-state restarts consume it too. Exhaustion returns blocked with an explicit incomplete-coverage reason.

Continuations check the captured page state before resuming. A browser-local mutation tracker and property/style sentinels let unchanged pages resume without another full ARIA capture. Change signals trigger a fresh ARIA comparison; animations conservatively retain full validation. Focus-only (active) differences are ignored by both freshness and exploration fingerprints. Semantic page changes restart observation; old cursors and ordinary targets cannot execute against another window. Previously visited disclosure/menu/pagination controls retain their exact DOM bindings in a separate bounded pool of 100 controls, so they remain available for UI exploration after paging. They still undergo semantic and DOM-identity checks before execution. Reading another window alone does not count as UI progress. The deterministic unique-label click shortcut is disabled for incomplete observations.

Candidate windows support semantic region routing and option pagination. Multi-window final verification is not implemented. Final verification receives a fresh initial window and its coverage; evidence outside it may leave the case blocked.

Semantic region routing

When the initial candidate window has more content and the captured page contains at least two semantic scopes, the default Jev provider first chooses a region. Small pages and pages with only one semantic scope keep the sequential path. Supported scopes include main content, navigation, dialogs, named regions/forms, complementary content, banners, footers, tab panels, and tables. Nested scopes own their descendants; parent scopes retain their own content outside nested scopes. An Other page content entry retains content outside these boundaries, and unnamed scopes remain selectable by role and ancestor context.

A directory page offers at most 30 regions plus next and none. Summaries contain bounded names, roles, semantic ancestors, observed node counts, and whether a scope was visited; routing does not send the full snapshot. Directory discovery visits at most 10,000 nodes and 64 ancestor levels, and marks incomplete discovery explicitly. Scope summaries describe captured ARIA content, not guaranteed executable controls.

Within a region, NEXT_WINDOW continues that region. REGIONS returns to the directory; a blocked or uncertain decision at the end of a region also returns while unread scopes remain. Region traversal progress is retained, so returning to an unfinished region resumes its next window. Completed regions disappear from the directory. none, low-confidence routing, and invalid custom-provider region IDs fall back to a real offered scope, preferring unvisited scopes. They do not establish target absence or authorize an action.

maxRegionDecisionsPerStep defaults to 20 and includes directory-page decisions and stale-page restarts. It is independent of candidate-window, option-window, exploration, and action budgets. 0 blocks a step that requires region routing. Region switches acquire a new candidate window; continuation within that region uses the candidate-window budget. Reading or switching scopes does not mutate the UI. The runner checks capture freshness before switching and invalidates prior ordinary target/option references; a changed page restarts observation within the same routing budget.

Custom providers can implement optional chooseRegion({ testCase, step, directory, observation, recentActions, onRequestMetric }), returning { regionId?, next?, probability, confidence }. directory contains observationId, regions, offset, total, nextOffset?, and truncated; next: true reads another directory page. Providers without this method retain sequential candidate paging. Region observations add coverage.regionId and coverage.remainingRegions; coverage.hasMore then means more content within the selected region. Region-local coverage never establishes page-wide completeness.

Native dropdown decisions

SELECT uses two dependent decisions: first choose a dropdown from the current control window, then choose an enabled option from that dropdown. The control Choice no longer expands control-option pairs, removing the previous 200-pair truncation. Control confidence must pass the action gate before reading options; the resulting action uses the minimum control/option confidence and probability.

Only the selected dropdown gets an option catalog. Each option window contains at most 30 enabled options, with observed native indexes, labels, values, and optional optgroup labels. Disabled options and disabled optgroups are excluded. next explicitly requests another option window; none also continues when options remain and becomes BLOCKED only after the catalog is exhausted. Each Choice has at most 32 entries including these outcomes. A match can be selected before exhausting the catalog; global semantic uniqueness across windows is not guaranteed.

maxSelectOptionWindowsPerStep defaults to 20 option windows including the first, independent of candidate, exploration, and action budgets. It is shared across decisions within the step; 0 disables option reads. Exhaustion blocks with an incomplete-option-coverage reason. Native catalogs are limited to 10,000 total options, with an explicit error above that limit.

Custom providers receive readSelectOptions(targetId, cursor?) and maxSelectOptionWindows in choose. The callback returns { targetId, snapshotId, offset, total, options, nextCursor }; use the opaque nextCursor only once to continue that catalog. To execute a selection, return operation: 'SELECT', targetId, optionValue, optionIndex, and optionSnapshotId from an offered option/window, plus probability and confidence. A fresh catalog, candidate window, or observation invalidates earlier option references. Legacy providers can still return only optionValue from the initial preview when it unambiguously identifies the observed option. A directly invoked Jev provider without the callback can choose only from the options supplied by its caller.

The runner rechecks the dropdown and catalog before executing, rejecting replaced/reordered/removed options, changed labels or values, and changed disabled states. It passes the exact option DOM node to Playwright, so duplicate values cannot silently select another option. Native option nodes are not offered as CLICK targets; custom ARIA option widgets retain their existing click behavior.

Decision metrics

Each case includes metrics.candidateWindows (windows acquired for action selection or routing), metrics.selectOptionWindows (successfully read option windows), metrics.regionDecisions (directory decisions), metrics.decisionDurationMs (time spent in choose and chooseRegion, including recovery), and metrics.requests. The default Jev provider records every region, choose, select-option, recovery, exploration, and verification request, including transport failures. Each entry contains phase, elapsed milliseconds, UTF-8 bytes for the entire JSON request, question count, maximum Choice size, and transport status. inputTokens and outputTokens use actual API usage and remain absent when unavailable; bytes are not treated as tokens. Metrics contain no request bodies or API credentials. CaseResult.durationMs remains the end-to-end case duration.

Custom providers receive an optional onRequestMetric callback in chooseRegion, choose, explore, and verify; they can emit the same metadata for their transport. Older providers can ignore it and will have an empty request list.

This follows the native-ref, semantic-context, and bounded-snapshot approach in OpenClaw's browser snapshots, using Playwright's public JSON ARIA snapshot API rather than a private snapshot API or a hand-built DOM role list.

Current boundaries

The runner supports click, fill, and native select on visible controls in the main document, including open shadow roots exposed by Playwright. It does not yet handle frame contents, closed shadow roots, file uploads, drag and drop, arbitrary keyboard widgets, or visual assertions. A multi-action natural-language step has a bounded action budget. If a case needs unsupported actions or its expected result cannot be observed in the ARIA tree, title, or URL, it returns blocked for review.

Run pnpm --filter easy-jev-e2e test for transport and real-browser tests, and pnpm --filter easy-jev-e2e build for type checking and package output. Browser tests use installed Playwright Chromium or local macOS Google Chrome; install Chromium with pnpm exec playwright install chromium if neither is available. Tests inject the decision provider and do not call the live Jev API.