Israeli Bank Scrapers

Scrape transactions from 19 Israeli banks and credit card companies with built-in Cloudflare WAF bypass and end-to-end PII redaction.

Maintained fork of eshaham/israeli-bank-scrapers, rewritten on a typed phase-based pipeline using Camoufox (Firefox anti-detect), Playwright, and TypeScript 6.0 strict mode.

npm install @sergienko4/israeli-bank-scrapers

Architecture at a glance

flowchart LR
    subgraph BB["Browser banks (13 pipeline)"]
      direction LR
      INIT --> HOME --> PRELOGIN["PRE-LOGIN (opt-in)"]
      PRELOGIN --> LOGIN
      LOGIN --> OTP["OTP-TRIGGER / OTP-FILL (opt-in)"]
      OTP --> AUTH["AUTH-DISCOVERY"]
      AUTH --> BIND["BIND-API-MEDIATOR"]
      BIND --> SCRAPE["API-DIRECT-SCRAPE<br/>(direct API calls — no nav / DOM walk)"]
      SCRAPE --> TERM["TERMINATE"]
    end

    subgraph API["API-direct banks (3) — OneZero · Pepper · PayBox"]
      direction LR
      CALL["API-DIRECT-CALL<br/>(login + OTP via JSON API)"] --> SCR["API-DIRECT-SCRAPE<br/>(.final emits balanceResolution)"]
    end

    BB -.->|"unified result shape"| RESULT(["IScraperScrapingResult"])
    API -.->|"unified result shape"| RESULT

Every pipeline bank scrapes direct API after login. On the browser banks (13), once AUTH-DISCOVERY proves the session, BIND-API-MEDIATOR binds an authenticated ApiMediator to the live page and API-DIRECT-SCRAPE walks a typed hard-model shape of REST/GraphQL calls — no post-auth page navigation, no DOM scraping. The api-direct banks (3) reach the same API-DIRECT-SCRAPE through API-DIRECT-CALL (headless JSON login) instead of the browser LOGIN → AUTH-DISCOVERY → BIND-API-MEDIATOR prefix. The retired generic ACCOUNT-RESOLVE → DASHBOARD → SCRAPE → BALANCE-RESOLVE chain is dormant (no bank triggers it). See Architecture for the contract that keeps the wiring honest.

📚 Full documentation: User Guide & Architecture (mkdocs) · API Reference (TypeDoc) · Changelog · Contributing

Table of Contents

• Quick Start
• Features
• Prerequisites
• Usage
  • Sample Output
• Supported Institutions
• OTP (Two-Factor Authentication)
  • Phone input contract
• Error Types
• Configuration
• Testing
• Logging & Bug Reports
  • What gets redacted, and what survives
  • Filing a bug report
  • How redaction stays correct over time
• Architecture
• Advanced Usage
• Contributing
• Version history
• Contributors
• Links
• Known Projects
• License

Quick Start

Three steps to get transactions from Bank Hapoalim:

import { CompanyTypes, createScraper } from '@sergienko4/israeli-bank-scrapers';

const scraper = createScraper({
  companyId: CompanyTypes.Hapoalim,
  startDate: new Date('2024-01-01'),
});

const result = await scraper.scrape({
  userCode: '1234567',
  password: 'mypassword',
});

if (result.success) {
  result.accounts?.forEach(acc => {
    console.log(`${acc.accountNumber}: ${acc.txns.length} txns, balance ${acc.balance}`);
  });
}

Replace userCode and password with real credentials. See Supported Institutions for other banks and credential fields, and OTP for banks that require an otpCodeRetriever callback.

Features

• WAF bypass on the first attempt — Camoufox (Firefox anti-detect) instead of Puppeteer, no stealth shims.
• Zero CSS selectors in interaction code — visible Hebrew text + a 7-strategy SelectorResolver. Site UI redesigns rarely break logins.
• Auto-detect + auto-fill OTP — either via a callback you provide or a stable long-term token (returned for API banks).
• End-to-end PII redaction — every log line, captured network body, and DOM snapshot goes through one redactor before it touches disk. Share traces publicly without leaking customer data.
• Direct-API scraping after login — once the browser proves the session, data is fetched through a typed hard-model shape of REST/GraphQL calls (BIND-API-MEDIATOR → API-DIRECT-SCRAPE), not DOM navigation. Every pipeline bank (browser and api-direct alike) reads its accounts, balances, and transactions straight from the bank's own API.
• Phase-based pipeline — isolated, single-responsibility phases; browser banks add a WAF-bypassing login front-end, api-direct banks run login as a JSON-API flow. One result shape across both paths.
• Single-phase balance ownership — the hard-model API-DIRECT-SCRAPE shape emits balanceResolution directly from each bank's balance endpoint; legitimate empty-month results pass through without a false "scrape failed" signal.
• Dual ESM + CJS — works with both import and require().
• 3 test surfaces — unit (Jest), pipeline coverage (with thresholds), mock + real E2E orchestrators.

Prerequisites

| Requirement | Minimum | Why | | -------------------- | --------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Node.js | >= 22.14.0 | ESM-by-default + node:crypto randomUUID used by the pipeline correlationId. | | npm | >= 10 | Workspaces + --access public provenance. | | OS | Windows / macOS / Linux | Camoufox ships native binaries for all three (downloaded on npm install). | | Disk | ~500 MB for the Camoufox browser bundle | Cached under ~/.cache/camoufox after first install. | | Bank credentials | Per-bank, real | See Supported Institutions. The library never registers accounts on your behalf. | | OTP callback | For banks that require it | A Promise<string> returning the SMS code. See OTP. |

Environment variables (all optional):

| Variable | Default | Effect | | --------------- | ------- | ------------------------------------------------------------------------------------------------------------ | | PII_REDACTION | on | Set to off to disable redaction for real-bank E2E only. Unit tests always run with redaction default-on. | | MOCK_MODE | unset | 1 switches test:mock to its fixture-driven path. |

Per-phase navigation timeout is controlled by the defaultTimeout scraper option (not an env var) — see Configuration → Scraper options.

Usage

import { CompanyTypes, createScraper } from '@sergienko4/israeli-bank-scrapers';

const scraper = createScraper({
  companyId: CompanyTypes.Amex,
  startDate: new Date('2024-01-01'),
});

const result = await scraper.scrape({
  id: '123456789',
  card6Digits: '123456',
  password: 'mypassword',
});

if (result.success) {
  for (const account of result.accounts!) {
    console.log(`${account.accountNumber}: ${account.txns.length} transactions`);
  }
} else {
  console.error(result.errorType, result.errorMessage);
  if (result.errorDetails) {
    console.error('Suggestions:', result.errorDetails.suggestions);
  }
}

Sample Output

{
  "success": true,
  "accounts": [
    {
      "accountNumber": "****1234",
      "balance": 0,
      "txns": [
        {
          "date": "2024-01-15",
          "description": "<merchant:12>",
          "originalAmount": -***,
          "chargedAmount": -***
        }
      ]
    }
  ]
}

This snippet shows the redacted shape produced by the redaction layer. Real balances, amounts, and merchant strings are masked; account numbers are tail-only. The balance field is populated by BALANCE-RESOLVE.final (browser banks) or API-DIRECT-SCRAPE.final (api-direct banks) — one source of truth across both paths.

Supported Institutions

19 institutions across three categories. CompanyTypes.<Name> is the discriminator passed to createScraper; credential field names are validated at runtime against the tables below.

| Institution | Engine | Credentials | | ------------------ | ---------- | ------------------------------------ | | Bank Hapoalim | Browser | userCode, password, OTP | | Bank Leumi | Browser | username, password | | Bank Otsar Hahayal | Browser | username, password, OTP | | Bank Yahav | Browser | num, nationalID, password | | Behatsdaa | Browser | id, password | | Beinleumi | Browser | username, password, OTP | | Beyahad Bishvilha | Browser | id, password | | Discount Bank | Browser | id, password, num | | Massad | Browser | username, password, OTP | | Mercantile Bank | Browser | id, password, num | | Mizrahi Bank | Browser | username, password | | One Zero | API-direct | email, password, OTP | | Pagi | Browser | username, password, OTP | | Pepper (by Leumi) | API-direct | phoneNumber, password, OTP |

| Issuer | Engine | Credentials | | -------- | ------- | ------------------------------- | | Amex | Browser | id, card6Digits, password | | Isracard | Browser | id, card6Digits, password | | Max | Browser | username, password | | Visa Cal | Browser | username, password |

| Wallet | Engine | Credentials | | ------------------------- | ---------- | ------------------ | | PayBox (by Discount Bank) | API-direct | phoneNumber, OTP |

Migration notice (wide-net policy): 3 banks are still on the legacy scraper path — Behatsdaa, Beyahad Bishvilha, Mizrahi Bank. They keep working through the same createScraper(...) entry point, but new features and bug fixes target the Pipeline architecture. They are scheduled for migration; the broader src/Scrapers/Base/, src/Common/, and legacy bank folders are also on the migration path and will be folded into src/Scrapers/Pipeline/ over time. Public API behavior is preserved.

OTP (Two-Factor Authentication)

createScraper({
  companyId: CompanyTypes.Beinleumi,
  startDate,
  otpCodeRetriever: async phoneHint => await getCodeFromUser(phoneHint),
});

Hapoalim: OTP is conditional. When the bank detects a login from an unrecognized device, it prompts for an SMS code, invoking the otpCodeRetriever callback. On remembered devices, no OTP is requested.

await scraper.scrape({
  email,
  password,
  phoneNumber: '972000000000',
  otpCodeRetriever: async () => '123456',
});
// result.persistentOtpToken — save to skip SMS next run

Phone input contract

Every API bank that accepts a phoneNumber credential reads digits-only international form (no +, no dashes). For example: '972000000000'. The pipeline edge rewrites this into each bank's wire format on the fly:

| Bank | Wire format | Example | | ------- | ------------------ | --------------- | | OneZero | international-plus | +972000000000 | | Pepper | international-flat | 972000000000 | | PayBox | international-dash | 972-000000000 |

You don't have to know which format each bank wants — pass the international digits and the mediator normalises before login.

Error Types

| Error | Meaning | | ------------------------------ | ------------------------------------------------------------------------------- | | INVALID_PASSWORD | Wrong credentials | | INVALID_OTP | Wrong/expired OTP code | | WAF_BLOCKED | Cloudflare block — check errorDetails.suggestions | | TIMEOUT | Page load timeout — increase defaultTimeout | | TWO_FACTOR_RETRIEVER_MISSING | OTP needed but no callback set | | GENERIC | Pipeline phase fail (BALANCE-RESOLVE universal miss, etc) — read errorMessage |

Camoufox passes most challenges automatically. If you still get WAF_BLOCKED:

| Scenario | Fix | | --------------------- | ----------------------------------------- | | 403 after login | Wait 1-2 hours, reduce frequency | | Datacenter IP blocked | Use residential proxy | | Turnstile CAPTCHA | Run once headed to pass initial challenge | | Parallel failures | Share browser, add 2-5s delay |

Configuration

All configuration goes through ScraperOptions at construction time and ScraperCredentials at scrape time. Nothing else reads from disk or env.

| Field | Type | Default | What it controls | | ---------------------- | ---------------------------------- | ------------------------- | ---------------------------------------------------------------------- | | companyId | CompanyTypes (enum) | (required) | Picks the bank — discriminates pipeline vs legacy path | | startDate | Date | (required) | Earliest transaction date to fetch | | defaultTimeout | number (ms) | 30000 | Per-phase navigation timeout | | navigationRetryCount | number | 0 | Retries on TIMEOUT from any phase before failing | | browserContext | Playwright.BrowserContext | Camoufox-launched per run | Reuse a shared context for parallel runs | | otpCodeRetriever | () => Promise<string> | (none) | Browser banks — callback invoked when OTP is required | | headless | boolean | true | Run Camoufox headless | | proxy | { server, username?, password? } | (none) | Residential proxy override (helps when datacenter IPs get WAF-blocked) |

Field names per bank live in the Supported Institutions tables. API banks additionally accept:

| Field | Banks | Purpose | | ------------------ | ----------------------- | --------------------------------------------------------------------------------------------- | | phoneNumber | OneZero, Pepper, PayBox | Digits-only international form (no +/-); the mediator rewrites to each bank's wire format | | otpCodeRetriever | OneZero, Pepper, PayBox | API banks — passed in credentials, not options | | otpLongTermToken | OneZero, Pepper, PayBox | Persistent token returned in result.persistentOtpToken — skip SMS on next run |

| Variable | Default | Effect | | --------------- | ------- | ------------------------------------------------------------------ | | PII_REDACTION | on | Set off for real-bank E2E only; unit tests always run default-on | | MOCK_MODE | unset | 1 switches test:mock to fixture-driven path |

| Path | Purpose | | -------------------------- | ------------------------------------------------------------------------------- | | ~/.cache/camoufox/ | Camoufox browser bundle (~500 MB, downloaded on first install) | | <cwd>/pipeline.log | Pino transcript (PII-redacted) | | <cwd>/network/*.json | Captured HTTP bodies (redacted before write) | | <cwd>/screenshots/*.html | DOM snapshots per phase (redacted in place) | | <cwd>/screenshots/*.png | Raster screenshots (not redacted — see Bug Reports) |

Testing

Three test surfaces, all copy-paste runnable:

# Run everything except E2E
npm run test:unit

Expected: ~4800 tests, 412 suites, finishes in under 4 minutes. Coverage is collected but not enforced here — see test:pipeline for thresholds.

npm run test:pipeline

Enforces global thresholds: statements ≥ 97%, branches ≥ 95%, functions ≥ 97%, lines ≥ 98%. Fails the run if any threshold is unmet. HTML coverage report lands under src/coverage/lcov-report/.

npm run test:e2e:mock

Drives the real pipeline against fixture-recorded bank responses (src/Tests/E2eMocked/<Bank>/fixtures/). Three suites pass, eleven skip when their fixtures are not present.

# All in-scope banks, parallel workers respecting per-bank sequencing
npm run test:e2e:real

# One bank only — useful when isolating a regression
npm run test:e2e:real:single -- --testPathPatterns=Amex

The orchestrator at scripts/run-real-suite.ts reads WORKER_GROUPS to decide which banks run together. Amex + Isracard share a sequential group (Amex must finish before Isracard logs in to the same customer-side session). Other banks each get their own group.

npm run lint                # eslint --max-warnings 0 + architecture + canaries + format:check
npm run lint:biome          # Biome rules (CI-equivalent)
npm run lint:dead-code      # detect-dead-code.ts — flags unused exports
npm run lint:architecture src/Scrapers/Pipeline   # cross-layer import validator
npm run type-check          # tsc --noEmit

All five exit non-zero on any finding. The pre-commit hook runs everything above plus test:pipeline, test:mock, and bank-tests in parallel — 12 gates total.

Logging & Bug Reports

The package auto-redacts PII before any line is written — terminal, log files, captured network bodies, captured DOM snapshots. You can share pipeline.log, network/*.json, or screenshots/*.html publicly without exposing your customers' data.

What gets redacted, and what survives

| Category | Example before → after | | ----------------------------------- | ------------------------------------------------- | | Account / card / Israeli ID / phone | 12-170-[REDACTED-DIGITS-6] → ***6789 | | Cardholder / customer name | דני משהו → <name:8> (length tag) | | Merchant description | סופר-פארם רמת גן → <merchant:14> | | Transaction amount | -247.50 → -*** (sign only) | | Auth tokens / cookies / OTP codes | eyJhbGc..., 123456 → [REDACTED], [OTP] | | URLs | host + path preserved; PII query keys redacted | | HTML snapshots | text nodes + value attributes scrubbed in place | | Anything unrecognized | [REDACTED] (default-deny) |

The "stable hints" (***NNNN, <merchant:N>, +***/-***, array-size markers) are deliberate — they preserve enough for us to correlate failures across phases without ever showing raw PII.

Filing a bug report

Attach all three if available:

1. pipeline.log — full Pino transcript of the run.
2. network/*.json — captured HTTP bodies (already redacted at write time).
3. screenshots/*.html — DOM snapshots per phase (already redacted).

Skip screenshots/*.png (raster images are not OCR-redacted today — they may contain unredacted PII rendered by the bank's UI). If a PNG is essential to the report, blur or crop before attaching.

How redaction stays correct over time

Two independent enforcement layers keep raw PII out of the log surface even as the codebase evolves:

• Runtime layer — PiiRedactor.ts is the single source of truth. Pino runs it as the redact.censor callback so every record is redacted before any transport. NetworkDiscovery and FixtureCapture route their byte streams through the same redactor before persisting.
• Commit-time layer — ESLint AST selectors (T09 / T16) and an architecture-validator regex (PII-Log rule) reject pull requests that try to bypass the runtime by interpolating PII identifiers into LOG.* template literals or passing full payload objects under result|accounts|transactions|... keys.

If you spot a pattern that leaks past both layers, please open an issue — that's a load-bearing bug, not cosmetic.

Architecture

Pipeline of typed phases. Each phase owns its mediator zone, its own well-known-selectors dictionary, and its own retry policy. Phases never reach into one another's state — communication happens via slim Option<T> fields on the pipeline context.

Browser banks — direct-API after login (hard model):
  INIT → HOME → [PRE-LOGIN] → LOGIN → [OTP-TRIGGER → OTP-FILL]
       → AUTH-DISCOVERY → BIND-API-MEDIATOR → API-DIRECT-SCRAPE → TERMINATE

API-direct banks (fully headless, 2 phases):
  API-DIRECT-CALL → API-DIRECT-SCRAPE

• [PRE-LOGIN] is opt-in — card banks with a separate "show login" toggle: Amex, Isracard, Max, VisaCal.
• [OTP-TRIGGER → OTP-FILL] is opt-in. Beinleumi-group banks (Beinleumi, Massad, Otsar Hahayal, Pagi) and Hapoalim use them. Hapoalim uses OTP-FILL only, conditionally.
• AUTH-DISCOVERY separates the credential exchange from the data handoff so post-auth signal capture (cookies, ids, tokens) is observable, redactable, and testable in isolation.
• BIND-API-MEDIATOR binds an authenticated ApiMediator to the live login page so the hard-model scrape dispatches REST/GraphQL calls over the same WAF-passing browser session (cookies / discovered tokens ride along). It reads the login-inclusive capture pool once to prime any bearer / session token, then hands off.
• API-DIRECT-SCRAPE replaces the retired generic ACCOUNT-RESOLVE → DASHBOARD → SCRAPE → BALANCE-RESOLVE chain for every pipeline bank — same PRE → ACTION → POST → FINAL lifecycle, but the action is a bank-supplied hard-model shape (a typed list of REST/GraphQL calls) instead of a DOM walk. .final emits ctx.balanceResolution directly. No post-auth page navigation.
• API-DIRECT-CALL replaces LOGIN [+ OTP-TRIGGER + OTP-FILL] for banks with a programmatic auth endpoint (OneZero, Pepper, PayBox). OTP, when required, is fetched via the same otpCodeRetriever callback during this phase.

Balance is resolved as part of API-DIRECT-SCRAPE: each bank's hard-model shape declares a balance call whose .final emits ctx.balanceResolution — a Map<accountNumber, number> that PipelineResult reads as the single source of truth. A universal miss (every account failed) is a real scrape failure; partial / legitimately-empty months pass through.

The standalone BALANCE-RESOLVE phase (v6) is retired for pipeline banks: it only ran on the generic ACCOUNT-RESOLVE → DASHBOARD → SCRAPE → BALANCE-RESOLVE chain, which no bank uses now. Its ESLint isolation canaries still guard the boundary so the phase cannot silently reach back into a scrape path.

| Sub-step | Responsibility | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | .pre | Read scrape.accountIdentities + scrape.balanceFetchTemplate; default-deny on absent inputs; emit balanceFetchPlan (one entry per unique bankAccountUniqueId, plus a __BULK__ entry for bulk endpoints). | | .action | Dispatch the plan under one Promise.all via api.fetchPost / api.fetchGet. Quarantine per-fetch failures (warn + downstream MISS for that bank account). Extract per-card balance from each response, supporting Visa Cal's nested-cards shape and Amex/Isracard's cardChargeNext shape. | | .post | Partition resolved vs missed cards. Hard-fail only if every card missed (universal miss = real scrape failure). Partial misses pass through. | | .final | Emit ctx.balanceResolution — a Map<accountNumber, number> that PipelineResult reads as the single source of truth. |

Three ESLint canaries (balance-resolve-isolation, no-balance-in-scrape, balance-fetch-only-in-balance-resolve) enforce the separation at commit time — any PR that reaches across the boundary fails lint.

Every api-direct bank reuses the same building blocks below the two phases, so no mediator code changes when a new bank is onboarded:

• Signer config is a discriminated union: asymmetric (ECDSA-P256 / RSA-2048, header-attached signature — Pepper) or symmetric (AES-CBC-PKCS7, signature written into the request body at an RFC-6901 pointer — PayBox). Banks declare the algorithm + canonical-string parts + key-ref in their config literal; the mediator dispatches without bank knowledge.
• Body templates are declarative JsonValueTemplate literals with $literal / $ref: creds.<field> / $ref: carry.<slot> / $ref: config.<dotted.path> tokens. The same hydration engine serves login (API-DIRECT-CALL) and scrape (API-DIRECT-SCRAPE) step bodies.
• Carry derivation at flow init — seedCarryFromCreds mirrors creds into the scope's carry slots; entries can declare a deterministic bootstrap (sha256-prefix-16 derives a stable identifier from another creds field — PayBox uses this to bind its long-term JWT to a phone-derived deviceId16Hex without asking the caller to persist it).
• Session-context bus — after login produces the final carry snapshot, the mediator publishes it via setSessionContext so the scrape phase reads post-login slots (uId, deviceId16Hex, …) through the same $ref: carry.<slot> syntax.
• Phone normaliser — every api-direct bank declares its wire format in PipelineBankConfig.headless.phoneNumberFormat (international-plus / international-flat / international-dash / local-only).
• CryptoField pre-hook — per-step optional encryption hook that takes a value from carry (e.g. the SMS OTP), AES-encrypts it with a key resolved from config.secrets.* or carry.<slot>, writes the ciphertext into the outbound body at an RFC-6901 pointer, and scrubs the plaintext from carry.
• Forensic-audit observability — both SCRAPE.post (browser banks) and API-DIRECT-SCRAPE.post (api-direct banks) call logForensicAudit, so every scrape path emits the per-account --- Account <masked> | <N> txns --- line into pipeline.log regardless of the underlying transport.

Interceptors don't own data, they observe and dismiss:

• PopupInterceptor — before HOME (and before the api-direct scrape), the interceptor scans for modal overlays (privacy banners, new-feature promos, "you have a message" dialogs) and dismisses them by visible-text.
• NetworkDiscovery + trace lifecycle — every HTTP request and response the page issues is observed and indexed. The discovery layer learns each bank's per-account / per-card / per-statement endpoints at runtime (no hand-maintained URL list) and feeds them into BIND-API-MEDIATOR (token / session priming) and the API-DIRECT-SCRAPE shape. Bodies are captured to disk only inside the configured boundary (post-auth onward) so pre-auth secrets never hit the trace; bodies + URLs flow through the central PiiRedactor before any write.

Inside the LOGIN phase, fields resolve through a 7-strategy SelectorResolver (visible Hebrew text → textContent walk-up → placeholder → aria-label → name → CSS → xpath). Once the first field matches, FormAnchor scopes the remaining fields to the discovered <form> so multi-form pages don't cross-pollute.

All institutions are configured via declarative LoginConfig objects. Adding a new bank means writing one config object — no bank-specific imperative code.

Advanced Usage

import { Camoufox } from '@hieutran094/camoufox-js';

const browser = await Camoufox({ headless: true });
const results = await Promise.all(
  banks.map(async ({ companyId, credentials }) => {
    const ctx = await browser.newContext();
    const scraper = createScraper({ companyId, startDate, browserContext: ctx });
    const result = await scraper.scrape(credentials);
    await ctx.close();
    return result;
  }),
);
await browser.close();

createScraper({
  companyId: CompanyTypes.Leumi,
  startDate,
  defaultTimeout: 60000,
  navigationRetryCount: 2,
});

- npm install israeli-bank-scrapers
+ npm install @sergienko4/israeli-bank-scrapers

Same createScraper + companyId + credentials API. Both import and require() work. Types now use I prefix (IScraper, IScraperScrapingResult) — old names still work as aliases.

The result shape now always includes accounts[].balance populated by the BALANCE-RESOLVE phase (browser banks) or the per-bank shape extractor (api-direct banks). Existing code that reads account.balance keeps working with no changes.

Contributing

Found a bug? Have an improvement? See CONTRIBUTING.md for the contribution workflow, branch strategy, and testing requirements.

Before opening a PR, run:

npm run test:unit         # all unit tests
npm run lint              # eslint + architecture + canaries + format
npm run test:pipeline     # coverage gates (97/95/97/98)

The pre-commit hook runs the same gates plus mock-E2E and bank tests. PRs are squash-merged; release-please cuts the next version automatically.

Version history

| Version | Milestone | | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | v6.7.2 | Initial fork from upstream | | v7.0.0 | Puppeteer to Playwright | | v7.9.0 | Camoufox anti-detect browser | | v7.10.0 | Full ESM migration | | v8.0.0 | Strict ESLint + JSDoc, I-prefix interfaces, form-anchor | | v8.1.0 | Integration test framework — 18 tests across 6 scrapers | | v8.2.0 | SonarCloud static-analysis workflow + Max selectors via Hebrew text | | v8.2.1 | All bank logins migrated from CSS/ID selectors to visible Hebrew text | | v8.3.0 | Pipeline architecture v2 — Strategy / Builder / Mediator / Result patterns, AUTH-DISCOVERY phase + 100% phase isolation, cross-bank test factory (Phase H), TIMING ceilings, Telegram OTP delivery, PII redaction across log/network/snapshots | | v8.4.0 | Unified api-direct primitives across OneZero / Pepper / PayBox — AES-CBC-PKCS7 body-pointer signer alongside the existing asymmetric header-attached one, declarative JsonValueTemplate bodies served by one hydration engine, seedCarryFromCreds + derivedCarry carry derivations (incl. deterministic sha256-prefix-16 bootstrap for warm-start-stable device identifiers), session-context bus method-pair, in-body cryptoField pre-hook for symmetric OTP encryption, per-bank phoneNumberFormat normaliser, forensic-audit observability hook on the api-direct scrape POST; PayBox onboarding lands as a pure consumer of these primitives.Single-phase balance ownership (BALANCE-RESOLVE v6) — SCRAPE.post emits accountIdentities + balanceFetchTemplate; BALANCE-RESOLVE owns live api.fetchPost / fetchGet, per-card extraction (Visa Cal nested cards + Amex cardChargeNext), quarantine on per-fetch failure, universal-miss hard-fail only when every card missed; ctx.balanceResolution becomes the single source of truth read by PipelineResult for both browser and api-direct paths; v5 attribution path (~370 LOC) removed; 3 new ESLint canaries lock the separation at commit time. | | v8.5.0 | Bank Leumi + Bank Yahav migrated to the Pipeline (Yahav via BaNCS Digital) — 16 of 19 institutions are now pipeline-native, 3 legacy remain, and Yahav fetches the full requested date range through monthly BaNCS OrigDt chunking (current-month bound capped at today). Config-driven BALANCE-RESOLVE with a required per-bank balanceKind (ACCOUNT vs CARD_CYCLE). Generic background WAF challenge interceptor — hCaptcha / Turnstile checkbox auto-solve. Phase 11 integration hard gate — Mode A static-HTML drive + Mode B mirror across all pipeline banks. New commit-time guards: an acyclic-dependencies cycle gate (baseline-ratcheted) and a test-duplication canary that blocks tests from copying production logic. | | v8.6.0 | Direct-API scraping after login for every pipeline bank. The 13 browser banks retire the generic ACCOUNT-RESOLVE → DASHBOARD → SCRAPE → BALANCE-RESOLVE navigation + DOM-scrape chain. After AUTH-DISCOVERY, the new BIND-API-MEDIATOR phase binds an authenticated ApiMediator to the live login page (bearer / session token primed from the capture pool), then API-DIRECT-SCRAPE walks each bank's typed hard-model shape — explicit accounts + balances + transactions REST/GraphQL calls with buildVars / extract* / pagination cursor / resultGuard — with no post-auth page navigation and no generic DOM extraction. balanceKind (ACCOUNT vs CARD_CYCLE) is emitted by the shape's .final, so all 16 pipeline institutions (13 browser + 3 api-direct) now share one direct-API scrape driver. The generic phases remain in the codebase (ESLint-canary-guarded) but no pipeline bank triggers them. |

Contributors

Thanks to the original israeli-bank-scrapers contributors whose work inspired this fork:

Links

• API Documentation (TypeDoc)
• Changelog
• Contributing
• Code of Conduct
• Security Policy

Known Projects

• israeli-bank-scrapers-to-actual-budget — Sync to Actual Budget
• Caspion — Auto-send to budget apps
• Moneyman — Save via GitHub Actions
• Firefly III Importer — Import to Firefly III

License

MIT. Maintained by @sergienko4. Based on eshaham/israeli-bank-scrapers.

This README follows the Standard README specification.