data-fetch-client

Node.js CLI to export Odoo model data through JSON-RPC and save it to disk with per-record folders.

What it does

• Authenticates to Odoo (common.login)
• Reads model fields (fields_get)
• Exports records from search_read (empty domain, optionally resumed from last exported ID)
• Creates one folder per record ID
• Writes main record JSON as <id>/<id>.json
• Exports related:
  • attachment_ids -> Files/ (metadata JSON + decoded binary files)
  • message_ids -> Messages/ (JSON + optional .eml for message_type=email)
• Shows live progress in terminal

Requirements

• Node.js 18+ (uses built-in fetch)

Install

Local development install (creates cxdfc command globally from this folder):

npm link

Global install from npm (after publish):

npm install -g cxdfc

Usage

cxdfc --status

cxdfc \
  --database <db-or-host> \
  --model <odoo.model.name> \
  --username <user> \
  --password '<pass>' \
  --out <output-folder>

Parameters

• --status
  • Health check, prints OK.
• --database (required)
  • Used as database name and host.
  • URL rules:
    • If value starts with http:// or https://, used directly.
    • Otherwise uses https://<value>/jsonrpc.
• --model (required)
  • Odoo model to export (example: website.support.ticket).
• --username (required)
• --password (required)
• --out (required)
  • Root folder for all export files.
• --limit <number> (optional)
  • Exports at most N records in current run.
• --eml-messages (optional)
  • Also writes .eml files for messages where message_type === "email".
• --concern-limits (optional)
  • Enables client-side throttling and 429 retry/backoff.
• --rps <number> (optional, requires --concern-limits)
  • Requests-per-second cap used by throttling.
• --csv (optional)
  • Creates/appends all_data.csv in root output with ID,Name.
• --csv-only (optional)
  • Only generates all_data.csv (ID,Name) and skips per-record folder/file export.
• --no-errors (optional)
  • Continue on errors and log to errors.csv (id,error) instead of stopping.

Output structure

Given --out /path/export:

/path/export/
  all_data.csv            (if --csv)
  errors.csv              (if --no-errors)
  <record_id>/
    <record_id>.json
    Files/
      <attachment_id>.json
      <attachment_id>_<filename>   (for binary attachments)
    Messages/
      <message_id>.json
      <message_id>.eml             (if --eml-messages and message_type=email)

With --csv-only, only all_data.csv is generated/updated (plus errors.csv if --no-errors is used).

Resume behavior

The CLI auto-resumes by scanning --out for numeric folders and finding max ID.

• If max folder is 1112, next run only requests records with id > 1112.
• CSV files are append-safe:
  • all_data.csv and errors.csv are created with headers only if missing.
• --csv-only does not use folder-based resume; it builds the CSV from query results.

Error handling

Default behavior:

• Any error stops the run.

With --no-errors:

• Record export errors are logged and skipped.
• Batch fetch errors are also handled:
  • Logs batch error.
  • Falls back to ID-only batch.
  • Fetches each record by ID one-by-one.
  • Bad records are logged; valid ones continue.

Server limit mode (--concern-limits)

When enabled:

• Client throttles requests (default 15 rps unless --rps is provided).
• On HTTP 429, retries with:
  • Retry-After header if present
  • Exponential backoff otherwise

Recommended for servers with Nginx/API rate limits.

Examples

Basic export:

cxdfc \
  --database volansyazilim.cloudoffix.com \
  --model website.support.ticket \
  --username support@cloudoffix \
  --password '123+Abc.' \
  --out /Users/purpol/Dev/other/repositories/data_fetch_client/export

Safer export with limits, EML, CSV, and skip-errors:

cxdfc \
  --database volansyazilim.cloudoffix.com \
  --model website.support.ticket \
  --username support@cloudoffix \
  --password '123+Abc.' \
  --out /Users/purpol/Dev/other/repositories/data_fetch_client/export \
  --limit 10 \
  --eml-messages \
  --concern-limits \
  --rps 10 \
  --csv \
  --no-errors

CSV only (no folder export):

cxdfc \
  --database volansyazilim.cloudoffix.com \
  --model website.support.ticket \
  --username support@cloudoffix \
  --password '123+Abc.' \
  --out /Users/purpol/Dev/other/repositories/data_fetch_client/export \
  --concern-limits \
  --rps 10 \
  --csv-only \
  --no-errors

Notes

• .eml is generated from Odoo message fields and HTML body.
• Sublime/Text editors show raw email text, not rendered HTML like mail clients.