Vision Electronic Indexing

Agent-assisted electronics parts/PCB photo indexing. The package processes images with Cloudflare Workers AI, extracts visible IC/package markings, prepares parts for datasheet lookup, and produces an enriched inventory CSV. It is intended to be used with a coding agent to populate data and exposes tools to do so with: Pi and an alternative MCP server for other agent harnesses.

Typical flow:

photos -> vision extraction -> raw JSON + evidence -> agent datasheet verification -> inventory.csv

The vision step does not perform datasheet lookup or invent part details. Datasheet enrichment is handled by a Pi agent with a web-search/browser tool or by manual review.

Which setup should I use?

• Using Pi? Install the package with pi install npm:vision-electronic-indexing-pi, then run /vision-inventory-setup.
• Using Claude Code, Codex CLI, OpenCode, Cursor, or another MCP-capable harness? Use the recommended universal installer in .universal/scripts/quick-install.sh.
• Using plain Python or manual MCP configuration? Install requirements.txt, configure Cloudflare credentials, and run vision_inventory_mcp.py directly.

Quick setup with Pi

1. Install the Pi package

pi install npm:vision-electronic-indexing-pi

2. Plan for datasheet web search

Datasheet enrichment requires a separate Pi web-search or browser tool/skill. This package intentionally does not bundle one and does not try to auto-detect one because detection is unreliable.

Examples of acceptable capabilities:

• a Brave/search Pi skill
• a browser automation skill
• another trusted web-search extension/tool

If no search/browser capability is available, the workflow can still generate parts_to_lookup.json, but it cannot verify datasheets. Fill datasheet_cache.json manually in that case.

3. Configure Cloudflare credentials

Start Pi and run:

/vision-inventory-setup

The setup command checks Python dependencies using the configured python3 command, offers to install missing packages with python3 -m pip install -r requirements.txt, warns that datasheet enrichment needs a separate web-search/browser capability, and prompts for Cloudflare Workers AI API token credentials the first time.

Credentials are stored at:

~/.pi/agent/vision-inventory/credentials.json

The file is written with chmod 600 when supported. Token input may be visible depending on your Pi UI; avoid entering credentials while screen sharing.

To change credentials later:

/vision-inventory-credentials

Environment variables also work and override stored credentials:

export CLOUDFLARE_ACCOUNT_ID=your_account_id
export CLOUDFLARE_AUTH_TOKEN=your_workers_ai_api_token
# CLOUDFLARE_API_TOKEN is also accepted as an alias:
export CLOUDFLARE_API_TOKEN=your_workers_ai_api_token

Optional model override:

export WORKERS_AI_MODEL=@cf/meta/llama-4-scout-17b-16e-instruct

Other harnesses / universal MCP compatibility

Contributed by user @Brun0-v

This repository also includes a harness-neutral compatibility layer in .universal/ for MCP-capable coding agents such as OpenCode, Claude Code, Codex CLI, Cursor, and similar clients.

The universal layer does not replace the Pi package integration. Pi users should keep using the commands above.

For other harnesses, use the universal installer:

curl -fsSL https://raw.githubusercontent.com/Pichi-Cell/vision-electronic-indexing-mcp/main/.universal/scripts/quick-install.sh -o /tmp/vei-install.sh && bash /tmp/vei-install.sh

This installs to ~/.vei/, sets up a Python venv, prompts for Cloudflare Workers AI API token credentials, installs the agent skill, and automatically configures the MCP server in your agent's settings. Requires an MCP-capable agent (OpenCode, Claude Code, Codex CLI, Cursor, etc.).

Warning: some MCP clients store environment variables in plaintext JSON config files. Prefer shell environment variables or your agent's secret storage if available.

Other harnesses can also connect directly to the Python MCP server:

python3 /path/to/vision-electronic-indexing-mcp/vision_inventory_mcp.py

1. Manual setup: install Python dependencies

From the repository root:

python3 -m pip install -r requirements.txt
# Optional for iPhone HEIC/HEIF photos:
# python3 -m pip install pillow-heif

2. Configure Cloudflare credentials

Either copy .env.example to .env in the repository root:

cp .env.example .env
# edit .env and set CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_AUTH_TOKEN
# CLOUDFLARE_AUTH_TOKEN should be your Cloudflare Workers AI API token.

or put the credentials directly in your harness MCP server configuration. Be aware that many harness config files store these values in plaintext.

3. Add the MCP server to your harness

Example config snippets are provided in:

.universal/configs/opencode.json.example
.universal/configs/claude.json.example
.universal/configs/codex.json.example
.universal/configs/cursor.json.example

Each config should point to the repository-root server file, for example:

{
  "mcpServers": {
    "vision-inventory": {
      "command": "python3",
      "args": ["/path/to/vision-electronic-indexing-mcp/vision_inventory_mcp.py"],
      "env": {
        "CLOUDFLARE_ACCOUNT_ID": "your_cloudflare_account_id",
        "CLOUDFLARE_AUTH_TOKEN": "your_cloudflare_workers_ai_api_token"
      }
    }
  }
}

The .universal/configs/*.json.example files are strict JSON; copy the appropriate file into your harness config location and adjust paths/credentials. OpenCode uses a harness-specific mcp shape; the other examples use mcpServers.

The older .universal/setup/install.sh and .universal/setup/install.ps1 scripts are manual/legacy helpers. Prefer .universal/scripts/quick-install.sh for automated universal setup.

The raw MCP server exposes these tool names:

| Tool | Purpose | |---|---| | process_image | Analyze one electronics/PCB image. | | process_image_folder | Analyze a folder of supported images. | | save_inventory | Save inventory output as JSON or quick CSV export. Use the batch workflow for full BOM/evidence output. |

4. Install the universal skill/prompt, if your harness supports them

Universal workflow assets are available at:

.universal/skills/vision-inventory-workflow/SKILL.md
.universal/prompts/vision-inventory-agent-bom.md

Copy them into your harness-specific skills/prompts location. The skill instructs the agent to run the deterministic workflow, read parts_to_lookup.json, verify datasheets with web search, fill datasheet_cache.json, regenerate the CSV with --skip-vision, and summarize uncertainties.

The deterministic workflow command is the same as the manual shell workflow:

python3 scripts/inventory_folder_to_csv.py ./photos ./output
python3 scripts/inventory_folder_to_csv.py ./photos ./output --skip-vision

Datasheet enrichment still requires a separate web-search/browser capability in the agent. This package does not bundle one.

Recommended workflow

1. Take photos

Take clear, close photos of IC groups or PCB sections. Cropped, well-lit IC close-ups work better than full-board photos for OCR.

Example folder:

photos/
  image_001.jpeg
  image_002.jpeg
  image_003.jpeg

2. Run the full agent workflow

In Pi:

/vision-inventory-agent-bom ./photos ./output

In other harnesses:

/vision-inventory-workflow ./photos ./output

Useful options:

/vision-inventory-agent-bom ./photos ./output --recursive
/vision-inventory-agent-bom ./photos ./output --limit 3
/vision-inventory-agent-bom ./photos ./output --segment-ics
/vision-inventory-agent-bom ./photos ./output --max-side 0 --jpeg-quality 85

The agent workflow will:

1. Run the image-processing batch script.
2. Write raw JSON files for auditability.
3. Build parts_to_lookup.json.
4. Search for datasheets, preferring official manufacturer sources.
5. Fill datasheet_cache.json.
6. Rerun the CSV generation with --skip-vision.
7. Summarize the BOM and call out uncertain rows.

3. Review output

Generated files:

output/
  raw/                              # one raw JSON result per image
  parts_to_lookup.json              # parts/evidence requiring datasheet lookup
  datasheet_cache.template.json     # enrichment template
  datasheet_cache.json              # agent/user-filled enrichment cache
  inventory.csv                     # deduplicated final BOM
  inventory_evidence.csv            # per-image/per-candidate evidence rows

Always review rows where:

needs_review=true
verified=false

Also review rows with notes mentioning Visual same-as hypothesis; those rows include one or more unreadable ICs counted by package/layout similarity rather than direct marking OCR.

CSV output columns

inventory.csv is deduplicated by normalized_part, the main/final part number column derived from the vision likely_part and datasheet enrichment. Multiple images, or multiple candidates from one image, can merge into one BOM row when they resolve to the same normalized_part.

Columns:

| Column | Description | |---|---| | normalized_part | Main dedupe key/final part number, usually from datasheet enrichment and based on the vision likely_part. | | candidate_parts | Candidate part numbers extracted from visual markings. | | amount | Estimated quantity for the merged BOM row. | | sighting_count | Number of evidence rows merged into this BOM row. | | description | Short datasheet-verified description. | | datasheet_url | Datasheet/source URL used for enrichment. | | manufacturer | Verified or likely manufacturer. | | verified | true only when datasheet match was verified. | | vision_confidence | Vision/OCR confidence values observed for the row. | | needs_review | true when OCR or datasheet enrichment is uncertain. | | images | Source images contributing to the row. | | observed_markings | Raw visible markings seen on packages. | | raw_json | Raw JSON files used as evidence. | | notes | Datasheet/enrichment notes or uncertainty explanations. |

inventory_evidence.csv keeps the non-deduplicated evidence rows used to build the BOM. A single photo can produce multiple evidence rows when it contains multiple different ICs.

Commands and tools

Pi commands

/vision-inventory-setup
/vision-inventory-credentials
/vision-inventory-restart
/vision-inventory-bom <image_folder> <output_dir> [options]
/vision-inventory-agent-bom <image_folder> <output_dir> [options]

Command summary:

| Command | Purpose | |---|---| | /vision-inventory-setup | Configure credentials and check dependencies. | | /vision-inventory-credentials | Change stored Cloudflare credentials. | | /vision-inventory-restart | Restart the local vision bridge process. | | /vision-inventory-bom | Run only the deterministic image-to-CSV workflow. | | /vision-inventory-agent-bom | Run the full agent-assisted workflow, including datasheet enrichment. |

Pi tools exposed to agents

| Tool | Purpose | |---|---| | vision_inventory_process_image | Analyze one electronics/PCB image. | | vision_inventory_process_folder | Analyze a folder of supported images. | | vision_inventory_save | Save inventory output as JSON or quick CSV export. For the full BOM workflow, use /vision-inventory-bom or scripts/inventory_folder_to_csv.py. |

Manual shell workflow

You can run the deterministic workflow without Pi commands:

python3 -m pip install -r requirements.txt
python3 scripts/inventory_folder_to_csv.py ./photos ./output --validate-setup
python3 scripts/inventory_folder_to_csv.py ./photos ./output
# Optional: crop dark IC-like packages before OCR, useful when ICs have mixed orientations.
python3 scripts/inventory_folder_to_csv.py ./photos ./output --segment-ics

Then fill output/datasheet_cache.json manually or with an agent, and regenerate the CSV without reprocessing images:

python3 scripts/inventory_folder_to_csv.py ./photos ./output --skip-vision

No-Cloudflare demo and sample output

A fixture-based demo is available at:

examples/no-cloudflare-demo/

Run it without Cloudflare credentials:

python3 scripts/inventory_folder_to_csv.py ./examples/no-cloudflare-demo/photos ./examples/no-cloudflare-demo/output --skip-vision

The demo includes mock raw vision JSON, a small datasheet_cache.json, and generated sample outputs:

examples/no-cloudflare-demo/output/parts_to_lookup.json
examples/no-cloudflare-demo/output/datasheet_cache.json
examples/no-cloudflare-demo/output/inventory.csv
examples/no-cloudflare-demo/output/inventory_evidence.csv

Use these files to understand the expected output shapes before running real image analysis.

Python requirements

Python 3.10 or newer is recommended.

Required versions are constrained in requirements.txt for reproducible installs:

mcp>=1.0,<2.0
requests>=2.31,<3.0
pillow>=10.0,<12.0
python-dotenv>=1.0,<2.0

Install:

python3 -m pip install -r requirements.txt

Optional HEIC/HEIF support for iPhone photos:

python3 -m pip install pillow-heif

Supported image formats:

.jpg .jpeg .png .webp .bmp .gif .heic .heif

HEIC/HEIF requires pillow-heif.

How image processing works

Before sending an image to Cloudflare Workers AI, the Python server:

1. Opens the image with Pillow.
2. Applies EXIF orientation correction.
3. Sends full resolution by default; resizes only when max_side is set to a positive value and the image is larger than that limit.
4. Converts transparency to a white background.
5. Converts the image to RGB.
6. Encodes it as JPEG.
7. Sends it as a base64 image data URL.

Defaults:

max_side: 0 (full resolution)
jpeg_quality: 85
model: @cf/meta/llama-4-scout-17b-16e-instruct

Multiple-IC and visual same-as behavior

Images may contain one IC or many different ICs. The workflow does not force all visible ICs in one image to share the same marking or part family.

The vision prompt asks the model to assign visual groups when ICs share package/body size, orientation, pin count, and board region. If an unreadable IC is visually very similar to repeated nearby readable ICs, the raw JSON may include a possible_same_as_likely_part hypothesis.

Important: visual same-as is only a quantity/grouping hypothesis. The unreadable IC keeps package_marking="unreadable", keeps likely_part="unknown", and forces needs_review=true. When the confidence is high, the batch CSV can count it under the visually matched part while adding a note such as Visual same-as hypothesis for unreadable IC(s): ....

The batch workflow builds one evidence row per image/candidate part, so one photo can contribute several BOM rows.

For photos where some IC markings are upside-down or rotated relative to others, use --segment-ics. This enables a Pillow-only segmentation pre-pass that detects dark IC-like rectangular packages, writes crops to output/crops/, and sends each crop with instructions to check 0/90/180/270 degree orientations before OCR. Segmented crop submissions are resized to max_side=250 regardless of the full-image --max-side setting. If no IC-like crops are found for an image, the workflow falls back to the full image.

Segmented crops are context-masked by default: pixels outside the detected IC body plus a small focus padding are whitened before saving the crop. This prevents partial neighboring ICs at crop edges from being counted as separate parts. Use --no-segment-mask-context only when you need to debug or preserve surrounding context.

Segmentation increases API calls because each crop is processed separately. Tune detection with --segment-max-crops, --segment-dark-threshold, --segment-detection-max-side, --segment-padding-ratio, and --segment-focus-padding-ratio if needed.

This improves handling of mixed IC photos, repeated logic chips, and rotated packages, but OCR can still miss, merge, or misread small markings. Review raw JSON and evidence rows when accuracy matters.

Datasheet enrichment rules

The agent should:

• Prefer official manufacturer datasheets or product pages.
• Keep descriptions short.
• If exact candidate search fails but official results strongly indicate a likely OCR correction, keep the original candidate as the datasheet_cache.json key and set normalized_part to the official datasheet part number.
• Example: if SN74AS283N has no official datasheet but official TI results match SN74LS283N and the image could plausibly confuse the characters, use key SN74AS283N, set normalized_part to SN74LS283N, and mention the correction in notes.
• Set verified=true for OCR corrections only when official source evidence and visual/package context make the correction highly likely.
• Set verified=false if the marking, part number, package, or source is uncertain.
• Do not invent part numbers, manufacturers, voltages, functions, or datasheet URLs.

Example datasheet_cache.json entry:

{
  "MAX232N": {
    "normalized_part": "MAX232N",
    "description": "MAX232N dual EIA-232/RS-232 driver receiver 5v DIP",
    "datasheet_url": "https://www.ti.com/lit/ds/symlink/max232.pdf",
    "manufacturer": "Texas Instruments",
    "verified": true,
    "notes": "Verified against TI datasheet."
  }
}

Local MCP server details

Internally, the vision step is implemented by vision_inventory_mcp.py, a local Python MCP stdio server. The Pi extension starts this server lazily and exposes its functionality as Pi tools.

MCP tools:

| MCP tool | Purpose | |---|---| | process_image | Analyze one image and return structured visible inventory data. | | process_image_folder | Analyze all supported images in a folder. | | save_inventory | Save inventory output to JSON or quick CSV export. Use scripts/inventory_folder_to_csv.py for the full BOM/evidence workflow. |

Run directly for MCP-compatible clients:

python3 vision_inventory_mcp.py

Example MCP client configuration:

{
  "mcpServers": {
    "vision-inventory": {
      "command": "python3",
      "args": ["/path/to/vision_inventory_mcp.py"],
      "env": {
        "CLOUDFLARE_ACCOUNT_ID": "your_cloudflare_account_id",
        "CLOUDFLARE_AUTH_TOKEN": "your_cloudflare_workers_ai_api_token"
      }
    }
  }
}

Error handling

The server returns structured errors where possible, for example:

{
  "error": true,
  "message": "Missing CLOUDFLARE_ACCOUNT_ID environment variable."
}

Handled cases include:

• Missing Cloudflare credentials.
• Invalid image paths.
• Unsupported image extensions.
• Failed image preprocessing.
• Cloudflare API errors.
• Invalid JSON from the model.
• Missing or invalid folders.
• Save/write failures.

Limitations

• Vision models can misread small, blurry, or low-contrast IC markings.
• Cropped close-ups usually help more than prompt changes.
• Datasheet enrichment depends on the quality of the installed web-search/browser tool.
• Official datasheets should still be reviewed for important work.
• The workflow deduplicates by normalized part number, not by physical component identity across overlapping photos.