Grafana Visual Diff

Compare two versions of a Grafana dashboard using real browser screenshots and semantic JSON differences. Generate HTML/PDF reports with Added, Modified, and Removed panels, exact before/after settings, and visual evidence.

The tool uses your existing Grafana instance and datasource queries. It creates two temporary dashboard copies for comparison, attempts to delete only those copies, and leaves deployed dashboards and datasource definitions unchanged. It does not provision Grafana infrastructure.

Install and Upgrade

For the public npm package, where access to npmjs.org is permitted:

npm install -g grafana-visual-diff@latest --registry=https://registry.npmjs.org/
grafana-visual-diff --version
grafana-visual-diff compare --help

Both grafana-visual-diff and grafana-diff invoke the same CLI. --version reads the installed package manifest. Repeat the install command to upgrade to the latest published release; unpublished local changes are not installed by @latest.

Global installation lets you run from any directory. No project checkout, cd, npm start, or --prefix is required when input and output paths are absolute. Provide your own dashboard exports and configuration: development examples, reports, and browser-session files are not included in the npm package.

For an organization-managed package or feed, use its approved package name and registry instead. If a corporate proxy returns E404, confirm whether it exposes the package; do not change corporate registry settings or bypass organizational policy. The --registry option above applies only to that command.

Requirements

• Node.js 22.12 or newer.
• Network access to the existing Grafana endpoint and datasource.
• Grafana permissions to read the baseline and to create, read, and delete dashboards in an approved preview folder.
• Classic dashboard JSON with positive, unique, stable panel IDs, either directly or in a dashboard export envelope.
• A running Edge, Chrome, or Chromium browser with the intended Grafana session and an organization-approved loopback debugging connection.

No browser downloads or setup scripts run during npm installation. To explicitly install the pinned Playwright Chromium for isolated-browser mode:

grafana-visual-diff install-browser

This is unnecessary when reusing a supported installed browser. Linux browser system dependencies may require separate administrator setup.

Quick Start

1. Export the original and changed dashboard JSON, retaining IDs for existing panels. New IDs are treated as additions; IDs absent from the changed version are removals.
2. Create the configuration below with your existing instance, approved preview folder, time range, and variables.
3. Keep Grafana signed in in your normal Edge browser. If permitted, enable Allow remote debugging for this browser instance at edge://inspect/#remote-debugging.
4. Run the comparison and approve the browser connection prompt if it appears:

grafana-visual-diff compare \
  --before "/absolute/path/before.json" \
  --after "/absolute/path/after.json" \
  --config "/absolute/path/visual-test.yaml" \
  --browser edge \
  --output "/absolute/path/reports/review-001"

Replace the example paths with your files. The output directory must be new for each run; existing reports are never overwritten. Omitting --output uses a timestamped directory under reports in your current working directory.

Normal compare includes input validation, authentication, semantic comparison, temporary-dashboard creation, real datasource queries, browser capture, preview cleanup, and HTML/PDF generation. No --cdp-url is needed for automatic discovery of a standard browser profile, and no --approve-preview-writes is required for comparison. The old approval flag is still accepted for compatibility. Queries may incur datasource costs; use only an approved instance and folder.

Read a Deployed Baseline

Choose exactly one of --before or --baseline-uid; both modes require --after and --config:

grafana-visual-diff compare \
  --baseline-uid EXISTING_DASHBOARD_UID \
  --after "/absolute/path/after.json" \
  --config "/absolute/path/visual-test.yaml" \
  --output "/absolute/path/reports/review-002"

The deployed baseline is read only. It is not used as the destination of an import.

Preview Without Writes

Add --dry-run to either comparison command to print the semantic diff as JSON without creating previews, capturing screenshots, or generating visual reports. With two files this is offline; a baseline UID still needs authenticated read access.

For an offline diff that needs no configuration or browser:

grafana-visual-diff diff \
  --before "/absolute/path/before.json" \
  --after "/absolute/path/after.json"

Configuration

YAML and JSON are supported. This YAML example comments every active field:

# Existing Grafana base URL; use HTTPS outside loopback development.
instanceUrl: https://YOUR-INSTANCE.grafana.azure.com

# Existing folder UID for temporary copies, not a dashboard UID or a permission grant.
previewFolderUid: YOUR_PREVIEW_FOLDER_UID

# Fixed historical query window applied to both versions; choose available data.
timeRange:
  # ISO 8601 start timestamp; Z means UTC.
  from: '2026-09-15T08:00:00Z'
  # ISO 8601 end timestamp; must be later than from. Relative "now" ranges are invalid.
  to: '2026-09-15T09:00:00Z'

# Set every non-constant dashboard variable; names are case-sensitive.
# Use {} only when no values are required. Example: { region: ['west', 'east'] }
variables: {}

# Grafana appearance for both captures: light or dark.
theme: light

# IANA timezone for dashboard rendering and capture tabs.
timezone: UTC

# Browser locale for date and number formatting.
locale: en-US

# Visible browser area in CSS pixels; both versions use the same dimensions.
viewport:
  # Width: 640-3840 CSS pixels.
  width: 1440
  # Height: 480-2160 CSS pixels.
  height: 1000

# Screenshot pixels per CSS pixel (1-2); higher values use more memory.
deviceScaleFactor: 1

# Render wait budget per dashboard in milliseconds (1000-300000).
# This example allows 2 minutes; the default when omitted is 60000.
# API cleanup and report generation are separate from this budget.
timeoutMs: 120000

# Required quiet period with no tracked requests or visible loaders (100-10000 ms).
# Separate from the five-second viewport screenshot stability retry window.
settleMs: 1000

# Per-pixel color tolerance for before/after differences (0-1).
# Higher values ignore subtler differences; not a permitted changed-pixel percentage.
# Does not relax screenshot stability checks.
pixelThreshold: 0.1

Optional settings:

• storageState: saved-session path for explicit isolated-browser mode; defaults to .auth/session.json, resolved relative to the configuration file. It is ignored when attaching to an existing browser or using --profile-dir. Never commit or share this file.
• panelSelector: custom CSS selector containing {id} when Grafana uses different panel markup. The default supports [data-panelid="{id}"], [data-panel-id="{id}"], and [data-viz-panel-key="panel-{id}"].

Command-line relative paths are resolved from the current working directory. storageState is resolved from the configuration directory. Do not put passwords, tokens, or cookies in configuration.

Browser Sessions

Existing Browser: Default

compare defaults to Edge. Use --browser chrome for an existing Google Chrome profile, or --browser chromium for Chromium; these are distinct browser choices.

The CLI reads only the small DevToolsActivePort file from the browser's standard user-data directory and connects to its loopback WebSocket endpoint. Standard locations on macOS, Windows, and Linux are supported. For Chrome/Chromium, the corresponding debugging page is chrome://inspect/#remote-debugging, where supported by the browser version and policy.

The tool verifies /api/user before creating capture tabs or temporary dashboards. It reuses the signed-in profile, preserves original tabs, and does not read cookie databases or export the browser's session. Browser consent, SSO, MFA, and device-compliance rules remain under your control.

Missing, malformed, stale, or denied debugging access stops comparison. An expired Grafana session asks you to sign in manually in the existing browser and retry. Neither case launches another browser or sign-in window as a fallback. Simply having a browser open does not expose an automation connection; approved remote debugging is still required.

On macOS, an interactive discovery or connection failure attempts to open the corresponding remote-debugging settings in a new tab of an already-running browser window. For Edge, this is edge://inspect/#remote-debugging. The attempt has a five-second timeout and may be blocked by macOS automation permissions. If it cannot open the tab, the error includes the URL to open manually. No browser or profile is launched; the tab is left open for you to complete setup.

The tool cannot determine whether the checkbox is unchecked before CDP connects. Allow remote debugging for this browser instance and the subsequent connection consent remain manual. Check the setting if permitted, then rerun the command; there is no automatic retry or change to browser permissions. Successful connections and expired Grafana sessions do not trigger this setup tab. --non-interactive, non-TTY/redirected output, and other operating systems only receive the manual URL instructions.

For custom profiles or user-data directories, --cdp-url overrides automatic discovery. Supply the actual credential-free loopback HTTP base URL or ws:// browser endpoint, for example ws://127.0.0.1:9222/devtools/browser/BROWSER-ID. Replace both port and browser ID with the current values. Edge's consent-enabled endpoint may require the direct WebSocket URL when HTTP discovery returns 404.

Separate Browser: Explicit Opt-In

Only compare --isolated-browser requests a separate browser using storageState. It reuses valid saved authentication or opens a normal sign-in window if needed. That separate profile does not inherit your normal browser's login and may be restricted by corporate policy.

• --headed shows the browser started by --isolated-browser or --profile-dir.
• --non-interactive disables automatic browser setup tabs and rejects missing or expired isolated authentication instead of opening a sign-in prompt. It does not automate browser debugging consent.
• --isolated-browser and --cdp-url cannot be combined.

Standalone login and dashboards retain saved-session behavior when no CDP URL is supplied. cleanup can use saved state, an explicit CDP URL, or --profile-dir. These commands do not perform automatic existing-browser discovery.

Pre-Authenticated Persistent Profile

An existing, dedicated automation profile can be loaded directly with Playwright's persistent-context API:

grafana-visual-diff compare \
  --before "/absolute/path/before.json" \
  --after "/absolute/path/after.json" \
  --config "/absolute/path/visual-test.yaml" \
  --browser edge \
  --profile-dir "/absolute/path/pre-authenticated-automation-user-data" \
  --headed \
  --output "/absolute/path/reports/new-profile-run"

The supplied directory must already be authenticated to the target Grafana instance, exist, belong to your user, and contain Local State. It must be the user-data root, not its Default/Profile 1 subfolder or a session JSON file. Close any browser using that directory first. The command refuses standard personal Edge/Chrome/Chromium profiles and does not remove locks, copy normal-profile credentials, or create a fresh profile if the directory is missing.

This mode does not use CDP attachment, an extension, or credential extraction. It loads browser-managed authentication, verifies /api/user, and by default fails instead of asking for login when the session is invalid. It does not prepare a pre-authenticated profile for you or bypass SSO/device policy. No remote-debugging checkbox is needed; Playwright controls only the browser it launches. Treat the directory as sensitive: Chromium may update its persisted state during use.

--profile-dir cannot be combined with --cdp-url or --isolated-browser; --headed is optional. storageState is ignored and no separate cookie-state file is exported. If you have an authenticated session JSON rather than a profile directory, use storageState with --isolated-browser --non-interactive instead. Default comparison without either explicit mode still attaches to the running browser.

For comparison only, add --select-account "[email protected]" --headed to opt in to selecting a saved Microsoft account when Grafana requires sign-in. Supply the exact email displayed on the tile. The tool opens Grafana in the same dedicated browser and clicks at most one visible matching account tile in Microsoft's #tilesHolder picker on https://login.microsoftonline.com; it never chooses a different account. A missing tile or changed picker layout requires manual selection; multiple matches stop the operation. Passwords, MFA, and consent remain manual, with a five-minute sign-in deadline. This option conflicts with --non-interactive.

After browser authentication and a subsequent API check pass, comparison continues in that same browser without a restart. Normal dashboard requests and datasource queries are allowed; no authentication-probe request blocking is installed. Account selection does not bypass Microsoft policy or guarantee that sessions stay valid. Without this explicit option, profile mode still fails on an invalid session. Standalone cleanup retains its existing fail-without-login behavior.

Capture and Stability

Screenshots come from actual Grafana rendering, not reconstructed charts. The tool keeps the dashboard's scroll-container dimensions intact, visits overlapping viewports, waits for tracked queries and visible loading indicators, and stitches accepted viewport strips. Kiosk mode reduces navigation noise; fixed footers are included once and sticky headers are excluded from overlaps.

After settling, each viewport has a five-second stability retry window, capped by the remaining dashboard render budget. It samples with up to 200 ms between screenshots and accepts the first consecutive byte-identical pair. Stable viewports finish early. The strict comparison is unchanged; pixelThreshold applies only to differences between the before and after evidence.

The Node-side render deadline also covers pending browser calls, fonts, animation-frame waits, response inspection, and screenshots. A stalled browser cannot leave capture waiting indefinitely. CDP teardown has a bounded grace period, and the API session stays available for preview cleanup.

Coverage gaps, clipping, changing layout, missing required panel IDs, unpainted images, and query/plugin errors are reported. Increasing timeoutMs alone does not extend the five-second per-viewport retry window. A viewport that keeps repainting may still fail; keep the browser visible and avoid interacting with capture tabs.

Long or Partially Captured Dashboards

Fully covered panel PNGs are retained as stable viewports arrive, including panels spanning multiple contiguous viewports. A later failure does not erase earlier verified panel evidence. An accepted dashboard prefix may be retained as partial dashboard context; otherwise a diagnostic viewport may be available. Incomplete panel slices never count as complete evidence.

Full capture is still attempted first. The result can complete with changed-panel scope when every changed and explicitly selected panel has all required images:

| Panel change | Required evidence | | --- | --- | | Modified | Before and after | | Added | After only | | Removed | Before only | | Extra ID selected with --panels | Each side on which it exists |

--panels 3,8 adds IDs to the automatically detected changes; it does not limit the dashboard sweep or replace required changed panels.

Missing required images still fail. Dashboard-level changes, selected row changes, and comparisons with no selected panel changes require full-dashboard coverage. Query/plugin/authentication errors, unresolved API inspection at timeout, cancellation, and cleanup/report failures are not converted into passes. If the first viewport fails before any required images are saved, the fallback cannot help.

Reduced scope is explicitly labeled in the terminal and HTML/PDF. Its original coverage failure remains a warning. Full-dashboard pixel counts stay unavailable, and uncaptured unchanged panels are not verified. Partial or diagnostic context is never used as a complete-dashboard comparison.

Reports and Terminal Output

HTML/PDF reports group evidence into Added Panels, Modified Panels, and Removed Panels. Unexpected visual changes in unchanged panel definitions appear separately, without being called configuration changes.

Within each group, panels follow dashboard position: top to bottom, then left to right. Added/modified panels use the after layout; removed panels use the baseline layout. Definitions without grid positions follow positioned panels in export order. Titles and aligned before/after screenshots appear first, followed by exact changed settings.

PDF output uses A4 print typography, consistent margins, clearer tables, and larger uncropped evidence images. Subsequent populated groups and subsequent panels start on new pages. Each title stays with its screenshot pair where it fits, while long settings tables can continue across pages.

The main report keeps clean before/after images and dashboard context, without panel overlay viewers, a complete-dashboard difference viewer, or a Run Details section. Technical metadata and difference images remain in separate artifacts. HTML embeds its images and styles, so it opens locally without a server or internet access.

Diagnostics, report paths, and cleanup recovery information appear before the final terminal summary. That summary includes a colored COMPLETED or FAILED status, a before/after comparison diagram, change counts, available full-dashboard pixel counts, duration, cleanup status, and reduced scope when applicable. Narrow terminals use a stacked diagram. Color is disabled for non-TTY output, unsupported terminals, NO_COLOR, NODE_DISABLE_COLORS, FORCE_COLOR=0, or TERM=dumb. Completed summaries go to stdout; failures go to stderr. Offline diff and dry-run output stays JSON.

Output Files

| Artifact | Contents | | --- | --- | | report.html, report.pdf | Human-readable comparison reports when generation succeeds | | semantic-diff.json | Panel and dashboard property changes | | result.json | Status, errors, warnings, counts, metadata, timestamps, and capture scope/coverage | | cleanup.json | Recovery manifest containing only this run's preview UIDs and cleanup states | | before.png, after.png | Complete, partial, or diagnostic dashboard context when available; check the recorded scope | | panel-ID-before.png, panel-ID-after.png | Available individual panel evidence | | diff.png, changes.png | Full-dashboard difference images, only when both complete captures are available | | panel-ID-diff.png, panel-ID-changes.png | Panel differences when both panel images exist |

result.json records comparisonScope as full-dashboard, changed-panels, or incomplete. Per-side captureCoverage lists expected, captured, required, and missing panel IDs. Early failures may not create all artifacts.

Exit code 0 means full comparison completed or all required panel evidence completed with an explicitly reduced scope. Pixel differences alone do not fail the command. Missing required evidence and execution, report, or cleanup errors return a nonzero exit code. Existing saved reports are not restyled or regenerated by installing an update.

Cleanup and Cancellation

Only new uniquely tagged previews are imported, with random UIDs, id: null, and overwrite disabled. Each UID is recorded before its import is attempted. Cleanup checks the recorded UID, folder, and ownership tags before deleting and verifies a subsequent GET returns HTTP 404. It does not scan for or delete unrelated dashboards.

Cleanup runs in finally, after capture and before report generation. A failed import or capture therefore still triggers cleanup. An in-flight import is awaited before reconciliation, including when its response is lost.

During comparison, SIGINT, SIGTERM, and SIGHUP request a controlled stop. Repeated signals do not skip preview cleanup or close the API session early. When cancellation is observed before report generation and an output directory exists, cleanup results and failure JSON are retained without starting HTML/PDF generation.

Deletion remains best-effort: SIGKILL, power loss, network failure, expired authentication, or insufficient permissions can prevent it. Keep the cleanup manifest until every listed preview is resolved. To retry using an existing browser:

grafana-visual-diff cleanup \
  --manifest "/absolute/path/reports/review-001/cleanup.json" \
  --cdp-url "ws://127.0.0.1:9222/devtools/browser/BROWSER-ID" \
  --approve-preview-writes

Replace the endpoint with the current approved connection. Unlike compare, standalone cleanup still requires --approve-preview-writes. For a deliberately saved tool session, replace --cdp-url with --state "/absolute/path/session.json". For persistent-profile runs, replace it with the same --browser edge --profile-dir "/absolute/path/pre-authenticated-automation-user-data"; profile authentication is verified without login fallback.

Reports, errors, variable values, and saved sessions may contain confidential operational data. Output directories and saved-session files use owner-only permissions. Browser credentials are excluded from reports; report text is HTML-escaped. Follow your organization's access, storage, and retention policies.

Additional Commands

| Command | Purpose | | --- | --- | | compare | Full visual workflow; add --dry-run for semantic-only output | | diff | Offline two-file semantic comparison; no config or browser required | | dashboards | Verify folder access and list dashboard titles/UIDs; requires --url and --folder-uid | | login | Explicitly prepare or verify authentication; requires --url | | cleanup | Retry deletion of previews in one manifest after explicit approval | | install-browser | Explicitly install the pinned Playwright Chromium |

Use grafana-visual-diff COMMAND --help for CLI subcommand options. The installation wrapper handles install-browser; it is not a comparison subcommand.

Compatibility Limits

• Uses Classic Grafana dashboard APIs. V2 resource dashboards, repeating panels/rows, ad hoc variables, and legacy embedded alerts are not supported.
• Expand collapsed rows in both input exports when their children need capturing. Complex nested scrolling and arbitrary plugin layouts require separate validation.
• Local JSON files are limited to 10 MB and 200 panel IDs, including rows. Screenshots and combined pixel-comparison canvases are limited to 32 megapixels. Higher device scale increases memory use.
• Historical data, variable-query options, browser fonts, and Grafana/plugin versions can change between captures. The tool does not rewrite production queries or freeze datasource results.
• Library-panel references are compared from the saved JSON; live referenced definitions are not historical snapshots.
• Existing-browser settings and service workers remain in effect. The tool adjusts its capture tabs, not the global browser profile.
• Generated dashboard-specific Playwright tests, machine-authenticated CI execution, and guaranteed support for every plugin are not implemented.

A separate VS Code companion extension uses this installed CLI rather than a second comparison engine. Set its Grafana Diff: Npm Package setting to your installed package name, or select the installed CLI script. The current extension requires a trusted local desktop workspace; remote and web workspaces are not supported. It does not install npm packages or browsers on activation.