autoBrowser

autoBrowser is a CLI for controlling Chrome through Native Messaging.

Install

npm i autobrowser-cli -g

After installation, export the bundled Chrome extension:

ab extension --path ./autobrowser-extension

Then load ./autobrowser-extension in Chrome through "Load unpacked".

Start the local service:

ab serve

After you manually load the extension in Chrome, install the Native Messaging host manifest with the extension ID:

ab install-host <chrome-extension-id>

Skills

Install the skill with:

npx skills add https://github.com/Mingbing-get/autoBrowser --skill auto-browser-use

auto-browser-use

Located at skills/auto-browser-use/SKILL.md.

Use this skill when a task needs to control Chrome through autoBrowser, especially for:

• opening a website in Chrome
• switching or inspecting tabs
• querying page content
• interacting with elements through browser automation

It treats autoBrowser as the default browser-control path in this repository and guides users through common command flows such as open, query, input, hover, drag, and click.

Selector examples below use descriptive placeholders such as "<search-input-selector>", not literal DOM ids. Replace them with selectors from the current page before running commands.

Then use the CLI from another shell:

ab open "https://www.baidu.com"
ab tabs
ab query "<target-selector>"
ab search "搜索"
ab search-from-point 120 84
ab summary
ab text "<content-selector>"
ab rect "<content-selector>"
ab click "<click-target-selector>"
ab click "<click-target-selector>" --maxObserveMs 1500 --stableWindowMs 300
ab hover "<menu-trigger-selector>" --maxObserveMs 1200 --stableWindowMs 240
ab drag "<drag-source-selector>" --target "<drop-target-selector>" --direction r
ab drag "<drag-source-selector>" --x 640 --y 320
ab scroll --y 400
ab input "<search-input-selector>" --value "hello world"
ab upload "<file-input-selector>" "/Users/name/Downloads/report.pdf"
ab close
ab status

input clicks the target element first, then types into it through native keyboard automation. On macOS, non-ASCII text such as Chinese is entered through a temporary clipboard paste fallback, and the command result includes the detected input source when available.

Chrome setup details are in docs/setup-native-messaging.md.

When loading the extension in Chrome, choose packages/extension/dist as the unpacked extension directory.

CLI Commands

All commands are available through ab <command> ....

help and version

ab -h
ab --help
ab -v
ab --version

open

Open a URL in Chrome.

ab open "https://www.baidu.com"

close

Close the active tab, or a specific tab with --tabId.

ab close
ab close --tabId 123

tabs

List the current browser tabs.

ab tabs

query

Inspect a DOM node and return the matched node plus surrounding context.

ab query "<target-selector>"
ab query "<target-selector>" --tabId 123

search

Search visible page content by text.

ab search "搜索"
ab search "搜索" --tabId 123

search-from-point

Search DOM layers from a browser coordinate.

ab search-from-point 120 84
ab search-from-point 120 84 --tabId 123

summary

Return a page summary for the active tab or a specific tab.

ab summary
ab summary --tabId 123

text

Extract text from the matched element.

ab text "<content-selector>"
ab text "<content-selector>" --tabId 123

rect

Return layout and viewport metrics for the matched element.

ab rect "<content-selector>"
ab rect "<content-selector>" --tabId 123

click

Click the matched element and return a post-click observation summary. The service will calibrate browser-to-screen coordinates when needed.

ab click "<click-target-selector>"
ab click "<click-target-selector>" --tabId 123
ab click "<click-target-selector>" --maxObserveMs 1500 --stableWindowMs 300

Optional flags:

• --tabId <number>
• --minObserveMs <number>
• --maxObserveMs <number>
• --stableWindowMs <number>
• --maxRegions <number>
• --maxItemsPerRegion <number>
• --maxTextLength <number>

hover

Move the pointer to the matched element with human-like motion, scroll it into view when needed, and return a post-hover observation summary.

ab hover "<hover-target-selector>"
ab hover "<hover-target-selector>" --tabId 123
ab hover "<hover-target-selector>" --maxObserveMs 1500 --stableWindowMs 300

Optional flags:

• --tabId <number>
• --minObserveMs <number>
• --maxObserveMs <number>
• --stableWindowMs <number>
• --maxRegions <number>
• --maxItemsPerRegion <number>
• --maxTextLength <number>

drag

Drag the matched element either toward an anchor on another element or to a specific viewport coordinate. The command returns a post-drag observation summary.

Anchor mode:

ab drag "<drag-source-selector>" --target "<drop-target-selector>" --direction r
ab drag "<drag-source-selector>" --target "<drop-target-selector>" --direction br --tabId 123

Viewport coordinate mode:

ab drag "<drag-source-selector>" --x 640 --y 320
ab drag "<drag-source-selector>" --x 640 --y 320 --tabId 123

Optional flags:

• --tabId <number>

Required options:

• Use either --target <selector> --direction <anchor> or --x <integer> --y <integer>
• Supported anchor values are t, tr, r, br, b, bl, l, and tl

scroll

Scroll the active tab or a specific tab. At least one of --x or --y is required.

ab scroll --y 400
ab scroll --x 120 --y -240 --tabId 123

input

Click the target element, then type text into it.

ab input "<search-input-selector>" --value "hello world"
ab input "<search-input-selector>" --value "hello world" --tabId 123

upload

Click the target element, wait for the native file chooser, then upload a file path through the platform-specific native dialog flow.

ab upload "<file-input-selector>" "/Users/name/Downloads/report.pdf"
ab upload "<file-input-selector>" "/Users/name/Downloads/report.pdf" --tabId 123

Behavior notes:

• The service waits 2 seconds after clicking before driving the file chooser.
• On macOS it uses Command+Shift+G, pastes the absolute path, then presses Enter twice with 1 second gaps.
• On Windows it blind-pastes the file path into the chooser and presses Enter twice with 1 second gaps.

flow

Execute multiple commands in sequence from a single JSON array argument.

ab flow '[{"action":"open","url":"https://www.baidu.com"},{"action":"input","selector":"<search-input-selector>","value":"自动化测试"},{"action":"hover","selector":"<suggestion-panel-selector>"},{"action":"click","selector":"<submit-button-selector>"}]'

flow currently supports aggregating these actions:

• upload
• input
• scroll
• drag
• click
• hover
• rect
• text
• summary
• search-from-point
• search
• query
• tabs
• open
• close

Example with multiple action types:

ab flow '[
  {"action":"open","url":"https://www.baidu.com"},
  {"action":"tabs"},
  {"action":"query","selector":"<search-input-selector>"},
  {"action":"drag","selector":"<drag-source-selector>","x":320,"y":180},
  {"action":"search","text":"百度"},
  {"action":"search-from-point","x":120,"y":84},
  {"action":"summary"},
  {"action":"text","selector":"body"},
  {"action":"rect","selector":"<search-input-selector>"},
  {"action":"hover","selector":"<menu-trigger-selector>","observe":{"stableWindowMs":240}},
  {"action":"click","selector":"<click-target-selector>","observe":{"maxObserveMs":1500}},
  {"action":"drag","selector":"<drag-source-selector>","targetSelector":"<drop-target-selector>","direction":"l"},
  {"action":"scroll","deltaX":0,"deltaY":400},
  {"action":"input","selector":"<search-input-selector>","value":"自动化测试"},
  {"action":"upload","selector":"<file-input-selector>","filepath":"/Users/name/Downloads/report.pdf"},
  {"action":"close"}
]'

serve

Start the local HTTP service and Native Messaging bridge.

ab serve

extension

Copy the bundled Chrome extension files to a local directory that Chrome can load as an unpacked extension.

ab extension --path ./autobrowser-extension
ab extension --path=/tmp/autobrowser-extension

After copying:

1. Open chrome://extensions
2. Enable Developer mode
3. Click "Load unpacked"
4. Select the exported directory
5. Copy the extension ID and run ab install-host <chrome-extension-id>

install-host

Install the Native Messaging host manifest for a Chrome extension ID.

ab install-host <chrome-extension-id>

On macOS, this writes the manifest to Chrome's NativeMessagingHosts directory and creates an executable shell launcher under ~/Library/Application Support/autoBrowser. On Windows, it writes the manifest under %APPDATA%\\autoBrowser, stages a local native-host runtime there, creates a native-host.cmd launcher, and registers the manifest with Chrome through HKCU\\Software\\Google\\Chrome\\NativeMessagingHosts\\com.autobrowser.host.

status

Return the local service status as JSON.

ab status