🤖 Browser4


---

English | [简体中文](README.zh.md) | 中国镜像

<!-- TOC --> Table of Contents

• [🤖 Browser4](#-browser4)

- [🌟 Introduction](#-introduction) - [✨ Key Capabilities](#-key-capabilities) - [Quick Start](#quick-start) - [🧭 Tool Selection Guide](#-tool-selection-guide) - [How to Interact with a Page](#how-to-interact-with-a-page) - [How to Extract Data](#how-to-extract-data) - [How to Process at Scale](#how-to-process-at-scale) - [How to Turn HTML into Spreadsheets — Zero Tokens](#how-to-turn-html-into-spreadsheets--zero-tokens) - [📦 Installation](#-installation) - [💡 CLI Guide for Humans](#-cli-guide-for-humans) - [Quick start](#quick-start-1) - [Mental model](#mental-model) - [Global options](#global-options) - [Key concepts before the command list](#key-concepts-before-the-command-list) - [Complete command reference](#complete-command-reference) - [Timeout environment variables](#timeout-environment-variables) - [State persistence](#state-persistence) - [🚀 Build from Source](#-build-from-source) - [Architecture](#architecture) - [📦 Modules Overview](#-modules-overview) - [🧪 Test Fixture Server (MockSite)](#-test-fixture-server-mocksite) - [🤝 Support & Community](#-support--community) - [📜 Documentation](#-documentation) - [🔧 Proxy Configuration](#-proxy-configuration---unblock-website-access) - [License](#license) <!-- /TOC -->

🌟 Introduction

💖 Browser4 — an AI-native browser engine for autonomous agents, intelligent extraction, and large-scale web automation. 💖

✨ Key Capabilities

• 🤖 Agent Browser — Enable AI agents to browse, interact, and automate real-world websites.
• 🧠 ML-Powered Extraction — Learn page structures and extract structured data without LLM token costs.
• ⚡ High-Performance Runtime — Coroutine-safe architecture supporting 100k–200k complex page visits per machine per day.
• 🧬 Hybrid Intelligence — Combine LLM, ML, X-SQL, and selectors for robust extraction and experience reuse.
• 📦 Enterprise-Scale Automation — Swarm crawling, CDP-native control, batch jobs, stateful sessions, plugins, extensions, and more.

Quick Start

Paste the following instruction to your favorite AI agent like claude, codex, workbuddy or openclaw and run it:

Read https://browser4.io/SKILL.md and install browser4-cli (if not installed) for browser automation to perform the following task:

1. Open the browser in headed mode (`open --headed`) so the window is visible — this is a human-facing demo
2. go to amazon.com
3. search for pens to draw on whiteboards
4. compare the first 4 ones
5. write the result to a markdown file

🧭 Tool Selection Guide

Choosing the right tool for your task:

How to Interact with a Page

Use snapshot -i --boxes to see clickable/typeable elements with refs like e15, then click <ref>, fill <ref> "<text>", type/press, select, hover/drag/scroll, and wait to drive the page. All interaction commands accept CSS selectors too. Chain multiple steps efficiently with batch.

Typical interactive flow:

# Humans usually want to see the browser — open it headed
browser4-cli open --headed https://example.com/login
browser4-cli snapshot -i --boxes
browser4-cli fill e3 "user@example.com"
browser4-cli fill e4 "secret" --submit
browser4-cli wait --load networkidle
browser4-cli snapshot -i

How to Extract Data

Need to extract data from a page?
├─ Interactive page (click, fill, scroll first)? → snapshot + refs, then extract
├─ Static page, one field? → htmlsnapshot get text "<selector>"
├─ Static page, all matches of one field? → htmlsnapshot get all text "<selector>"
├─ Static page, multiple correlated fields (title+price+url per item)?
│  → htmlsnapshot query --sql @query.sql
├─ Live JS / complex DOM logic? → eval --json
├─ Natural language ("find the product price")? → extract (needs LLM key)
└─ High volume, many pages? → crawl or swarm with --sql

How to Process at Scale

Need to process multiple pages?
├─ Single list page (search results)? → htmlsnapshot query with DOM_LOAD_AND_SELECT
├─ List of known URLs (in a file)? → crawl --seed-file urls.txt --depth 0 --sql @query.sql
├─ Crawl from a start URL (follow links)? → crawl <url> --out-link-selector "..." --depth N
├─ Need parallel execution (high throughput)? → swarm create → swarm query --seed-file ...
├─ Repeated monitoring (check every hour)? → loop -- eval "..." -i 3600
└─ Just a few URLs in a shell script?
   → browser4-cli open --headed "https://first-url"   # humans: open once, visibly
   → for url in ...; do browser4-cli goto "$url"; ... done

How to Turn HTML into Spreadsheets — Zero Tokens

WebMiner runs ML clustering on downloaded HTML files to produce structured spreadsheets and interactive reports — no LLM tokens, everything runs locally.

Have HTML files and want structured data — without tokens?
├─ < 20 pages? → browser4-cli crawl --seed-file urls.txt --depth 0 --sql @query.sql
├─ < 1,000 pages (small to medium)? → WebMiner Free (SMILE ML engine)
│  java -jar scent-miner.jar all ./pages/
│  → Interactive HTML report + Excel spreadsheets — local, zero cost
├─ > 1,000 pages (production scale)? → WebMiner Commercial (Apache Spark ML)
│  Same encode → cluster → views pipeline, distributed across machines
└─ Need to acquire pages first?
   ├─ Single pages: browser4-cli htmlsnapshot export
   ├─ Bulk download: browser4-cli crawl --seed-file urls.txt --depth 0
   └─ High throughput: browser4-cli swarm create → swarm query --seed-file ...
       Then feed the HTML directory to WebMiner

> Pipeline: encode (HTML → feature vectors → CSV) → cluster (KMeans, auto-detected K) → views (HTML report + Excel). Free tier uses the SMILE ML library for single-machine clustering (< 1,000 pages). Requires JDK 17+. See web-miner for install instructions.

---

📦 Installation

Manually installation is optional since your AI agent is smart enough to install it after reading the SKILL.

Install browser4-cli globally using npm (requires Node.js):

npm install -g browser4-cli
browser4-cli install

Or bootstrap the native binary directly with a single command. The scripts also install the Browser4 backend (runtime bundle) afterwards — running browser4-cli install on a fresh machine, or browser4-cli upgrade when a backend is already present (pass --skip-backend to skip this step):

Windows (PowerShell):

irm https://browser4.oss-cn-beijing.aliyuncs.com/scripts/install-browser4-cli.ps1 | iex

Linux / macOS (bash):

curl -fsSL https://browser4.oss-cn-beijing.aliyuncs.com/scripts/install-browser4-cli.sh | bash

💡 CLI Guide for Humans

browser4-cli is a human-usable browser automation shell, not just an agent backend. You can drive a real browser, inspect state, extract structured data, run X-SQL, orchestrate crawl/swarm jobs, manage server plugins and skills, and hand long-running work to built-in AI features.

If you want the embedded agent-facing instructions, see [skills/browser4-cli/SKILL.md](skills/browser4-cli/SKILL.md). This section is the human reference.

Quick start

# Open a visible browser session (humans usually want to see the window;
# agents should prefer the default headless mode — see SKILL.md)
browser4-cli open --headed https://browser4.io

# Inspect the page and get element refs
browser4-cli snapshot --boxes

# Interact using a ref from the snapshot
browser4-cli click e15
browser4-cli fill e16 "Browser4" --submit

# Extract data from the live page
browser4-cli get text "h1"

# Capture a static DOM snapshot for repeated extraction
browser4-cli htmlsnapshot
browser4-cli htmlsnapshot get text "#main-content"
browser4-cli htmlsnapshot query --sql @query.sql

# Save output
browser4-cli screenshot --full-page --filename page.jpg
browser4-cli pdf --filename page.pdf

Mental model

1. Session-oriented: commands work against the current browser session; use -s <name> for isolated named sessions. 2. Two page views: snapshot is for interactive work with element refs like e15; htmlsnapshot is for DOM/X-SQL extraction with CSS selectors. 3. Interactive vs static extraction: use click, fill, type, press, wait when the page must be manipulated first; use htmlsnapshot query when you need structured extraction from the DOM. 4. Synchronous vs async jobs: agent, swarm, crawl, and async chat-style commands return task IDs you poll later.

Global options

These flags can appear before any command.

| Flag | Meaning | |---|---| | -h, --help [command\|category] | Show top-level help, category help, or detailed command help | | --help-json | Emit the machine-readable command reference | | -v, --version | Print the CLI version | | -s, --session <name> | Use a named session instead of the default session | | --server <url> | Override the Browser4 server URL | | --timeout <seconds> | Override the HTTP timeout for the current command | | --proxy <url> | Proxy used for runtime downloads/install operations | | --json | Emit machine-readable JSON only | | --pretty | Pretty-print JSON output | | -q, --quiet | Suppress normal human-readable output | | -tip, --show-tip | Show a relevant tip on stderr after commands |

Key concepts before the command list

#### Element refs vs CSS selectors

• snapshot returns accessibility-tree refs such as e5, e12, e42
• most interaction commands accept either a snapshot ref or a CSS selector
• htmlsnapshot commands use CSS selectors, not accessibility refs

#### snapshot vs htmlsnapshot

#### LLM configuration

AI-powered commands such as extract, summarize, chat, agent run, and X-SQL llm_* functions require an LLM provider key.

export DEEPSEEK_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxx

Complete command reference

#### Session lifecycle and server administration

| Command | Description | |---|---| | open [url] | Open a browser session or reconnect to an existing one. Headless by default. Supports --headed (visible window), --headless, --profile <path>, --profile-mode <DEFAULT\|SYSTEM_DEFAULT\|SEQUENTIAL\|TEMPORARY>, --interact-level <FASTEST\|FAST\|DEFAULT>. | | attach | Attach to an existing browser via CDP or the Browser4 extension. Supports --cdp <url\|port\|channel> and remote endpoint options. | | close | Close the active browser session. | | list | List browser sessions with their status and next-open behavior. Supports --all. | | session-default <name> | Make a named session become the default unnamed session. | | close-all | Close all sessions without stopping the backend. | | kill-all | Force-stop the backend and Browser4-managed browser processes. | | stop | Gracefully stop the Browser4 server. | | status | Show server version, port, and health. | | doctor | Run diagnostics: build info, LLM status, stale daemon cleanup, optional repair. Supports --verbose and --fix. | | doctor log [name] | List, view, tail, or grep backend log files. Supports --tail, grep-style flags, and doctor log <name> grep <pattern>. | | doctor metrics [filter] | List, filter, or grep backend metrics. Supports doctor metrics grep <pattern>. | | delete-data | Delete session data. | | install | Install the Browser4 runtime bundle. Supports --tag <version> and --force. | | upgrade | Upgrade the CLI/runtime bundle. Supports --tag <version> and --force. | | uninstall | Remove global installs and runtime data. Supports -y, --yes, and --dry-run. |

browser4-cli open --headed https://example.com
browser4-cli attach --cdp chrome
browser4-cli doctor --verbose
browser4-cli doctor log server.log --tail
browser4-cli doctor metrics grep request

#### Navigation

#### Core interaction

All interaction commands accept a snapshot ref such as e15 or a CSS selector unless noted otherwise. Most of them also support --no-snapshot to skip the automatic post-action accessibility snapshot.

wait --load accepts domcontentloaded, load, and networkidle.

browser4-cli click e8 --follow
browser4-cli fill e4 "john@example.com" --submit
browser4-cli type "Browser4" e7 --verify
browser4-cli wait --text "Success"
browser4-cli wait --load networkidle

#### Keyboard and mouse

#### Page inspection and live extraction

get modes:

browser4-cli snapshot -i --boxes
browser4-cli snapshot grep -C 2 "button"
browser4-cli eval "document.title"
browser4-cli eval --file script.js --await
browser4-cli console warn
browser4-cli cdp Runtime.evaluate --json '{"expression":"document.title"}'

#### HTML snapshot and X-SQL extraction

htmlsnapshot captures a stored raw DOM snapshot and is the center of Browser4's structured extraction workflow.

| Command | Description | |---|---| | htmlsnapshot | Short form of htmlsnapshot capture. | | htmlsnapshot capture | Capture and store a static HTML snapshot with metadata about the page and interactive elements. | | htmlsnapshot get <field> [selector] [name] | Extract the first matching text, html, or attr from the stored snapshot. | | htmlsnapshot get all <field> [selector] [name] | Extract all matching values from the stored snapshot. Supports --offset and --limit. | | htmlsnapshot query [url] | Run X-SQL. Supports --sql <query\|@file>, --sql-stdin, --sql-base64, result pagination, and extraction-focused output flags. | | htmlsnapshot export | Export stored HTML to a file. Supports positional file path or --file <path> plus --clean. | | htmlsnapshot summary | Generate a compressed Web Page Summary Index (WPSI). | | htmlsnapshot grep <pattern> | Search stored HTML with grep-style flags. | | htmlsnapshot inspect [selector] | Discover recurring DOM patterns and selector candidates. Supports --max, --depth, --stdin, --selector-base64. |

Important rules:

• use snapshot when you need refs and interaction
• use htmlsnapshot when you need repeated DOM extraction
• htmlsnapshot query --sql @query.sql is the recommended way to avoid shell quoting issues
• for correlated list extraction, prefer htmlsnapshot query over repeated get all

browser4-cli htmlsnapshot
browser4-cli htmlsnapshot get text "#productTitle"
browser4-cli htmlsnapshot get all text ".result-title" --offset 10 --limit 5
browser4-cli htmlsnapshot inspect ".s-result-item" --depth 6 --max 20
browser4-cli htmlsnapshot export --file page.html --clean
browser4-cli htmlsnapshot query --sql @query.sql

For deep X-SQL usage, see [skills/browser4-cli/references/htmlsnapshot.md](skills/browser4-cli/references/htmlsnapshot.md) and [skills/browser4-cli/references/x-sql-dom-load-select.md](skills/browser4-cli/references/x-sql-dom-load-select.md).

#### Screenshots and PDF

#### Tabs

| Command | Description | |---|---| | tab-list | List open tabs with indexes, titles, and URLs; use --json for full GUIDs. | | `tab-n

…