wp-site-replicator

CLI tool to scrape a WordPress site and replicate it to another WordPress instance — including content, images, menus, SEO metadata, theme, and settings.

Features

• Scrapes pages and posts via sitemap discovery
• Downloads and re-uploads all media assets (images, SVGs, etc.)
• Converts HTML content to Markdown then publishes via WP REST API
• Installs and activates any WordPress.org theme on the target site
• Installs and activates Yoast SEO plugin
• Populates Yoast SEO meta fields (title, description, focus keyword)
• Configures site title, tagline, permalink structure, and RSS settings
• Recreates navigation menus with correct link mapping
• Interactive CLI prompts — no config files needed

Requirements

• Node.js >= 16
• Target WordPress site with:
  • REST API enabled
  • XML-RPC enabled
  • Application Passwords enabled
  • A user account with Administrator role

Installation

Global install

npm install -g wp-site-replicator
wp-site-replicator

Run with npx (no install)

npx wp-site-replicator

From source

git clone <repo-url>
cd cli-scrapper
npm install
node index.js

Usage

Run the command and answer the interactive prompts:

$ wp-site-replicator

  Source domain to scrape: https://example.com
  WordPress site URL to publish to: https://target.com
  WordPress username: admin
  WordPress application password: ****
  WordPress theme slug to install: flavor-developer

The tool then executes a 7-step pipeline:

| Step | Action | |------|--------| | 1 | Scrape — Fetches sitemap, scrapes all pages/posts, downloads assets | | 2 | Install Theme — Installs & activates the specified theme from WordPress.org | | 3 | Install Yoast SEO — Installs & activates the Yoast SEO plugin | | 4 | Configure Settings — Sets site title, tagline, permalinks, RSS settings | | 5 | Clean WordPress — Deletes default posts, pages, and sample content | | 6 | Publish Content — Uploads all scraped pages/posts with media and SEO metadata | | 7 | Create Menus — Recreates navigation menus with mapped internal links |

How It Works

• Scraping: Discovers URLs via page-sitemap.xml and post-sitemap.xml. Extracts content using Cheerio, converts to Markdown with Turndown, and saves locally with front matter (title, excerpt, Yoast SEO fields).
• Theme & Plugin Install: Uses the Code Snippets plugin as a helper to execute PHP on the target site (for operations the REST API doesn't support). The helper plugin is automatically installed, used, then removed.
• Publishing: Uploads media via the WP REST API, creates pages/posts with correct featured images, and writes Yoast SEO custom fields via XML-RPC.
• Menus: Filters scraped menu items to internal links only, creates a "Primary Menu", and assigns it to the first registered theme menu location.

Output

Scraped content is saved to an output/ directory:

output/
├── pages/
│   └── about.md
├── posts/
│   └── hello-world.md
├── assets/
│   └── <hashed-filenames>
└── menus.json

License

MIT