repo-sync

Synchronize specific files from remote Git repositories into your local project using a declarative manifest.

Purpose

This tool acts as a bridge between separate repositories, allowing files to exist in your project at runtime without manually copying or maintaining forks. It also serves to keep specified files in sync with their upstream sources.

Common use cases:

• Pull shared configurations, CI workflows, or utility scripts from a separate repository into multiple projects
• Keep documentation snippets or data files synchronized with their canonical source
• Compose a project from files scattered across different repositories without vendoring them

[!note] An example with real use is in this repository.

There are two separate branches: one for the application's source code and the other for blog posts. When the project is compiled, the blog posts are included during the build process. This keeps the actual code separate from the data variables and databases.

Installation

npm install -g @rodny/repo-sync

Quick Start

Create a syncfile.json:

{
  "repo": "https://github.com/owner/repo.git",
  "branch": "main",
  "include": ["docs/*.md"],
  "target": "./synced-docs"
}

Run:

repo-sync

Files matching docs/*.md from the remote repository will be copied into ./synced-docs.

[!tip] You can also use JavaScript/TypeScript configuration files for dynamic config generation. See Configuration File Formats for details.

Configuration

The syncfile accepts a single source object or an array of sources. Each source requires:

| Field | Required | Description | | ----------- | -------- | ------------------------------------------------------------------------------- | | repo | Yes | Git repository clone URL | | branch | No | Branch to track (defaults to the remote default branch) | | include | No | Glob patterns for files to sync (defaults to **/*) | | exclude | No | Glob patterns to exclude | | target | No | Target directory within the project (defaults to current directory) | | mappings | No | Array of fine-grained mapping rules (takes precedence over include/exclude) | | gitignore | No | Whether to add synced files to .gitignore (defaults to false) |

Mapping Rules

Each entry in mappings supports:

| Field | Required | Description | | --------- | -------- | ------------------------------------------------------------------------------------------ | | include | Yes | Glob pattern(s) for files to match | | exclude | No | Glob pattern(s) to exclude from matches | | dest | Yes | Destination path; ends with / for a directory, otherwise treated as a single file rename |

Configuration File Formats

The syncfile supports multiple file formats. By default, repo-sync will look for a configuration file in the following order:

1. syncfile.json
2. syncfile.js
3. syncfile.mjs
4. syncfile.cjs

You can also specify a custom path using the -f / --file CLI option.

JSON (syncfile.json)

Standard static configuration. Declare your sources directly as JSON:

{
  "repo": "https://github.com/owner/repo.git",
  "branch": "main",
  "include": ["README.md"],
  "target": "./synced"
}

CommonJS (syncfile.js / syncfile.cjs)

Use module.exports to export your configuration. This allows dynamic logic:

const version = process.env.SYNC_VERSION || 'main';

module.exports = [
  {
    repo: 'https://github.com/owner/repo.git',
    branch: version,
    include: ['config/*.json'],
    target: './synced-config',
  },
];

If using TypeScript compiled to CJS with export default, the default export is automatically unwrapped for compatibility.

ES Modules (syncfile.mjs)

Use export default to export your configuration:

export default [
  {
    repo: 'https://github.com/owner/repo.git',
    branch: 'main',
    include: ['src/**/*.ts'],
    target: './synced-src',
  },
];

CLI Options

Usage: repo-sync [options]

Options:
  -f, --file <path>       path to a syncfile (JSON, JS, MJS, CJS)
                           auto-discovers syncfile.json, .js, .mjs, .cjs when omitted
  -c, --cache-root <dir>  base directory for repository caches (default: ".cache/repo-sync")
  -t, --target <dir>      root target directory (default: current working directory)
  --gitignore             create .gitignore entries for synced files

Examples

Sync a single file from one repository:

{
  "repo": "https://github.com/owner/repo.git",
  "branch": "main",
  "include": ["LICENSE"],
  "target": "./legal"
}

Sync multiple repositories with explicit mappings:

[
  {
    "name": "astro-blog",
    "repo": "https://github.com/author/astro-blog.git",
    "branch": "main",
    "target": "./dist/content",
    "mappings": [
      {
        "include": ["src/data/post/*.md"],
        "dest": "posts/"
      },
      {
        "include": ["src/data/post/featured-post.mdx"],
        "dest": "featured.mdx"
      }
    ]
  },
  {
    "name": "ci-workflows",
    "repo": "https://github.com/org/shared-workflows.git",
    "branch": "main",
    "target": "./dist",
    "mappings": [
      {
        "include": [".github/workflows/deploy.yml"],
        "dest": ".github/workflows/shared-deploy.yml"
      }
    ]
  }
]

Dynamic configuration with environment variables (syncfile.js):

const env = process.env.DEPLOY_ENV || 'staging';

module.exports = [
  {
    name: 'shared-config',
    repo: 'https://github.com/org/config.git',
    branch: env === 'production' ? 'main' : 'develop',
    include: ['**/*.json'],
    target: './config',
  },
];

How It Works

1. Discovers and reads the syncfile manifest (JSON, JS, MJS, or CJS)
2. Clones each repository (shallow clone) into a local cache directory, or updates it if already cached
3. Copies matched files to the target directories
4. Optionally appends entries to .gitignore to prevent accidental commits of synced files

Repositories are cached in .cache/repo-sync by default and only fetched on subsequent runs.