Payload CMS Import/Export Plugin

A comprehensive Payload CMS plugin that enables seamless import and export of collection data with support for CSV and JSON formats, featuring advanced field mapping, duplicate handling, and batch processing capabilities.

✨ Features

• Multiple Format Support: Import/export data in CSV and JSON formats
• Smart Field Mapping: Automatic field mapping with manual override options
• Duplicate Handling: Choose to skip, replace, or update existing records
• Batch Processing: Handle large datasets efficiently with job queuing
• Field Selection: Choose specific fields to export or import
• Multi-language Support: Built-in translations for 30+ languages
• Preview Mode: Preview data before importing
• Error Handling: Comprehensive error reporting and validation
• Permission Integration: Respects Payload's access control system

📦 Installation

npm install payloadcms-import-export-plugin
# or
pnpm add payloadcms-import-export-plugin
# or
yarn add payloadcms-import-export-plugin

🚀 Quick Start

1. Add the Plugin to Your Payload Config

import { buildConfig } from 'payload'
import { importExportPlugin } from 'payloadcms-import-export-plugin'

export default buildConfig({
  // ... your existing config
  plugins: [
    importExportPlugin({
      // Enable for all collections
      collections: [], // Empty array means all collections
      
      // Or specify specific collections
      // collections: ['users', 'posts', 'categories']
    }),
  ],
})

2. Configure Collections (Optional)

You can disable import/export for specific fields by adding custom configuration:

import type { CollectionConfig } from 'payload'

export const Posts: CollectionConfig = {
  slug: 'posts',
  fields: [
    {
      name: 'title',
      type: 'text',
      required: true,
    },
    {
      name: 'content',
      type: 'textarea',
    },
    {
      name: 'internalId',
      type: 'text',
      custom: {
        'plugin-import-export': {
          disabled: true, // This field won't be available for import/export
        },
      },
    },
  ],
}

📖 Usage Guide

Exporting Data

1. Navigate to Collection: Go to any collection where the plugin is enabled
2. Click Export: Find the "Export" option in the collection list menu
3. Configure Export:
   • Select target collection
   • Choose export format (CSV or JSON)
   • Select specific fields (optional)
   • Configure sorting and filtering
4. Download: Click "Export" to generate and download your file

Export Configuration Options

{
  collection: 'posts',           // Target collection
  format: 'csv',                 // 'csv' or 'json'
  fields: ['title', 'content'],  // Specific fields (optional)
  where: { status: 'published' }, // Filter criteria (optional)
  sort: 'createdAt',             // Sort field (optional)
  limit: 1000,                   // Max records (optional)
}

Importing Data

1. Navigate to Collection: Go to any collection where the plugin is enabled
2. Click Import: Find the "Import" option in the collection list menu
3. Upload File: Select your CSV or JSON file
4. Configure Import:
   • Select target collection
   • Choose duplicate handling strategy
   • Map fields if needed
5. Preview: Review the data preview
6. Import: Execute the import process

Duplicate Handling Strategies

• Skip Existing: Leave existing records unchanged, only add new ones
• Replace Existing: Update existing records with new data from the file
• Create Duplicates: Always create new records (ignores duplicates)

Supported File Formats

CSV Format:

name,email,age,isActive
John Doe,[email protected],30,true
Jane Smith,[email protected],25,false

JSON Format (Array):

[
  {
    "name": "John Doe",
    "email": "[email protected]",
    "age": 30,
    "isActive": true
  },
  {
    "name": "Jane Smith",
    "email": "[email protected]",
    "age": 25,
    "isActive": false
  }
]

JSON Format (Single Object):

{
  "name": "John Doe",
  "email": "[email protected]",
  "age": 30,
  "isActive": true
}

⚙️ Configuration

Plugin Options

export interface ImportExportPluginConfig {
  /**
   * Collections to enable the plugin for
   * Empty array or undefined means all collections
   */
  collections?: string[]
  
  /**
   * Disable the plugin entirely
   * @default false
   */
  disabled?: boolean
  
  /**
   * Custom export functions for specific field types
   */
  customExportFunctions?: Record<string, ToCSVFunction>
  
  /**
   * Maximum file size for imports (in bytes)
   * @default 10485760 (10MB)
   */
  maxFileSize?: number
  
  /**
   * Enable debug logging
   * @default false
   */
  debug?: boolean
}

Field-Level Configuration

Control import/export behavior at the field level:

{
  name: 'sensitiveField',
  type: 'text',
  custom: {
    'plugin-import-export': {
      disabled: true, // Exclude from import/export
      toCSV: (value, field) => {
        // Custom export transformation
        return value ? '***' : ''
      }
    }
  }
}

Custom Export Functions

Define custom export transformations for complex field types:

importExportPlugin({
  customExportFunctions: {
    relationship: (value, field) => {
      // Transform relationship fields
      return Array.isArray(value) 
        ? value.map(v => typeof v === 'object' ? v.id : v).join(',')
        : typeof value === 'object' ? value.id : value
    },
    upload: (value, field) => {
      // Transform upload fields
      return typeof value === 'object' ? value.filename : value
    }
  }
})

🔧 Advanced Usage

Programmatic Export

import { exportCollection } from 'payloadcms-import-export-plugin/rsc'

// Export data programmatically
const exportData = await exportCollection({
  collection: 'posts',
  format: 'json',
  fields: ['title', 'content', 'status'],
  where: { status: 'published' },
  req, // Payload request object
})

Programmatic Import

import { importCollection } from 'payloadcms-import-export-plugin/rsc'

// Import data programmatically
const result = await importCollection({
  collection: 'posts',
  data: [
    { title: 'Post 1', content: 'Content 1' },
    { title: 'Post 2', content: 'Content 2' },
  ],
  duplicateHandling: 'skip',
  req, // Payload request object
})

console.log(result) // { imported: 2, updated: 0, skipped: 0, errors: [] }

Error Handling

The plugin provides comprehensive error reporting:

{
  success: false,
  imported: 5,
  updated: 2,
  skipped: 1,
  errors: [
    {
      row: 3,
      field: 'email',
      message: 'Email is required',
      value: null
    }
  ],
  processedAt: '2024-01-15T10:30:00Z'
}

🌐 Internationalization

The plugin includes built-in translations for 30+ languages. To use specific translations:

// Import specific language
import { translations } from 'payloadcms-import-export-plugin/translations/languages/en'

// Import all languages
import { allTranslations } from 'payloadcms-import-export-plugin/translations/languages/all'

Supported languages include: Arabic, Chinese, English, French, German, Japanese, Spanish, and many more.

🔒 Security Considerations

• File Upload Security: Only CSV and JSON files are accepted
• Permission Integration: Respects Payload's access control system
• Data Validation: All imported data goes through Payload's validation
• Size Limits: Configurable file size limits prevent abuse
• Sanitization: Input data is sanitized before processing

🧪 Testing

The plugin includes comprehensive test coverage:

# Run all tests
pnpm test

# Run integration tests
pnpm test:int

# Run end-to-end tests
pnpm test:e2e

🐛 Troubleshooting

Common Issues

Import fails with "Collection not found":

• Verify the collection slug is correct
• Ensure you have access permissions to the collection

CSV parsing errors:

• Check that your CSV has proper headers
• Ensure consistent column count across all rows
• Verify special characters are properly escaped

Memory issues with large files:

• Use smaller batch sizes
• Consider splitting large files into smaller chunks
• Increase Node.js memory limit: node --max-old-space-size=8192

Field mapping issues:

• Ensure field names in your import file match exactly
• Check for typos in field names
• Verify field types are compatible

Debug Mode

Enable debug logging for troubleshooting:

importExportPlugin({
  debug: true, // Enables detailed logging
})

📝 Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our GitHub repository.

Development Setup

# Clone the repository
git clone https://github.com/yourusername/payloadcms-import-export-plugin.git

# Install dependencies
pnpm install

# Start development server
pnpm dev

# Run tests
pnpm test

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Built for the amazing Payload CMS community
• Inspired by the need for better data migration tools
• Special thanks to all contributors and users

📞 Support

• 🐛 Bug Reports: GitHub Issues
• 💬 Discussions: GitHub Discussions
• 📖 Documentation: GitHub Wiki

Made with ❤️ for the Payload CMS community