Places Data Script

A powerful CLI tool and programmatic API for scraping business place data from the ValueSERP API with support for pagination, detailed information fetching, and export to JSON and CSV formats.

Features

• 🔧 Interactive Setup - Easy configuration wizard for API keys
• 🔄 Smart Pagination - Automatically handles pagination until all results are fetched
• 📊 Dual Output - Exports data to both JSON and CSV formats
• 🔁 Retry Logic - Automatic retry with exponential backoff for failed requests
• 🎯 Deduplication - Removes duplicate entries by data_cid
• ⚡ Rate Limiting - Configurable delays to respect API limits
• 📝 Progress Tracking - Real-time progress updates during scraping
• 🔌 Programmatic API - Use as a module in your own applications

Installation

Using pnpm (recommended)

pnpm install

Using npm

npm install

Global Installation

pnpm install -g .
# or
npm install -g .

CLI Usage

Setup

Configure your API key and preferences:

places setup

Search

Search for places and export data:

places search --query "pizza" --location "New Jersey,United States"

Options

• -q, --query <term> - Search query (required)
• -l, --location <location> - Location as "City,State" (required)
• -o, --output <name> - Custom output filename prefix
• -k, --api-key <key> - Override saved API key
• -d, --delay <ms> - Delay between API calls (default: 500ms)

Examples

# Basic search
places search -q "pizza" -l "New Jersey,United States"

# Custom output name
places search -q "sushi" -l "Los Angeles,California" -o "sushi-restaurants"

# Slower rate limiting
places search -q "pizza" -l "New York,New York" -d 1000

# One-time API key override
places search -q "pizza" -l "Chicago,Illinois" -k "TEMP_API_KEY"

Programmatic API

Use Places Data Script as a module in your own applications:

import PlacesDataScript from 'places-data-script';

// Create instance with options
const scraper = new PlacesDataScript({
  apiKey: 'YOUR_API_KEY',
  delay: 500,
  verbose: true
});

// Search and export
const results = await scraper.search({
  query: 'pizza',
  location: 'New Jersey,United States',
  output: 'my-results', // optional
  export: true // optional, default true
});

console.log(`Found ${results.count} places`);
console.log(`JSON: ${results.export.json}`);
console.log(`CSV: ${results.export.csv}`);

API Methods

search(params)

Search for places and optionally export results.

const results = await scraper.search({
  query: 'pizza',
  location: 'New Jersey,United States',
  output: 'custom-name', // optional
  export: true // optional, default true
});

fetchPlaceIds(params)

Fetch only the list of place IDs without details.

const ids = await scraper.fetchPlaceIds({
  query: 'pizza',
  location: 'New Jersey,United States'
});

fetchDetails(dataCids)

Fetch details for specific place IDs.

const details = await scraper.fetchDetails(['id1', 'id2', 'id3']);

export(data, query, location, output)

Export data to JSON and CSV files.

const result = await scraper.export(
  placeData,
  'pizza',
  'New Jersey,United States',
  'my-export'
);

Individual Function Exports

You can also import individual functions:

import {
  fetchAllPlaces,
  fetchAllPlaceDetails,
  exportData,
  saveToJSON,
  saveToCSV,
  loadConfig,
  saveConfig
} from 'places-data-script';

Output Format

JSON

[
  {
    "data_cid": "1866378315924306344",
    "name": "Joe's Pizza",
    "phone": "+1 555-0123",
    "website": "https://joespizza.com",
    "address": "123 Main St, Newark, NJ 07102",
    "latitude": 40.7357,
    "longitude": -74.1724,
    "hours": "Mon-Sun: 11:00 AM - 10:00 PM",
    "review_count": 245,
    "rating": 4.5
  }
]

CSV

data_cid,name,phone,website,address,latitude,longitude,hours,review_count,rating
1866378315924306344,Joe's Pizza,+1 555-0123,https://joespizza.com,"123 Main St, Newark, NJ 07102",40.7357,-74.1724,"Mon-Sun: 11:00 AM - 10:00 PM",245,4.5

Configuration

Configuration is stored in ~/.config/places/settings.json:

{
  "apiKey": "YOUR_VALUESERP_API_KEY",
  "defaultDelay": 500
}

Configuration Priority

1. Command-line --api-key flag (highest)
2. Constructor options (programmatic API)
3. Saved config in ~/.config/places/settings.json
4. Default embedded API key (lowest)

Development

Install Dependencies

pnpm install

Run Tests

The project uses Vitest for testing with comprehensive unit tests for all modules.

# Run all tests
pnpm test

# Run tests in watch mode
pnpm test:watch

# Run tests with coverage report
pnpm test:coverage

Test Coverage:

• ✅ Config module (17 tests) - Configuration management and fallbacks
• ✅ Export module (13 tests) - JSON/CSV export and deduplication
• ✅ API module (11 tests) - API calls, pagination, retry logic, rate limiting
• ✅ Programmatic API (20 tests) - PlacesDataScript class and all methods

Total: 61 tests passing

Lint Code

pnpm lint

Format Code

pnpm format

Link for Local Development

pnpm link

Documentation

• Architecture - Technical architecture and design decisions
• API Documentation - Detailed API documentation
• Workflow Diagram - Visual representation of the data scraping process

Project Structure

places/
├── bin/
│   └── places.js          # CLI entry point
├── lib/
│   ├── index.js           # Programmatic API
│   ├── config.js          # Configuration management
│   ├── api.js             # API client functions
│   └── export.js          # JSON/CSV export functions
├── docs/
│   ├── README.md          # API documentation
│   ├── architecture.md    # Architecture documentation
│   └── workflow.puml      # PlantUML workflow diagram
├── test/
│   └── *.test.js          # Test files
├── package.json
└── README.md              # This file

License

MIT

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.