Spacecat Shared - DRS Client

A JavaScript client for the Data Retrieval Service (DRS) API, part of the SpaceCat Shared library. It supports job submission (prompt generation, web scraping), scrape result lookups, and brand detection triggers.

Installation

Install the package using npm:

npm install @adobe/spacecat-shared-drs-client

Configuration

Set the following environment variables:

• DRS_API_URL — Base URL of the DRS API
• DRS_API_KEY — API key for authentication

Usage

Creating an instance from Helix UniversalContext

import DrsClient from '@adobe/spacecat-shared-drs-client';

const client = DrsClient.createFrom(context);

Constructor

import DrsClient from '@adobe/spacecat-shared-drs-client';

const client = new DrsClient({
  apiBaseUrl: '<DRS_API_URL>',
  apiKey: '<DRS_API_KEY>',
}, log);

Methods

submitScrapeJob(params)

Submits a web scraping job via the Bright Data provider.

import { SCRAPE_DATASET_IDS } from '@adobe/spacecat-shared-drs-client';

const result = await client.submitScrapeJob({
  datasetId: SCRAPE_DATASET_IDS.YOUTUBE_VIDEOS,
  siteId: 'site-uuid',
  urls: ['https://www.youtube.com/watch?v=abc123'],
  priority: 'HIGH', // optional, defaults to 'HIGH'. Also accepts 'LOW'
});
// Returns: { job_id: '...', ... }

Valid datasetId values (available via SCRAPE_DATASET_IDS):

• youtube_videos
• youtube_comments
• reddit_posts
• reddit_comments
• wikipedia

lookupScrapeResults(params)

Looks up scraping results for an array of URLs.

const lookup = await client.lookupScrapeResults({
  datasetId: SCRAPE_DATASET_IDS.REDDIT_POSTS,
  siteId: 'site-uuid',
  urls: ['https://www.reddit.com/r/technology/comments/abc123/post_title/'],
});
// Returns:
// {
//   results: [
//     { url: '...', status: 'available', presigned_url: '...', scraped_at: '...', expires_in: 3600 },
//     { url: '...', status: 'scraping', job_id: '...', message: '...' },
//     { url: '...', status: 'not_found', message: '...' },
//   ],
//   summary: { total: 3, available: 1, scraping: 1, not_found: 1 }
// }

submitPromptGenerationJob(params)

Submits a prompt generation job.

const result = await client.submitPromptGenerationJob({
  baseUrl: 'https://example.com',
  brandName: 'Example',
  audience: 'consumers',
  siteId: 'site-uuid',
  imsOrgId: 'org-uuid',
  region: 'US',       // optional, defaults to 'US'
  numPrompts: 50,     // optional, defaults to 50
  source: 'onboarding', // optional, defaults to 'onboarding'
});

triggerBrandDetection(siteId, options?)

Triggers brand detection re-analysis for a site.

await client.triggerBrandDetection('site-uuid', { batchId: 'batch-abc', priority: 'HIGH' });

getJob(jobId)

Retrieves job status and details.

const job = await client.getJob('job-uuid');

submitJob(params)

Submits a generic job to DRS. Used internally by the higher-level methods, but available for custom job types.

const result = await client.submitJob({
  provider_id: 'custom-provider',
  parameters: { /* ... */ },
});

Testing

To run tests:

npm test

Linting

Lint your code:

npm run lint

Cleaning

To remove node_modules and package-lock.json:

npm run clean

Additional Information

• Repository: GitHub
• Issue Tracking: GitHub Issues
• License: Apache-2.0