Instagram Media Service

Multi-tenant API service for working with Instagram via Graph API, entirely on Redis. The service collects media (photos) from Instagram accounts of several projects and returns data instantly via API.

Features

• ⚡ Ultra-fast read: all data is stored in Redis
• 🏢 Multi-tenant: each project is isolated via namespace
• 🔄 Background sync: media is synchronized in the background via QStash queue
• ☁️ CDN integration: Images are uploaded to Cloudinary for fast delivery
• 🔒 Security: Project API token for access, Instagram token is encrypted at rest
• 📸 Instagram OAuth: Support for Instagram Login flow
• 🔑 Token refresh: Automatic token refresh before sync

Prerequisites

• Node.js 18+
• npm or yarn
• Upstash Redis account
• Upstash QStash account (for background sync)
• Cloudinary account (for image CDN)
• Instagram App (for OAuth)

Installation

1. Clone the repository:

git clone https://github.com/Toprent-app/instagram-media-service.git
cd instagram-media-service

2. Install dependencies:

npm install

3. Set up environment variables:

cp .env.example .env
# Edit .env with your credentials

Environment Variables

| Variable | Required | Description | |----------|----------|-------------| | UPSTASH_REDIS_REST_URL | Yes | Upstash Redis REST URL | | UPSTASH_REDIS_REST_TOKEN | Yes | Upstash Redis REST token | | QSTASH_TOKEN | Yes | Upstash QStash token | | QSTASH_CURRENT_SIGNING_KEY | Yes | QStash current signing key | | QSTASH_NEXT_SIGNING_KEY | Yes | QStash next signing key | | WEBHOOK_BASE_URL | Yes | Public URL for QStash webhooks (HTTPS) | | REDIRECT_URI | Yes | Instagram OAuth redirect URI | | ENCRYPTION_KEY | Yes | AES-256 key (32 bytes base64). Generate: openssl rand -base64 32 | | SYNC_INTERNAL_SECRET | Yes | Secret for protecting /sync endpoint | | PORT | No | Server port (default: 3000) | | INSTAGRAM_PHOTOS_TTL_SECONDS | No | TTL for cached photos (default: 604800 = 7 days) | | INSTAGRAM_API_TIMEOUT_MS | No | Timeout for Instagram API requests (default: 10000) | | REDIS_KEY_PREFIX | No | Key prefix for Redis when sharing with other apps | | CLOUDINARY_CLOUD_NAME | Yes | Cloudinary cloud name | | CLOUDINARY_API_KEY | Yes | Cloudinary API key | | CLOUDINARY_API_SECRET | Yes | Cloudinary API secret |

Available Scripts

# Development mode (with hot reload)
npm run dev

# Build for production
npm run build

# Production mode
npm run start

# Run linting
npm run lint

# Run tests with coverage
npm run test

# Type check without emitting
npm run typecheck

# Release management (using changesets)
npm run release:prepare    # Add a changeset
npm run release:update     # Version packages
npm run release:publish    # Publish to npm

Architecture

┌─────────────────────────────────────────────────────────┐
│                      REST API                           │
├─────────────────────────────────────────────────────────┤
│  GET /health          - Health check                    │
│  GET /instagram-callback - OAuth callback               │
│  POST /auth/obtain-token - Exchange code for token      │
│  POST /auth/refresh-token - Refresh access token        │
│  POST /register       - Register new project            │
│  GET /photos          - Get cached photos               │
│  POST /sync           - Trigger sync for all tenants    │
│  POST /sync/process/:tenantId - QStash webhook          │
│                                                         │
│  External Services:                                     │
│  - Instagram Graph API (source)                         │
│  - Cloudinary (image CDN)                               │
└─────────────────────────────────────────────────────────┘
                           │
                           v
┌─────────────────────────────────────────────────────────┐
│                Tenant Resolver                          │
│         (find tenantId by project_token)                │
└─────────────────────────────────────────────────────────┘
                           │
                           v
┌──────────────────────────────┐     ┌──────────────────────────┐
│      Redis Storage           │ <-> │    Background Worker     │
│──────────────────────────────│     │──────────────────────────│
│ tenant:{id}                  │     │ Upstash QStash Queue     │
│ auth_token:{token}           │     │ Sync Instagram → Redis   │
│ instagram:{id}:photos        │     │ Auto token refresh       │
└──────────────────────────────┘     └──────────────────────────┘
                           ^
                           │
                           v
┌─────────────────────────────────────────────────────────┐
│              Instagram Graph API                        │
└─────────────────────────────────────────────────────────┘

Redis Namespaces

| Key Pattern | Description | |-------------|-------------| | tenant:{tenantId} | Hash with project info (name, authToken, encrypted tokens) | | auth_token:{token} | String mapping auth token to tenantId | | instagram:{tenantId}:photos | JSON array of cached photos |

API Reference

Health Check

GET /health

Returns service health status.

Response (200):

{
  "success": true,
  "data": {
    "message": "ok"
  }
}

Instagram OAuth Callback

GET /instagram-callback?code={code}

Callback endpoint for Instagram OAuth. Displays the authorization code.

Response (200):

{
  "success": true,
  "data": {
    "code": "AQB..."
  }
}

Obtain Access Token

POST /auth/obtain-token

Exchanges Instagram authorization code for a long-lived access token.

Request:

{
  "embedUrl": "https://...",
  "code": "AQB...",
  "instagramAppId": "your_app_id",
  "instagramAppSecret": "your_app_secret"
}

Response (200):

{
  "success": true,
  "data": {
    "success": true,
    "longLivedToken": "IGQ...",
    "testData": "{...}"
  }
}

Errors:

• 400 - Validation failed
• 500 - Token exchange failed

Refresh Access Token

POST /auth/refresh-token

Refreshes the Instagram access token for the authenticated tenant.

Headers:

Authorization: Bearer <authToken>

Response (200):

{
  "success": true,
  "data": {
    "result": "success"
  }
}

Errors:

• 401 - Unauthorized (missing or invalid token)
• 404 - Tenant not found
• 500 - Internal server error

Register Project

POST /register

Registers a new project/tenant. If instagramAccessToken is provided, photos are synced immediately.

Request:

{
  "name": "my-project",
  "instagramAccountId": "178...",
  "instagramAccountSecret": "secret...",
  "instagramAccessToken": "IGQ...",
  "apiType": "INSTAGRAM_LOGIN"
}

Fields: | Field | Type | Required | Description | |-------|------|----------|-------------| | name | string | Yes | Project name (min 3 chars) | | instagramAccountId | string | Yes | Instagram account ID | | instagramAccountSecret | string | Yes | Instagram app secret | | instagramAccessToken | string | No | Long-lived access token | | apiType | enum | Yes | FACEBOOK_LOGIN or INSTAGRAM_LOGIN |

Response (201):

{
  "success": true,
  "data": {
    "authToken": "287cdb85-defc-42b9-8341-10fc5f5ffb2a",
    "photosCount": 50,
    "tenantId": "tenant-uuid"
  }
}

Errors:

• 400 - Validation failed
• 500 - Internal server error

Get Photos

GET /photos

Returns cached Instagram photos for the authenticated tenant (cache-first, no Instagram API call). Photos are served from Cloudinary CDN URLs when available.

Headers:

Authorization: Bearer <authToken>

Response (200):

{
  "success": true,
  "data": {
    "photos": [
      {
        "id": "18046772009220504",
        "mediaUrl": "https://res.cloudinary.com/.../instagram-media/...",
        "caption": "Post caption or empty string",
        "mediaType": "IMAGE",
        "permalink": "https://www.instagram.com/p/ABC123/"
      }
    ]
  }
}

mediaType can be: IMAGE, VIDEO, CAROUSEL_ALBUM

Note: mediaUrl contains Cloudinary CDN URL when sync completed successfully, or original Instagram URL as fallback.

Errors:

• 401 - Unauthorized
• 404 - Tenant not found
• 500 - Internal server error

Sync All Tenants

POST /sync

Triggers background synchronization of Instagram media for ALL tenants using QStash queue. During sync, images are uploaded to Cloudinary CDN for faster delivery.

Request:

{
  "secret": "your_sync_internal_secret"
}

Response (200):

{
  "success": true,
  "data": {
    "summary": {
      "totalTenants": 3,
      "queuedTasks": ["msg_abc123...", "msg_def456..."],
      "errors": []
    }
  }
}

Flow:

1. Get all tenant IDs from Redis
2. Publish a task to QStash queue for each tenant
3. QStash asynchronously calls POST /sync/process/:tenantId for each tenant
4. Each webhook refreshes token if needed and syncs photos
5. Images are uploaded to Cloudinary CDN with tenant-specific tags
6. Old Cloudinary images are cleaned up before new upload
7. Failed tasks are automatically retried (up to 3 times)

Errors:

• 401 - Unauthorized (invalid secret)
• 500 - Internal server error

Process Single Tenant Sync (QStash Webhook)

POST /sync/process/:tenantId

Internal webhook endpoint called by QStash to process sync for a single tenant. Requires valid QStash signature.

Response (200):

{
  "success": true,
  "data": {
    "result": { "photosCount": 50 },
    "refreshTokenResult": "success"
  }
}

Data Types

InstagramMedia

{
  id: string;           // Media ID
  mediaUrl: string;     // Direct URL to media
  caption: string;      // Post caption (may be empty)
  mediaType: "IMAGE" | "VIDEO" | "CAROUSEL_ALBUM";
  permalink: string;    // Instagram post URL
}

Tenant

{
  name: string;                    // Project name
  authToken: string;               // API access token
  instagramAccessToken: string;    // Encrypted access token
  instagramAccountSecret: string;  // Encrypted account secret
  instagramAccountId: string;      // Encrypted account ID
  apiType: "FACEBOOK_LOGIN" | "INSTAGRAM_LOGIN";
  last_sync: string;               // ISO timestamp
  token_update: string;            // ISO timestamp
}

Multi-Tenant & Security

• Namespace isolation: Each tenant's data is stored in isolated Redis keys (tenant:{id}, instagram:{id}:photos)
• Authentication: Bearer token required for all endpoints except /health, /instagram-callback, /auth/obtain-token, and /sync/process/*
• Encryption: Instagram tokens are encrypted with AES-256 before storage
• Webhook verification: QStash webhooks are verified using signature validation

Background Sync Setup

1. Configure QStash

Set up the following environment variables:

QSTASH_TOKEN=your_qstash_token
QSTASH_CURRENT_SIGNING_KEY=your_current_signing_key
QSTASH_NEXT_SIGNING_KEY=your_next_signing_key
WEBHOOK_BASE_URL=https://your-domain.com

2. Schedule Periodic Sync

Set up QStash scheduled publishing to call POST /sync periodically:

# Example: every hour
curl -X POST https://qstash.upstash.io/v2/schedules \
  -H "Authorization: Bearer $QSTASH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "destination": "https://your-domain.com/sync",
    "body": "{\"secret\":\"your_sync_internal_secret\"}",
    "cron": "0 * * * *"
  }'

3. Monitor

• Check QStash dashboard for task status
• Monitor application logs for errors
• Use QStash DLQ (Dead Letter Queue) for persistent failures

Error Responses

All errors follow this format:

{
  "success": false,
  "code": "ERROR_CODE",
  "errorMessage": "Human readable message"
}

Error Codes: | Code | HTTP Status | Description | |------|-------------|-------------| | VALIDATION_FAILED | 400 | Request validation failed | | UNAUTHORIZED | 401 | Missing or invalid authentication | | NOT_FOUND | 404 | Resource not found | | INTERNAL_ERROR | 500 | Internal server error |

Development

# Run in development mode with hot reload
npm run dev

# Run tests
npm run test

# Run linting
npm run lint

# Type check
npm run typecheck

License

MIT

Support

For issues and feature requests, please use GitHub Issues.