Meta Ads Scraper

A powerful TypeScript library for scraping Meta (Facebook) Ad Library data and performing AI-powered competitive analysis using Google's Gemini 2.0 Flash model.

Features

• 🚀 Meta Ad Library Scraping: Extract ad data from Facebook's Ad Library using Apify
• 🤖 AI-Powered Analysis: Analyze competitor ads with Google Gemini 2.0 Flash
• 📊 Competitive Intelligence: Generate strategic insights and ad concepts
• 🔄 Smart Caching: Reuse existing scraped data to save time and costs
• 📝 TypeScript Support: Full type safety and IntelliSense support

Installation

npm install meta-ads-scraper

Prerequisites

You'll need API keys for:

• Apify: For scraping Meta Ad Library data
• Google Gemini: For AI analysis

Get your keys:

• Apify API Token
• Google AI Studio API Key

Quick Start

Basic Usage

import { MetaAdsScraper, GeminiAnalyzer, performCompetitiveAnalysis } from 'meta-ads-scraper';

// Set up your API keys
process.env.APIFY_TOKEN = 'your-apify-token';
process.env.GEMINI_API_KEY = 'your-gemini-api-key';

// Perform complete competitive analysis
const result = await performCompetitiveAnalysis({
  clientDetails: `
    Your company is a leading e-commerce platform specializing in 
    fast delivery and competitive pricing...
  `,
  competitors: [
    {
      name: 'Competitor 1',
      url: 'https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=123456789'
    },
    {
      name: 'Competitor 2', 
      url: 'https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=987654321'
    }
  ],
  scrapingOptions: {
    maxResults: 20,
    useExistingRun: true
  },
  apiKeys: {
    apifyToken: 'your-apify-token',      // Optional: falls back to APIFY_TOKEN env var
    geminiApiKey: 'your-gemini-api-key'  // Optional: falls back to GEMINI_API_KEY env var
  }
});

console.log('Analysis Results:', result);

// Parse the results into structured JSON format
import { parseCompetitiveAnalysisResult } from 'meta-ads-scraper';

const parsedResult = parseCompetitiveAnalysisResult(result);

console.log('Parsed Restructured Ads:', parsedResult.restructuredAds);
console.log('Parsed Industry Analysis:', parsedResult.industryAnalysis);

Step-by-Step Usage

import { MetaAdsScraper, GeminiAnalyzer } from 'meta-ads-scraper';

// 1. Scrape ads
const scraper = new MetaAdsScraper(process.env.APIFY_TOKEN);
    const ads = await scraper.getAds(competitorUrl, {
      maxResults: 10,
      useExistingRun: true
    });
    
// 2. Analyze with AI
const analyzer = new GeminiAnalyzer(process.env.GEMINI_API_KEY);

// Restructure ads by company
const restructuredAds = await analyzer.restructureAds(ads);

// Analyze each company
const companyAnalysis = await analyzer.getTopIdeasPerCompany(
  'Company Name', 
  restructuredAds.text
);

// Generate industry insights
const industryAnalysis = await analyzer.industryWriteup([companyAnalysis]);

// Create ad concepts
const adConcepts = await analyzer.generateAdIdeas(
  [companyAnalysis],
  industryAnalysis,
  'Your company details...'
);

API Reference

Core Classes

MetaAdsScraper

Scrapes Meta Ad Library data using Apify.

const scraper = new MetaAdsScraper(apiToken?: string);

// Get ads from a Meta Ad Library URL
const ads = await scraper.getAds(url: string, options?: {
  maxResults?: number;
  useExistingRun?: boolean;
});

GeminiAnalyzer

Performs AI-powered analysis of scraped ad data.

const analyzer = new GeminiAnalyzer(apiKey?: string);

// Restructure raw ad data into organized format
const restructured = await analyzer.restructureAds(ads: MetaAdData[]);

// Extract company-specific data
const companyText = analyzer.extractCompanyText(text: string, companyName: string);

// Analyze a specific company
const analysis = await analyzer.getTopIdeasPerCompany(companyName: string, text: string);

// Generate industry insights
const industry = await analyzer.industryWriteup(analyses: CompanyAnalysisResult[]);

// Create ad concepts
const concepts = await analyzer.generateAdIdeas(
  companyAnalyses: CompanyAnalysisResult[],
  industryAnalysis: IndustryAnalysisResult,
  clientDetails: string
);

Main Function

performCompetitiveAnalysis(config: CompetitiveAnalysisConfig)

Performs a complete competitive analysis pipeline.

interface CompetitiveAnalysisConfig {
  clientDetails: string;
  competitors: Competitor[];
  scrapingOptions?: ScrapingOptions;
  apiKeys?: {
    apifyToken?: string;
    geminiApiKey?: string;
  };
}

interface Competitor {
  name: string;
  url: string;
}

interface ScrapingOptions {
  maxResults?: number;
  useExistingRun?: boolean;
}

Parsing Functions

Convert the text-based results into structured JSON format:

import { 
  parseCompetitiveAnalysisResult, 
  parseRestructuredAds, 
  parseIndustryAnalysis,
  parseCompanyAnalysis
} from 'meta-ads-scraper';

// Parse the complete result
const parsedResult = parseCompetitiveAnalysisResult(result);

// Or parse individual components
const parsedAds = parseRestructuredAds(result.restructuredAds);
const parsedIndustry = parseIndustryAnalysis(result.industryAnalysis.text);
const parsedCompany = parseCompanyAnalysis(result.companyAnalyses[0].text);

// Example parsed structure:
console.log(parsedResult.restructuredAds.companies[0].ads[0]);
// {
//   adId: "768050109455085",
//   assetType: "Unknown (likely motion based on the presence of \"cards\")",
//   format: "Carousel (2 cards)",
//   visualHooks: ["💧 water droplet emoji, implying skincare or hydration."],
//   audiencesAddressed: ["Shoppers interested in Early Bird Deals and the Big Billion Days sale."],
//   ageGroup: "18-45 (typical online shoppers).",
//   graphicIdentity: ["Flipkart branding (colors not specified in data). The focus is on promoting the sale event."],
//   copywritingHooks: ["Early Bird Deals Starts 8th September", "💧 Glow season starts now!"],
//   copywritingAngles: ["Limited-time offer, emphasis on deals, pre-sale hype for Big Billion Days. Focus on skincare/beauty category."],
//   usps: ["Early access to deals before the main Big Billion Days sale."],
//   cta: "Learn more",
//   adSummary: "A carousel ad promoting Flipkart's Early Bird Deals...",
//   engagementMetrics: "Not available",
//   duration: "Null (likely short video or multiple static images)",
//   targetingAssumptions: ["Existing Flipkart users, individuals interested in online shopping..."],
//   publisherPlatform: ["FACEBOOK", "INSTAGRAM"]
// }

// Example parsed company analysis structure:
console.log(parsedResult.companyAnalyses[0]);
// {
//   companyName: "Flipkart",
//   creativeSummary: "The Flipkart ads analyzed consistently promote their Big Billion Days sale...",
//   topCreatives: {
//     description: "All the ads follow a similar structure...",
//     primaryHook: "Early Bird Deals Starts 8th September, combined with Big Billion Days branding...",
//     cta: "Learn More",
//     ctaFraming: "framed as a way to access early deals and wide product selection",
//     visualComposition: "The visuals likely utilize a vibrant and festive color scheme...",
//     copywritingTone: "The tone is promotional and benefit-oriented..."
//   },
//   whatWorks: [
//     {
//       title: "Consistent Branding",
//       description: "Strong, consistent use of Big Billion Days branding..."
//     }
//   ],
//   missedOpportunities: [
//     {
//       title: "Generic Learn More CTA",
//       description: "The consistent use of Learn More is a missed opportunity..."
//     }
//   ],
//   actionableTakeaways: [
//     {
//       title: "Test Category-Specific CTAs",
//       insight: "Generic CTAs limit conversion potential.",
//       suggestedFormat: "Carousel ads with category-specific CTAs on each card...",
//       hypothesis: "Category-specific CTAs will increase click-through rates..."
//     }
//   ]
// }

Types

interface MetaAdData {
  metadata: {
    page_name: string;
    ad_archive_id: string;
    scraped_at: string;
  };
  ad_content: {
    title: string;
    body: string;
    cta_text: string;
    cta_type: string;
    link_url: string;
    cards: Array<{
      title: string;
      body: string;
    }>;
  };
  distribution: {
    publisher_platform: string[];
  };
  status: {
    is_active: boolean;
  };
  performance: {
    impressions?: number;
    spend?: number;
    currency: string;
  };
}

interface CompetitiveAnalysisResult {
  clientDetails: string;
  competitors: Competitor[];
  totalAdsScraped: number;
  restructuredAds: string;
  companyAnalyses: CompanyAnalysisResult[];
  industryAnalysis: IndustryAnalysisResult;
  adConcepts: AdConceptResult;
  analysisMetadata: {
    timestamp: string;
    scrapingOptions?: ScrapingOptions;
    totalCompanies: number;
  };
}

Configuration

API Keys

You can provide API keys in two ways:

Option 1: Environment Variables (Recommended for production)

# Required
APIFY_TOKEN=your-apify-token
GEMINI_API_KEY=your-gemini-api-key

# Optional
NODE_ENV=production

Option 2: Pass directly in config (Useful for testing or multiple accounts)

const result = await performCompetitiveAnalysis({
  clientDetails: 'Your company description...',
  competitors: [...],
  apiKeys: {
    apifyToken: 'your-apify-token',
    geminiApiKey: 'your-gemini-api-key'
  }
});

Note: If you provide API keys in the config, they will take precedence over environment variables. If neither is provided, the function will throw an error.

Meta Ad Library URLs

The scraper works with Meta Ad Library URLs in this format:

https://www.facebook.com/ads/library/?active_status=active&ad_type=all&country=ALL&is_targeted_country=false&media_type=all&search_type=page&view_all_page_id=PAGE_ID

Advanced Usage

Custom Analysis Pipeline

import { MetaAdsScraper, GeminiAnalyzer } from 'meta-ads-scraper';

async function customAnalysis() {
const scraper = new MetaAdsScraper();
const analyzer = new GeminiAnalyzer();

  // Scrape multiple competitors
  const allAds = [];
  for (const competitor of competitors) {
    const ads = await scraper.getAds(competitor.url);
    allAds.push(...ads);
  }
  
  // Custom analysis steps
  const restructured = await analyzer.restructureAds(allAds);
  
  // Process each company individually
  const analyses = [];
  for (const company of companies) {
    const companyText = analyzer.extractCompanyText(restructured.text, company);
    const analysis = await analyzer.getTopIdeasPerCompany(company, companyText);
    analyses.push(analysis);
  }
  
  return analyses;
}

Error Handling

try {
  const result = await performCompetitiveAnalysis(config);
  console.log('Analysis completed successfully');
} catch (error) {
  if (error.message.includes('API key')) {
    console.error('Invalid API key provided');
  } else if (error.message.includes('URL')) {
    console.error('Invalid Meta Ad Library URL');
  } else {
    console.error('Analysis failed:', error.message);
  }
}

Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• 📧 Email: [email protected]
• 🐛 Issues: GitHub Issues
• 📖 Documentation: GitHub Wiki

Changelog

v1.0.0

• Initial release
• Meta Ad Library scraping with Apify
• AI-powered competitive analysis with Gemini 2.0 Flash
• Complete TypeScript support
• Smart caching and reuse of existing data