scraper-wrapper

Halo aku Natz dan ini adalah library scraper yang ku buat dengan tujuan untuk mempermudah saya dan para developer yang ingin membuat bot atau sekadar bereksperimen dengan scraping. Jangan lupa untuk mencantumkan nama saya sebagai pembuat scraper ini dan juga memberikan credit kepada saya.

Table of Contents

1. Features
2. Installation
3. Quick Start
4. Usage Examples
   • Instagram
   • TikTok
   • YouTube
   • Spotify
   • Pinterest
   • Twitter
   • AI Tools
   • Image Scraper (Booru)
   • Stalk Scraper
   • BMKG
   • Drakor
   • Mediafire
   • Utilities (AIO, Unwatermark, Brat Generator, Faceswap, OCR)
5. Planned Features (Coming Soon)
6. API Reference & TypeScript Types
   • Type Definitions
7. Configuration
8. Troubleshooting & FAQ
9. Developer Guide (Contribution Rules)
10. License & Contribution

Features

• ✅ Instagram: Scrape post (Photo, Video, Reel, Carousel) & Search (User, Hashtag, Place)
• ✅ TikTok: Video info, No-watermark download, Music download, User info
• ✅ YouTube: Video info, Download (Video/Audio), Search, Playlist info
• ✅ Spotify: Search, Track info, Download (Youtube Fallback)
• ✅ Pinterest: Search & Download
• ✅ Twitter: Profile info & Media Download
• ✅ Image Boards: Support 13 situs (Gelbooru, Rule34, Danbooru, Safebooru, Konachan, Yande.re, e621, dll)
• ✅ Stalk Info: GitHub User, NPM Package, YouTube Channel, FreeFire ID, Mobile Legends ID
• ✅ BMKG: Info Gempa & Cuaca Indonesia
• ✅ Drakor: Search & Get Episode Links
• ✅ Mediafire: Get direct download links
• ✅ AI Tools: Image Upscaler (upscale & imgUpscaler), Background Remover, Faceswap, OCR
• ✅ Utilities: AIO (All-In-One Downloader), Unwatermark, Brat Generator
• ✅ Developer Friendly: TypeScript support, Complete Type Definitions, & Easy-to-use API

Installation

npm install scraper-wrapper
# atau
yarn add scraper-wrapper
# atau
pnpm add scraper-wrapper

Quick Start

Cara 1: Menggunakan all() (Recommended)

Mendapatkan semua instance scraper sekaligus dalam satu object. Cocok jika Anda ingin menggunakan beberapa scraper sekaligus di project Anda (contoh: untuk bot WhatsApp/Discord/Telegram).

import natzScraper from "scraper-wrapper";

// Inisialisasi semua scraper
const scrap = natzScraper.all();

// Sekarang semua scraper siap digunakan (mendukung alias):
// scrap.instagram  atau scrap.ig
// scrap.tiktok     atau scrap.tt
// scrap.youtube    atau scrap.yt
// scrap.spotify    atau scrap.spot
// scrap.pinterest  atau scrap.pin
// scrap.twitter    atau scrap.tw
// scrap.image      atau scrap.img
// scrap.stalk
// scrap.bmkg
// scrap.drakor
// scrap.mediafire  atau scrap.mf
// scrap.upscale
// scrap.imgUpscaler
// scrap.removebg   atau scrap.rmbg
// scrap.faceswap
// scrap.ocr
// scrap.unwatermark
// scrap.aio
// scrap.brat

Cara 2: Import Individual

Cocok jika Anda hanya butuh 1-2 scraper untuk menghemat memori.

import natzScraper from "scraper-wrapper";

const ig = natzScraper.instagram();
const tt = natzScraper.tiktok();
const tw = natzScraper.twitter();
const aio = natzScraper.aio();
// ... dst

Usage Examples

Berikut adalah dokumentasi dan panduan super lengkap untuk setiap scraper di library ini beserta contoh kode dan expected output-nya.

1. Instagram Scraper

Download Post/Reel (No Login Required)

Mendukung Photo, Video, Reel, dan Album (Carousel).

// Support: Photo, Video, Reel, Carousel
const post = await scrap.instagram.getPost("https://www.instagram.com/p/ABC123xyz/");

if (post.success) {
  console.log("Caption:", post.data.caption);
  console.log("Author:", post.data.owner.username);
  
  // URL download (video or image)
  console.log("Download:", post.data.video_url || post.data.display_url);

  // Jika berupa sidecar (album/carousel)
  if (post.data.is_sidecar) {
     const edges = post.data.edge_sidecar_to_children?.edges || [];
     edges.forEach(edge => {
         console.log("Media URL:", edge.node.video_url || edge.node.display_url);
     });
  }
}

Search (Requires Login)

Fitur search memerlukan authentication using cookies. Jika dipakai tanpa config cookie, request akan sering mendapat blokir 403 Forbidden atau 429 Too Many Requests dari Instagram.

import { InstagramScraper } from "scraper-wrapper";

const scraper = new InstagramScraper({
  cookie: "mid=...; ig_did=...; sessionid=...; ds_user_id=...; csrftoken=...;",
  csrfToken: "your_csrf_token" // opsional tapi direkomendasikan
});

const users = await scraper.searchUsers("tokyo");
if (users.success) {
   console.log(users.data.users.map(u => ({ username: u.user.username, name: u.user.full_name })));
}

const hashtags = await scraper.searchHashtags("jakarta");
if (hashtags.success) {
   console.log(hashtags.data.hashtags[0].hashtag.name);
}

const places = await scraper.searchPlaces("cafe");
if (places.success) {
   console.log(places.data.places[0].place.location.name);
}

2. TikTok Scraper

Get Video Info & Download (No Watermark)

Bisa menggunakan short url (vt.tiktok.com) atau full url.

const url = "https://vt.tiktok.com/ZS6TdYLns/";

// Get Video Metadata (Judul, stats, url mentah)
const video = await scrap.tiktok.getVideo(url);
if (video.success) {
  console.log("Desc:", video.data.description);
  console.log("Stats:", video.data.playCount, "views");
  console.log("Author:", video.data.author.nickname);
}

// Download Video secara langsung (Buffer mode) - Tanpa Watermark
const download = await scrap.tiktok.downloadVideo(url);
if (download.success) {
  // Save buffer to file local
  require('fs').writeFileSync("tiktok_video_no_wm.mp4", download.videoBuffer);
  console.log("Video berhasil disimpan!");
}

Download Music/Audio & User Info

// Download Audio (MP3) dari video
const music = await scrap.tiktok.downloadMusic(url);
if (music.success) {
  require('fs').writeFileSync("tiktok_audio.mp3", music.data.buffer);
}

// Scrape profil akun TikTok
const user = await scrap.tiktok.getUser("khaby.lame");
if (user.success) {
   console.log("Followers:", user.data.stats.followerCount);
   console.log("Likes:", user.data.stats.heartCount);
   console.log("Bio:", user.data.user.signature);
}

3. YouTube Scraper

Mendukung download video dan audio menggunakan berbagai internal failover logic (yt-dlp) untuk kepastian keberhasilan download.

Video Info & Search

const ytUrl = "https://youtu.be/dQw4w9WgXcQ";

// Get Detailed Information (Durasi, resolusi, format URL)
const info = await scrap.youtube.getVideoInfo(ytUrl);
if (info.success) {
   console.log("Title:", info.data.title);
   console.log("Duration:", info.data.duration, "seconds");
}

// Search Video on YouTube
const search = await scrap.youtube.search("typescript full course", 5);
if (search.success) {
   search.data.items.forEach(vid => {
       console.log(`[${vid.duration}] ${vid.title} - ${vid.url}`);
   });
}

// Get Playlist Videos
const playlist = await scrap.youtube.getPlaylistInfo("https://youtube.com/playlist?list=PL...");
if (playlist.success && playlist.data) {
   console.log("Total Videos:", playlist.data.videos.length);
}

Download Video/Audio to Buffer

// Download File Lengkap (Video + Audio) - Resolusi tertinggi by default
const video = await scrap.youtube.downloadVideo(ytUrl);
if (video.success) {
  require('fs').writeFileSync(`${video.data.title}.mp4`, video.data.buffer);
}

// Extract Audio Only (M4A/MP3 highest quality format)
const audio = await scrap.youtube.downloadAudio(ytUrl);
if (audio.success) {
  require('fs').writeFileSync(`${audio.data.title}.m4a`, audio.data.buffer);
}

4. Spotify Scraper

Mendapatkan lagu dari Spotify. Karena API Spotify tidak mengizinkan direct download DRM-protected stream, scraper ini secara pintar akan mencari judul lagu tersebut di YouTube Audio dan mendownload-nya sebagai media fallback (sehingga bebas DRM).

Search & Download

// Search Track/Song
const search = await scrap.spotify.searchTrack("Blinding Lights", "The Weeknd");
if (search.success && search.data.tracks.length > 0) {
    console.log("Lagu ditemukan:", search.data.tracks[0].name);
}

// Download Track (Otomatis cross-reference dengan YT Music)
const download = await scrap.spotify.downloadByName("Starboy", "The Weeknd");

if (download.success) {
  console.log("Downloaded:", download.track.name);
  console.log("Album:", download.track.album);
  
  // Buffer berisi raw audio MP3 file
  require('fs').writeFileSync("song.mp3", download.audioBuffer);
}

5. Pinterest Scraper

Scrape ide image & video dari Pinterest.

// Search Pins (Gambar/Ide)
const pins = await scrap.pinterest.search("aesthetic room decor", { limit: 15 });
if (pins.success) {
    console.log("Ide pertama:", pins.data.items[0].description);
    console.log("URL Gambar:", pins.data.items[0].images.orig.url);
}

// Download single Pin URL (Gambar / Video)
const pinUrl = "https://id.pinterest.com/pin/520165825732168923/";
const download = await scrap.pinterest.download(pinUrl);

if (download.success) {
    console.log("Downloaded title:", download.data.title);
    
    // Save buffer media (Gambar statis ato MP4)
    require('fs').writeFileSync("pinterest_img.jpg", download.data.buffer);
}

6. Twitter Scraper

Scrape profil, tweet, dan download media Twitter via metode alternatif API (syndication / vxtwitter / external scrapers).

Profile Info

const tw = await scrap.twitter.getProfile("elonmusk");
if (tw.success) {
    console.log("Name:", tw.data.name);
    console.log("Followers:", tw.data.followers);
    console.log("Bio:", tw.data.description);
}

Download Twitter Media (Video/Gif/Image)

const tweetUrl = "https://x.com/username/status/123456789";
const media = await scrap.twitter.downloadMedia(tweetUrl);

if (media.success) {
   media.data.mediaUrl.forEach((v_url, idx) => {
       console.log(`Media ${idx + 1}: ${v_url}`);
   });
}

7. AI Tools

Platform yang bisa menghasilkan atau mentransformasi gambar menggunakan AI models dari source provider yang di-scrape API-nya.

Image Upscaler (scrap.upscale dan scrap.imgUpscaler)

Memperbesar dan memperjelas foto resolusi rendah. Tersedia 2 jenis scraper dari provider web yang berbeda:

1. upscale: Provider pertama
2. imgUpscaler: Provider get1.imglarger.com

// Menggunakan Provider ImgUpscaler (direkomendasikan)
// Param 1: path, Param 2: scale (contoh: 2, 4, 8)
const up1 = await scrap.imgUpscaler.upscale("./low_res_waifu.jpg", 4);
if (up1.success) {
   console.log("URL Hasil AI:", up1.data.resultUrl);
   console.log("Disimpan lokal di:", up1.data.outputPath);
}

// Menggunakan Provider lama
const up2 = await scrap.upscale.upscale("./photo.png");
if (up2.success) {
  require('fs').writeFileSync("upscaled.png", up2.output);
  console.log("Upscaled Buffer disimpan.");
}

Background Remover (scrap.removebg)

Menghapus background (Auto-segmentation) dari gambar. Menggunakan upload endpoint internal.

const bg = await scrap.removebg.remove("./product_photo.jpg");
if (bg.success) {
  require('fs').writeFileSync("transparent.png", bg.output); // ArrayBuffer
  console.log("Background dihapus!");
}

8. Image Scraper (13 Sites)

Mendukung 13 situs booru populer dengan unified API. Anda bisa mencari tags gambar di situs-situs komunitas illustrasi/anime terbesar.

[!WARNING] Beberapa ISP (terutama Telkomsel / Indosat) memberlakukan blokir DNS/DPI / SSL Interception terhadap situs booru seperti Rule34, Danbooru, Gelbooru. Membingungkan jika Anda mendapat error ECONNRESET atau ETIMEDOUT. Solusinya sangat disarankan menggunakan proxy/VPN global, atau gunakan method safebooru yang jarang diblokir as usual indonesia.

Supported Modes:

• gelbooru, rule34xxx, safebooru (Aman untuk indo ✅), danbooru, konachan, yandere, e621, realbooru, xbooru, tbib, rule34us, lolibooru, zerochan

Generic Unified Search

// Parameter (siteName, searchOptions)
const result = await scrap.image.search('safebooru', { tags: '1girl cat_ears', limit: 10, page: 0 });

if (result.success) {
    console.log(`Mendapatkan ${result.data.posts.length} image array`);
    console.log("URL File Mentah:", result.data.posts[0].fileUrl);
    console.log("URL Thumbnail:", result.data.posts[0].previewUrl);
}

Specific Methods (Untuk mempermudah tanpa string name)

// Method khusus per provider (auto tipe aman / spesifik config)

// Safebooru (Recommended for ID User)
const safeRes = await scrap.image.safebooru("anime girl blue_hair", 0, 10);

// Konachan
const wallRes = await scrap.image.konachan("scenery", 0, true); // true parameter terakhir = Safe Mode on Konachan
if (wallRes.success) console.log(wallRes.data.posts[0].fileUrl);

// Rule34
const r34 = await scrap.image.rule34xxx("nahida", 0, 5); 

9. Stalk Scraper

Mendapatkan informasi public profile atau metadata tools.

Programmer Info (Github & NPM)

// GitHub User (Data profile)
const gh = await scrap.stalk.github("octocat");
if (gh.success) {
    console.log(`Name: ${gh.data.name}, Repos: ${gh.data.public_repos}`);
}

// NPM Package (Data library, version, author, dwnload stats)
const npm = await scrap.stalk.npm("react");
if (npm.success) {
    console.log(`Versi Terbaru: ${npm.data.version}`);
    console.log(`Desc: ${npm.data.description}`);
}

E-Sports Player Database (Mobile Legends & FreeFire)

Mengambil in-game nickname validation berdasarkan ID akun.

// Mobile Legends ID (Butuh Region/Server ID di param ke 2)
const ml = await scrap.stalk.mobileLegends("12345678", "1234");
if (ml.success) console.log("Nickname ML Anda adalah:", ml.data.nickname);

// FreeFire ID (Global ID)
const ff = await scrap.stalk.freefire("1234567890");
if (ff.success) console.log("Nickname FF Anda adalah:", ff.data.nickname);

YouTube Channel Stalk

const ytCh = await scrap.stalk.youtubeChannel("mrbeast");
if (ytCh.success) {
    console.log("Subscriber:", ytCh.data.subscriberCount);
    console.log("Deskripsi Kanal:", ytCh.data.description);
}

10. BMKG Scraper

Data dan peringatan dini langsung dari institusi cuaca dan geofisika Indonesia (BMKG API Public).

// 1. Info Gempa Bumi Terkini
const gempaTerkini = await scrap.bmkg.gempaTerkini();
if (gempaTerkini.success) {
   console.log(`[GEMPA TERKINI] M${gempaTerkini.data.Magnitude} di ${gempaTerkini.data.Wilayah}`);
   console.log("Waktu:", gempaTerkini.data.Tanggal, gempaTerkini.data.Jam);
   console.log("Potensi:", gempaTerkini.data.Potensi);
}

// 2. Daftar Gempa Dirasakan (Log 10-15 gempa terakhir)
const gempaDirasa = await scrap.bmkg.gempaDirasakan();
if (gempaDirasa.success) {
   console.log(gempaDirasa.data[0].Dirasakan);
}

// 3. Prakiraan Cuaca Harian Lengkap (berdasarkan nama provinsi)
const cuaca = await scrap.bmkg.cuaca("DKI Jakarta");
if (cuaca.success) {
   const jakartaPusat = cuaca.data.find(kota => kota.nama === "Jakarta Pusat");
   console.log("Prediksi Cuaca:", jakartaPusat.cuacaHariIni);
}

// 4. Nowcasting (Cuaca Satelit MMI real-time Indonesia)
const nowcasting = await scrap.bmkg.nowcasting();
if (nowcasting.success) console.log(nowcasting.data.image_url);

11. Drakor Scraper

Scraper untuk mem-bypass dan mendapatkan video direct play/download dan detail Series Drama Korea tanpa iklan dari web drakor Indo populer.

Search & Mendapatkan Download Link

// 1. Search Query
const search = await scrap.drakor.search("Vincenzo");
if (search.success && search.data.length > 0) {
    console.log("Title Found:", search.data[0].title);
    console.log("Drakor Page URL:", search.data[0].url);
    
    // 2. Get Detail Episodes & Links (menggunakan url dari search result)
    const detail = await scrap.drakor.getDetail(search.data[0].url);
    if (detail.success) {
        console.log("Sinopsis:", detail.data.sinopsis);
        console.log("Total Eps:", detail.data.episodes.length);
        
        // 3. Lihat link download resolusi 720p untuk episode 1
        console.log("Link HD:", detail.data.episodes[0].links['720p']);
    }
}

12. Mediafire Scraper

Melewati halaman countdown / click to verify dari Mediafire dan langsung mengekstrak direct URL link download downloadXXXX.mediafire.com.

const mfUrl = "https://www.mediafire.com/file/xxxxx/file.zip/file";
const result = await scrap.mediafire.download(mfUrl);

if (result.success) {
    console.log("Nama File:", result.data.filename);
    console.log("Ukuran:", result.data.size);
    // URL raw/direct yg bisa langsung dimasukkan ke wget/browser/axios
    console.log("Direct Link:", result.data.url);
}

13. Utilities (AIO, Unwatermark, Brat, Faceswap, OCR)

Fitur-fitur tambahan pelengkap untuk melengkapi library scraper All-In-One ini. Sangat berguna untuk Bot Utilities.

AIO (All-In-One Downloader)

Sistem ini menggunakan third-party tool yang mendukung belasan platform social media sekaligus mulai dari facebook, reddit, coub, capcut dll. Tidak perlu pakai individual scraper jika hanya butuh link videonya.

// url bisa video ig, fb, reddit, dlldll
const vid = await scrap.aio.download("https://www.reddit.com/r/.../video");
if (vid.success) {
    console.log(vid.data.medias[0].url);
}

Unwatermark Video

Menghapus object berupa tulisan atau logo watermark dari file video.

const uw = await scrap.unwatermark.process("./videoAsliDenganWatermark.mp4");
if (uw.success) {
   console.log("URL Hasil Video Bersih:", uw.data.resultUrl);
}

Brat Generator

Menciptakan tulisan dengan style "BRAT" (Hijau lemon, resolusi blur ciri khas meme BRAT x Charli XCX).

// Anda akan mendapat buffer base64/image langsung
const brat = await scrap.brat.generateBratImage("scraper-wrapper is awesome!");
if (brat.success) {
   require('fs').writeFileSync("brat_style.jpg", brat.data.buffer);
}

// Generate Brat Animated Banner (Sticker)
const animatedBrat = await scrap.brat.generateBratSticker("wow!");
if (animatedBrat.success) {
   console.log(animatedBrat.data.frameUrls); // array frame
}

Faceswap Scraper

AI Deepfake face replacement. Menyuntik wajah gambar A ke object di gambar B.

// Parameter Face Swap (Source Face Image -> Target Body Image)
const target = await scrap.faceswap.swap("./wajah_artis.jpg", "./badan_saya.jpg");
if (target.success) {
    console.log("URL Gambar Hasil:", target.data.swappedUrl);
}

OCR (Image to Text)

Mengekstrak karakter alfabet di dalam sebuah file gambar.

// Mendukung url img
const ocrUrl = await scrap.ocr.scan("https://example.com/screenshot_ktp.jpg");
if (ocrUrl.success) {
    console.log("Text didalam gambar:", ocrUrl.data.text);
}

Planned Features (Coming Soon)

Berikut adalah daftar fitur & scraper yang direncanakan untuk ditambahkan di versi mendatang, stay tuned dan siap-siap perbarui package npm kalian!

| No | Fitur / Scraper | Status | |:--:|:------------------------------|:-----------:| | 1 | Sticker Pack Download | ⏳ Planned | | 2 | Github Scraper (Detailed) | ⏳ Planned | | 3 | NPM Scraper (Detailed) | ⏳ Planned | | 4 | Drive Scraper (Pussh / Gdrive)| ⏳ Planned | | 5 | Crunchyroll Scraper | ⏳ Planned | | 6 | Hdvid | ⏳ Planned | | 7 | Dramabox | ⏳ Planned | | 8 | Wwchar | ⏳ Planned | | 9 | Mconverter | ⏳ Planned | | 10 | Videy | ⏳ Planned | | 11 | Sfiledl | ⏳ Planned |

API Reference & TypeScript Types

Seluruh library ini sangat bergantung pada Typescript. Kami telah mengekspor puluhan interface dan type untuk memudahkan Intellisense saat ada integrasi data.

Berikut penjabaran objek Response yang selalu konsisten dari framework arsitektur wrapper ini.

Struktur ApiResponse<T> Wrapper Utama

Tiap scraper API pasti menghasilkan Object JS dengan spesifikasi ini sehingga error tracking akan sangat mudah memakai if (result.success). TIDAK AKAN PERNAH menghasilkan throw new Error() dari API classes jika method digunakan normal (Error tertangkap as success=false).

export type SuccessResponse<T> = {
  success: true;
  data: T;
};

export type ErrorResponse = {
  success: false;
  error: string;
};

// Return Tipe Resmi
export type ApiResponse<T> = SuccessResponse<T> | ErrorResponse;

Type Definitions

Meskipun IDE Anda sudah memiliki auto-detect Types, di bawah ini kami jelaskan properties paling krusial untuk type defs dari scraping provider populer: InstagramPostData, TikTokVideo, dan BooruImage.

File bmkgTypes.ts

export interface WeatherNowcasting {
    success: boolean;
    data: any; // Ideally more specific if possible, but BMKG ArcGIS response can be complex
    totalFeatures: number;
    timestamp: string;
}

export interface EarthquakeInfo {
    Tanggal: string;
    Jam: string;
    DateTime: string;
    Coordinates: string;
    Lintang: string;
    Bujur: string;
    Magnitude: string;
    Kedalaman: string;
    Wilayah: string;
    Potensi: string;
    Dirasakan: string;
    Shakemap?: string;
}

export interface EarthquakeData {
    Infogempa: {
        gempa: EarthquakeInfo | EarthquakeInfo[];
    };
}

export interface EarthquakeResponse {
    success: boolean;
    data: EarthquakeData;
    timestamp: string;
    totalGempa?: number;
}

export interface ShakemapDownload {
    success: boolean;
    filename: string;
    path: string;
    size: number;
    url: string;
    timestamp: string;
}

export interface ForecastResponse {
    success: boolean;
    data: any; // XML converted to JSON or raw XML string? The original code returns axios response.data, which is XML string usually unless parsed.
    // However, looking at original code: `const response = await axios.get(url); return { ... data: response.data }`.
    // Axios usually returns string for XML unless configured otherwise or using a transformer.
    // We'll assume string for now, or maybe the user wants JSON?
    // The original code doesn't parse XML.
    provinsi: string;
    timestamp: string;
}

export interface FormattedEarthquake {
    tanggal: string;
    jam: string;
    datetime: string;
    coordinates: string;
    lintang: string;
    bujur: string;
    magnitude: string;
    kedalaman: string;
    wilayah: string;
    potensi: string;
    dirasakan: string;
    shakemap?: string;
}

File bratTypes.ts

export interface BratConfig {
    size?: number;
    bg?: string;
    color?: string;
    padding?: number;
    gifDelay?: number;
    gifHoldMs?: number;
    gifSize?: number;
    gifQuality?: number;
}

export interface BratStickerConfig {
    fontSize?: number;
    fontFamily?: string;
    color?: string;
    strokeColor?: string;
    strokeWidth?: number;
}

File cleanResponseTypes.ts

/**
 * Clean Response Types
 * Simplified response types containing only essential data fields
 */

// TikTok Clean Types
export interface CleanTikTokUser {
    uniqueId: string;
    nickname: string;
    avatar: string;
    signature?: string;
    verified: boolean;
    followers: number;
    following: number;
    likes: number;
    videos: number;
}

export interface CleanTikTokVideo {
    id: string;
    description: string;
    createdAt: string;
    duration: number;
    author: {
        uniqueId: string;
        nickname: string;
        verified: boolean;
    };
    stats: {
        likes: number;
        comments: number;
        shares: number;
        plays: number;
    };
    downloadURL: string;
    cover?: string;
}

export interface CleanTikTokMusic {
    title: string;
    author: string;
    duration: number;
    playUrl: string;
}

// Instagram Clean Types
export interface CleanInstagramUser {
    username: string;
    full_name: string;
    profile_pic_url: string;
    is_verified: boolean;
    is_private: boolean;
}

export interface CleanInstagramHashtag {
    name: string;
    media_count: number;
}

export interface CleanInstagramSearchResponse {
    users: CleanInstagramUser[];
    hashtags: CleanInstagramHashtag[];
    has_more: boolean;
}

// Twitter Clean Types (for future use)
export interface CleanTwitterUser {
    username: string;
    name: string;
    profile_image_url: string;
    verified: boolean;
    followers_count: number;
}

// Spotify Clean Types (for future use)
export interface CleanSpotifyTrack {
    name: string;
    artist: string;
    album: string;
    duration_ms: number;
    preview_url?: string;
}

File drakorTypes.ts

export interface DrakorConfig {
    debug?: boolean;
}

export interface Genre {
    name: string;
    url: string;
}

export interface Artist {
    name: string;
    url: string;
}

export interface DramaItem {
    title: string;
    url: string;
    image: string;
    type: string; // Movie or TV
    duration?: string;
    quality?: string;
    rating?: string;
    releaseDate?: string;
    slug: string;
}

export interface Episode {
    title: string;
    url: string | undefined;
    episode: string;
    date: string;
}

export interface DramaDetail {
    title: string; // English title
    koreanTitle?: string;
    image: string | undefined;
    synopsis: string;
    rating: string;
    type: string;
    status: string;
    episodeCount?: string;
    season?: string;
    firstAirDate?: string;
    country?: string;
    director?: string;
    videoLength?: string;
    views?: string;
    postedOn?: string;
    genres: Genre[];
    artists: Artist[];
    episodes: Episode[];
}

export interface Pagination {
    currentPage: number;
    hasNext: boolean;
    hasPrev: boolean;
}

export interface DramaList {
    dramas: DramaItem[];
    pagination: Pagination;
}

File faceswapTypes.ts

export interface FaceswapConfig {
    timeout?: number;
    userAgent?: string;
}

export interface FaceswapResult {
    job_id: string;
    image: string;
}

File gif-encoder-2.d.ts

declare module 'gif-encoder-2';

File imageScraperTypes.ts

// ===== Tag Types (ported from Pascal BooruScraper.Interfaces) =====
export enum BooruTagType {
    General = 'general',
    Copyright = 'copyright',
    Character = 'character',
    Artist = 'artist',
    Metadata = 'metadata',
    Species = 'species',  // e621 specific
}

export interface BooruTag {
    value: string;
    kind: BooruTagType;
    count?: number;
}

export interface BooruComment {
    id: number;
    postId?: number;
    creatorId?: number;
    username?: string;
    timestamp?: string;
    text: string;
    score?: number;
    isDeleted?: boolean;
}

// ===== Unified BooruPost (ported from Pascal IBooruPost) =====
export interface BooruPost {
    id: number;
    thumbnail?: string;       // Preview image URL
    contentUrl?: string;      // Full-size image/video URL (file_url)
    sampleUrl?: string;       // Sample/large size URL
    score?: number;
    uploader?: string;
    uploaderId?: number;
    tags?: string[];          // Simple tag strings
    tagDetails?: BooruTag[];  // Rich tag objects with type + count
    comments?: BooruComment[];
    rating?: string;          // s=safe, q=questionable, e=explicit
    md5?: string;
    width?: number;
    height?: number;
    sampleWidth?: number;
    sampleHeight?: number;
    previewWidth?: number;
    previewHeight?: number;
    fileSize?: number;
    fileExt?: string;
    sourceUrl?: string;
    createdAt?: string;
    parentId?: number | null;
    hasChildren?: boolean;
    hasComments?: boolean;
    hasNotes?: boolean;
    favCount?: number;
    [key: string]: any;
}

// ===== Booru Search Options =====
export interface BooruSearchOptions {
    tags: string;
    page?: number;
    limit?: number;
    safe?: boolean;
}

// ===== Booru Search Result =====
export interface BooruSearchResult {
    site: string;
    posts: BooruPost[];
    total?: number;
    page: number;
    limit: number;
}

// ===== Supported sites enum =====
export type BooruSiteName =
    | 'gelbooru'
    | 'rule34xxx'
    | 'danbooru'
    | 'safebooru'
    | 'konachan'
    | 'yandere'
    | 'e621'
    | 'realbooru'
    | 'xbooru'
    | 'tbib'
    | 'rule34us'
    | 'lolibooru'
    | 'zerochan';

// ===== Legacy types (backward compat) =====
export interface ImageScraperConfig {
    debug?: boolean;
    timeout?: number;
    headers?: Record<string, string>;
    httpsAgent?: any;
}

export interface BooruResult {
    result: any;
    random: BooruPost;
    posts: BooruPost[];
}

export interface KonachanPost extends BooruPost {
    author?: string;
    jpeg_url?: string;
    jpeg_width?: number;
    jpeg_height?: number;
    jpeg_file_size?: number;
    actual_preview_width?: number;
    actual_preview_height?: number;
}

File imgUpscalerTypes.ts

export interface ImgUpscalerConfig {
    pollInterval?: number;
    maxRetries?: number;
    timeout?: number;
    userAgent?: string;
}

export interface ImgUpscalerResult {
    originalPath: string;
    outputPath: string;
    resultUrl: string;
    scale: number;
}

File instagramTypes.ts

/**
 * Instagram data type definitions
 */

export interface InstagramDimensions {
  height: number;
  width: number;
}

export interface InstagramDisplayResource {
  src: string;
  config_width: number;
  config_height: number;
}

export interface InstagramOwner {
  id: string;
  username: string;
  full_name?: string;
  profile_pic_url?: string;
  is_verified?: boolean;
}

export interface InstagramLocation {
  id: string;
  name: string;
  slug?: string;
  address_json?: string;
}

export interface InstagramClipsMusicInfo {
  artist_name?: string;
  song_name?: string;
  uses_original_audio?: boolean;
  should_mute_audio?: boolean;
  audio_id?: string;
}

export interface InstagramSidecarEdge {
  node: {
    __typename: string;
    id: string;
    shortcode: string;
    dimensions: InstagramDimensions;
    display_url: string;
    display_resources: InstagramDisplayResource[];
    is_video: boolean;
    video_url?: string;
    has_audio?: boolean;
  };
}

export interface InstagramPostData {
  __typename?: string;
  shortcode?: string;
  dimensions?: InstagramDimensions;
  display_url?: string;
  display_resources?: InstagramDisplayResource[];
  has_audio?: boolean;
  video_url?: string;
  video_view_count?: number;
  video_play_count?: number;
  is_video?: boolean;
  caption?: string;
  is_paid_partnership?: boolean;
  location?: InstagramLocation;
  owner?: InstagramOwner;
  product_type?: string;
  video_duration?: number;
  thumbnail_src?: string;
  clips_music_attribution_info?: InstagramClipsMusicInfo;
  sidecar?: InstagramSidecarEdge[];
}

export interface InstagramRawMedia {
  __typename?: string;
  shortcode?: string;
  dimensions?: InstagramDimensions;
  display_url?: string;
  display_resources?: InstagramDisplayResource[];
  has_audio?: boolean;
  video_url?: string;
  video_view_count?: number;
  video_play_count?: number;
  is_video?: boolean;
  edge_media_to_caption?: {
    edges: Array<{
      node: {
        text: string;
      };
    }>;
  };
  is_paid_partnership?: boolean;
  location?: InstagramLocation;
  owner?: InstagramOwner;
  product_type?: string;
  video_duration?: number;
  thumbnail_src?: string;
  clips_music_attribution_info?: InstagramClipsMusicInfo;
  edge_sidecar_to_children?: {
    edges: InstagramSidecarEdge[];
  };
}

export interface InstagramGraphQLResponse {
  data: {
    xdt_shortcode_media: InstagramRawMedia;
  };
}

// Search result types
export interface InstagramSearchUser {
  pk?: string;
  username?: string;
  full_name?: string;
  is_private?: boolean;
  is_verified?: boolean;
  profile_pic_url?: string;
  profile_pic_id?: string;
  follower_count?: number;
  latest_reel_media?: number;
}

export interface InstagramSearchHashtag {
  id?: string;
  name?: string;
  media_count?: number;
  profile_pic_url?: string;
  search_result_subtitle?: string;
}

export interface InstagramSearchPlace {
  location?: {
    pk?: string;
    name?: string;
    address?: string;
    city?: string;
    short_name?: string;
    lng?: number;
    lat?: number;
    external_source?: string;
    facebook_places_id?: string;
  };
  title?: string;
  subtitle?: string;
}

export interface InstagramTopSearchResponse {
  users?: Array<{
    position?: number;
    user?: InstagramSearchUser;
  }>;
  places?: Array<{
    position?: number;
    place?: InstagramSearchPlace;
  }>;
  hashtags?: Array<{
    position?: number;
    hashtag?: InstagramSearchHashtag;
  }>;
  has_more?: boolean;
  rank_token?: string;
  clear_client_cache?: boolean;
  status?: string;
  [key: string]: any;
}

export type PostType = 'p' | 'reels' | 'reel' | 'stories';

export interface ScraperConfig {
  userAgent?: string;
  xIgAppId?: string;
  xFbLsd?: string;
  xAsbdId?: string;
  timeout?: number;
  /** Cookie string for authenticated requests (required for search) */
  cookie?: string;
  /** CSRF token extracted from cookie (required for search) */
  csrfToken?: string;
  /** Use automatic token generator for search (alternative to manual cookie) */
  useTokenGenerator?: boolean;
}

// Alias untuk konsistensi
export type InstagramConfig = ScraperConfig;

File liputan6.types.ts

export interface Liputan6News {
    title: string;
    link: string;
    image: string;
    tag: string;
    source: string;
}

File mediafireTypes.ts

export interface MediafireConfig {
    timeout?: number;
    userAgent?: string;
}

export interface MediafireResult {
    downloadUrl: string;
    fileName: string;
    fileSize: string;
    fileType: string;
}

File myanimelistTypes.ts

export interface MalSeasonAnime {
    id: string | null;
    title: string;
    image: string | undefined;
    type: string;
    episodes: string;
    source: string;
    genres: string[];
    synopsis: string;
    score: string;
    members: string;
}

export interface MalSearchResult {
    type: string;
    id: string | null;
    title: string;
    image: string | undefined;
    synopsis: string;
    type_detail: string;
    episodes: string;
    score: string;
}

export interface MalAnimeStaff {
    name: string;
    url: string;
    role: string;
    image: string;
}

export interface MalAnimeCharacter {
    name: string;
    url: string;
    image: string;
    role: string;
    voice_actor: {
        name: string;
        url: string;
        language: string;
        image: string;
    }[];
}

export interface MalAnimeDetail {
    title: string;
    image: string | undefined;
    synopsis: string;
    synonim: {
        synonym?: string;
        japanese?: string;
        english?: string;
        [key: string]: string | undefined;
    };
    type: string;
    episodes: string;
    status: string;
    aired: string;
    premiered: string;
    broadcast: string;
    producers: { name: string; url: string }[];
    licensors: { name: string; url: string }[];
    studios: { name: string; url: string }[];
    source: string;
    genres: { name: string; url: string }[];
    themes: { name: string; url: string }[];
    demographic: string;
    duration: string;
    rating: string;
    score: string;
    scoreCount: string;
    ranked: string;
    popularity: string;
    members: string;
    favorites: string;
    characters: MalAnimeCharacter[];
    staff: MalAnimeStaff[];
}

export interface MalCharacterDetail {
    id: string | null;
    name: string;
    nameKanji: string | null;
    image: string | undefined;
    description: string;
    nicknames: string[];
    animeography: {
        id: string | null;
        title: string;
        role: string;
        image: string | undefined;
    }[];
    mangaography: string[]; // Not scraped in current implementation but kept for structure
    voiceActors: {
        id: string | null;
        name: string;
        language: string;
        image: string | undefined;
    }[];
    memberFavorites: string | null;
}

export interface MalVoiceActorDetail {
    id: string | null;
    name: string;
    nameKanji: string | null;
    image: string | undefined;
    givenName: string | null;
    familyName: string | null;
    birthday: string | null;
    website: string | null;
    memberFavorites: string | null;
    more: string | null;
    voiceActingRoles: {
        animeId: string | null;
        animeTitle: string;
        characterId: string | null;
        characterName: string;
        role: string;
        image: string | undefined;
    }[];
    animeStaffPositions: string[]; // Not scraped in current implementation but kept for structure
}

export interface MalNewsArticle {
    id: string | null;
    title: string;
    image: string | undefined;
    text: string;
    author: string;
    date: string;
    comments: string;
    tags: string[];
}

export interface MalRecommendation {
    user: string;
    date: string;
    animes: {
        id: string | null;
        title: string | undefined;
        image: string | undefined;
    }[];
    description: string;
}

export interface MalPopularAnime {
    rank: string;
    id: string | null;
    title: string;
    image: string | undefined;
    type: string | null;
    episodes: string | null;
    aired: string | null;
    members: string;
    score: string;
}

File other-services.types.ts

export interface Holiday {
    summary: string;
    days: string;
    dateMonth: string;
}

export interface NationalHolidayResult {
    nextLibur: string;
    libnas_content: Holiday[];
}

export interface MyInstantsResult {
    title: string;
    soundLink: string;
    pageLink: string;
}

export interface WallpaperResult {
    type: string;
    source: string;
    image: string[];
}

export interface TranscriptionResult {
    author: string;
    transcribe: string;
}

File pinterestTypes.ts

/**
 * Pinterest data type definitions
 */

export interface PinterestPin {
  id: string;
  title?: string;
  description?: string;
  link?: string;
  image_url?: string;
  image_large_url?: string;
  image_medium_url?: string;
  image_small_url?: string;
  video_url?: string;
  is_video?: boolean;
  board?: {
    id: string;
    name: string;
    url?: string;
  };
  created_at?: string;
  pinner?: {
    username?: string;
    full_name?: string;
    profile_url?: string;
    image_url?: string;
  };
  domain?: string;
  media_type?: string;
  alt_text?: string;
  product_type?: string;
}

export interface PinterestSearchResponse {
  resource_response?: {
    status?: string;
    code?: number;
    message?: string;
    endpoint_name?: string;
    data?: {
      results?: any[];
      bookmark?: string;
    };
  };
  client_context?: any;
  resource?: any;
}

export interface PinterestImageVariant {
  url: string;
  width: number;
  height: number;
}

export interface PinterestMedia {
  images?: {
    orig?: PinterestImageVariant;
    '170x'?: PinterestImageVariant;
    '236x'?: PinterestImageVariant;
    '474x'?: PinterestImageVariant;
    '736x'?: PinterestImageVariant;
    '1200x'?: PinterestImageVariant;
    [key: string]: PinterestImageVariant | undefined;
  };
  videos?: {
    video_list?: {
      [key: string]: {
        url: string;
        width: number;
        height: number;
        duration?: number;
      };
    };
  };
  type?: string;
  has_video?: boolean;
}

export interface PinterestSearchOptions {
  scope?: 'pins' | 'boards' | 'users';
  bookmark?: string;
  page_size?: number;
}

export interface PinterestConfig {
  userAgent?: string;
  timeout?: number;
  cookie?: string;
  csrfToken?: string;
}

export interface PinterestDownloadOptions {
  quality?: 'orig' | '1200x' | '736x' | '474x' | '236x' | '170x';
  videoQuality?: 'V_720P' | 'V_HLSV4' | 'V_HLSV3_WEB';
}

File removebgTypes.ts

/**
 * RemoveBg data type definitions
 */

export interface RemoveBgConfig {
  timeout?: number;
  userAgent?: string;
}

export interface RemoveBgResult {
  success: boolean;
  url?: string;
  output?: string;  // Alias untuk url untuk konsistensi dengan upscaler
  message?: string;
  error?: string;
}

export interface RemoveBgUploadResponse {
  server_filename: string;
  [key: string]: any;
}

export interface RemoveBgTokens {
  token: string;
  task: string;
}

export interface UguuUploadResponse {
  success: boolean;
  files: Array<{
    hash: string;
    name: string;
    url: string;
    size: number;
  }>;
}

// Alias untuk konsistensi penamaan
export type RemovebgConfig = RemoveBgConfig;
export type RemovebgResult = RemoveBgResult;

File responseTypes.ts

/**
 * Common response wrapper types
 */

export interface SuccessResponse<T> {
  success: true;
  data: T;
}

export interface ErrorResponse {
  success: false;
  error: string;
}

export type ApiResponse<T> = SuccessResponse<T> | ErrorResponse;

File spotifyTypes.ts

/**
 * Spotify Scraper Types
 */

export interface SpotifyConfig {
    clientId?: string;
    clientSecret?: string;
    timeout?: number;
    userAgent?: string;
}

export interface SpotifyArtist {
    id: string;
    name: string;
    uri: string;
    external_urls?: {
        spotify: string;
    };
}

export interface SpotifyImage {
    url: string;
    height: number;
    width: number;
}

export interface SpotifyAlbumSimple {
    id: string;
    name: string;
    uri: string;
    album_type: string;
    release_date: string;
    total_tracks: number;
    images: SpotifyImage[];
    artists: Array<{
        id: string;
        name: string;
    }>;
    external_urls?: {
        spotify: string;
    };
}

export interface SpotifyTrack {
    id: string;
    uri: string;
    name: string;
    duration: number;
    explicit: boolean;
    popularity: number;
    preview_url: string | null;
    track_number: number;
    disc_number: number;
    is_playable: boolean;
    audiodownload?: string | null;
    artists: SpotifyArtist[];
    album: SpotifyAlbumSimple | null;
    external_urls?: {
        spotify: string;
    };
    external_ids?: {
        isrc?: string;
    };
}

export interface SpotifyAlbum {
    id: string;
    uri: string;
    name: string;
    album_type: string;
    total_tracks: number;
    release_date: string;
    label: string;
    popularity: number;
    genres: string[];
    images: SpotifyImage[];
    copyrights: Array<{
        text: string;
        type: string;
    }>;
    artists: SpotifyArtist[];
    tracks: SpotifyTrack[];
    external_urls?: {
        spotify: string;
    };
}

export interface SpotifyPlaylist {
    id: string;
    name: string;
    description: string;
    owner: {
        id: string;
        display_name: string;
    };
    public: boolean;
    collaborative: boolean;
    followers: number;
    total_tracks: number;
    images: SpotifyImage[];
    tracks: Array<{
        added_at: string;
        track: {
            id: string;
            name: string;
            duration: number;
            audio: string | null;
            audiodownload?: string | null;
            artists: Array<{
                id: string;
                name: string;
            }>;
            album: {
                id: string;
                name: string;
                images: SpotifyImage[];
            } | null;
        };
    }>;
    external_urls?: {
        spotify: string;
    };
}

export interface SpotifySearchResults {
    tracks: SpotifyTrack[];
    albums: SpotifyAlbumSimple[];
    artists: Array<{
        id: string;
        uri: string;
        name: string;
        popularity: number;
        followers: number;
        genres: string[];
        images: SpotifyImage[];
        external_urls?: {
            spotify: string;
        };
    }>;
    playlists: Array<{
        id: string;
        uri: string;
        name: string;
        description: string;
        owner: string;
        total_tracks: number;
        images: SpotifyImage[];
        public: boolean;
        external_urls?: {
            spotify: string;
        };
    }>;
}

export interface SpotifyDownloadResult {
    success: boolean;
    track?: SpotifyTrack;
    audioBuffer?: Buffer;
    message?: string;
    source?: 'spotify' | 'youtube';  // Indicates where audio was downloaded from
    youtubeUrl?: string;              // YouTube URL if YouTube was used
}

export interface SpotifyAlbumDownloadResult {
    success: boolean;
    type: 'album';
    metadata: {
        title: string;
        artists: string;
        cover: string;
        releaseDate: string;
    };
    trackList: Array<{
        success: boolean;
        metadata: SpotifyTrack;
        audioBuffer?: Buffer;
        error?: string;
    }>;
}

export interface SpotifyPlaylistDownloadResult {
    success: boolean;
    type: 'playlist';
    metadata: {
        title: string;
        artists: string;
        cover: string;
    };
    trackList: Array<{
        success: boolean;
        metadata: any;
        audioBuffer?: Buffer;
        error?: string;
    }>;
}

File stalkerTypes.ts

// Twitter Types
export interface TwitterTweet {
    author: {
        username: string;
        nickname: string;
        profile_pic: string;
        upload_at: string;
    };
    title: string;
    media: string;
    retweet: string;
    likes: string;
}

export interface TwitterProfile {
    username: string;
    nickname: string;
    background: string;
    profile: string;
    desc_text: string;
    join_at: string;
    map: string;
    tweets_count: string;
    followers: string;
    following: string;
    media_count: number;
    media: TwitterTweet[] | string;
}

export interface TwitterDownloadMedia {
    quality: string;
    type: string;
    url: string;
}

export interface TwitterDownload {
    username?: string;
    caption?: string;
    thumbnail?: string;
    likes?: string;
    media: TwitterDownloadMedia[];
    source: 'ssstwitter' | 'twmate' | 'twitter';
}

// NPM Types
export interface NpmCollaborator {
    name: string;
    url: string;
}

export interface NpmInfoItem {
    type: string;
    result: string;
}

export interface NpmPackage {
    title: string;
    language: string;
    publish: string;
    readme: string;
    explore: string;
    dependencies: string;
    dependents: string;
    version_count: string;
    keywords: string[];
    install: string;
    info: NpmInfoItem[];
    collaborator: NpmCollaborator[];
}

// YouTube Types
export interface YoutubeImage {
    url: string;
    sizes: string;
}

export interface YoutubeProfileImage {
    image: string | undefined;
    images: YoutubeImage[];
}

export interface YoutubeBannerImage {
    image: string | undefined;
    images: YoutubeImage[];
}

export interface YoutubeChannel {
    status: number;
    name: string;
    channel_url: string | undefined;
    id: string;
    profile: string | undefined;
    creation_date: string;
    video_count: string;
    subscriber_count: string;
    total_views: string;
    country: string;
    monetization: string | undefined;
    description: string;
    profile_image: YoutubeProfileImage;
    banner_image: YoutubeBannerImage;
}

// Game Types
export interface FreeFireProfile {
    status: number;
    id: string;
    nickname: string;
}

export interface MobileLegendsProfile {
    status: number;
    id: string;
    zoneId: string;
    nickname: string;
    // The structure depends on what duniagames API returns, assuming similar to FF or minimal info
    // Looking at original code: return response.data.gameDetail;
    // We should probably type `gameDetail` as any or specific if known.
    // Let's use `any` for gameDetail properties for now as it's external API response without clear schema in original code.
    [key: string]: any;
}

File tiktokTypes.ts

/**
 * TikTok Scraper Types
 * Converted from scraper-x JavaScript interfaces to TypeScript
 */

export interface TikTokConfig {
    cookiesPath?: string;
    timeout?: number;
    userAgent?: string;
    /** @deprecated No longer used - tikwm API is used instead */
    usePuppeteer?: boolean;
    /** Return cleaned responses with only essential fields (default: true) */
    cleanResponse?: boolean;
}

export interface TikTokAuthor {
    id: string;
    uniqueId: string;
    nickname: string;
    avatarThumb?: string;
    avatarMedium?: string;
    avatarLarger?: string;
    signature?: string;
    verified?: boolean;
    secUid?: string;
    privateAccount?: boolean;
}

export interface TikTokVideo {
    id: string;
    description: string;
    createdAt: string;
    height: number;
    width: number;
    duration: number;
    resolution: string;
    shareCount: number;
    likesCount: number;
    commentCount: number;
    playCount: number;
    downloadURL: string;
    cover?: string;
    dynamicCover?: string;
    playURL?: string;
    format?: string;
    author?: TikTokAuthor;
    directVideoUrl?: string;
}

export interface TikTokUser {
    id: string;
    uniqueId: string;
    nickname: string;
    avatar: string;
    signature: string;
    createdAt: string;
    verified: boolean;
    secretUID: string;
    bioLink: string;
    privateAccount: boolean;
    followers: number;
    following: number;
    likes: number;
    videos: number;
}

export interface TikTokMusic {
    id: string;
    title: string;
    playUrl: string;
    coverLarge: string;
    coverThumb: string;
    authorName: string;
    duration: number;
    original: boolean;
    album: string;
}

export interface TikTokDownloadOptions {
    path?: string;
    watermark?: boolean;
}

export interface TikTokDownloadResult {
    success: boolean;
    videoPath?: string;
    videoInfo?: TikTokVideo;
    videoBuffer?: Buffer;
    message?: string;
    error?: string;
}

export interface TikTokNoWatermarkResult {
    success: boolean;
    downloadUrl?: string;
    error?: string;
}

export interface TikTokSearchResult {
    videos: TikTokVideo[];
    hasMore: boolean;
    cursor?: string;
}

File unwatermarkTypes.ts

export interface UnwatermarkConfig {
    timeout?: number;
    userAgent?: string;
}

export interface UnwatermarkResult {
    job_id: string;
    input_url: string;
    output_url: string;
}

File upscaleTypes.ts

export interface UpscalerResult {
  success: boolean;
  id?: string;
  input?: string;
  output?: string;
  error?: string;
}

export interface UpscalerConfig {
  timeout?: number;
  pollInterval?: number;
  maxPoll?: number;
  userAgent?: string;
}

File youtubeTypes.ts

/**
 * YouTube Scraper Types
 */

export interface YouTubeConfig {
    timeout?: number;
    userAgent?: string;
    quality?: 'highest' | 'lowest' | 'highestaudio' | 'lowestaudio';
}

export interface YouTubeVideoFormat {
    itag: number;
    url: string;
    mimeType: string;
    bitrate: number;
    width?: number;
    height?: number;
    quality: string;
    qualityLabel?: string;
    fps?: number;
    hasVideo: boolean;
    hasAudio: boolean;
    container: string;
    codecs: string;
    audioQuality?: string;
    approxDurationMs?: string;
    audioBitrate?: number;
}

export interface YouTubeVideoInfo {
    videoId: string;
    title: string;
    description: string;
    lengthSeconds: string;
    channelId: string;
    channelName: string;
    thumbnail: string;
    thumbnails: Array<{
        url: string;
        width: number;
        height: number;
    }>;
    viewCount: string;
    publishDate: string;
    uploadDate: string;
    formats: YouTubeVideoFormat[];
    category: string;
    keywords?: string[];
    isLiveContent: boolean;
    averageRating?: number;
}

export interface YouTubeDownloadOptions {
    quality?: 'highest' | 'lowest' | 'highestaudio' | 'lowestaudio';
    filter?: 'audioandvideo' | 'videoonly' | 'audioonly';
    format?: string;
}

export interface YouTubeSearchResult {
    id: string;
    title: string;
    description: string;
    thumbnail: string;
    channelName: string;
    channelId: string;
    duration: string;
    viewCount: string;
    publishedTime: string;
    url: string;
}

export interface YouTubePlaylistInfo {
    id: string;
    title: string;
    description: string;
    channelName: string;
    channelId: string;
    thumbnail: string;
    videoCount: number;
    videos: Array<{
        id: string;
        title: string;
        thumbnail: string;
        duration: string;
        url: string;
    }>;
}

export interface YouTubeDownloadResult {
    success: boolean;
    videoInfo?: YouTubeVideoInfo;
    audioBuffer?: Buffer;
    videoPath?: string;
    error?: string;
}

Configuration

Beberapa scraper menerima opsi konfigurasi opsional saat new Scraper() atau natzScraper.all() dipanggil. Semua properties bersifat Optional.

export interface ScraperLibraryConfig {
    timeout?: number;             // Default Axios network timeout
    userAgent?: string;           // Global custom UA
    proxy?: string;               // Gunakan config HttpsProxyAgent
    
    // Spotify Custom Credentials (Optional) jika Anda punya Dev Token sendiri
    clientId?: "YOUR_ID",
    clientSecret?: "YOUR_SECRET"
    
    // IG Sesion Default
    cookie?: "sessionid=XXXX;",   
}

const scrap = natzScraper.all({ timeout: 60000 });

Troubleshooting & FAQ

1. Error ISP Blocking (Booru Sites & Certain Domains)

Situs seperti Gelbooru, Rule34.xxx, Danbooru, dll seringkali diblokir oleh provider internet Indonesia (Internet Positif / DPI Block).

• Gejala: Error ECONNRESET, ETIMEDOUT, atau Axios Network Error 503.
• Solusi: Gunakan VPN global di host kalian, gunakan safebooru (metode ini hampir tidak pernah dicekal di IP rumahan), atau forward axios kalian menggunakan custom HttpsAgent (Proxy HTTP).

2. HTTP 403 / 429 Too Many Requests

Situs seperti (TikTok/Instagram) memiliki proteksi Rate-Limit dan Human CAPTCHA yang sangat ketat. Library ini sudah berusaha mem-bypass menggunakan teknik Cookie, Header manipulation Origin, Referer, dll untuk bertindak seperti Chrome tulen. Namun tetap bisa gagal / mendadak blok jika terdeteksi aneh. Gunakan delay request antar-scrapes (await sleep(5000)) pada environment multi-thread.

3. npm error invalid package name

Pastikan nama package saat install benar: scraper-wrapper (huruf kecil).

Developer Guide (Contribution Rules)

Bagi kamu yang mau berkontribusi mengembangkan scraper-wrapper, silakan forking repo ini! Tapi, WAJIB membaca arsitektur kode file ini sebelum menulis satu baris kode pun! Codebase ini memiliki 3-layer architecture design principle.

1. Gambaran Arsitektur

User / Consumer
      ↓
  NatzScraper          ← Entry point utama (src/NatzScraper.ts)
      ↓
  API Layer            ← src/api/XxxScraper.ts  (public interface)
      ↓
  Service Layer        ← src/services/xxx.service.ts  (logic scraping)
      ↓
  External HTTP / Cheerio / Axios

• NatzScraper menyediakan factory method memanggil class API.
• API class merepresentasikan layer thin wrapper — bertugas melakukan try catch dan membungkus hasil dalam ApiResponse<T>.
• Service class mengandung seluruh business/scraping logic.
• Module Types di export pada satu pintu: src/types/index.ts.

2. Dilarang Direct File Import di Eksternal Layer

Import silang folder WAJIB melalui /index.ts, dilarang merujuk file ext langsung: ✅ BENAR: import { SpotifyService } from '../services'; ❌ SALAH: import { SpotifyService } from '../services/spotify.service';

3. Check Daftar Menambahkan Scraper

Gunakan 6 tahapan flow wajib jika Anda menambah fitur:

1. Buat FooConfig & FooResult di src/types/fooTypes.ts.
2. Export interfaces tsb di src/types/index.ts.
3. Tulis class Service (contoh FooService) di src/services/foo.service.ts. Exception ditarik (thrown throw Error), jangan direturn wrap.
4. Tulis class Web APi (FooScraper) di src/api/FooScraper.ts, panggil this.service.foo() dan bungkus via try-catch menjadi object ApiResponse.
5. Export di API index.ts dan Service index.ts.
6. Tancapkan Method baru pada file master factory src/NatzScraper.ts pada section object all() dan sebagai class public function.

License & Contribution

MIT License - Dibuat dan dikembangkan oleh Natz

Copyright © 2026. All rights reserved.

Peringatan penting: Gunakan Scraper Library ini untuk hal legal akademik/non-commercial saja, semua penyalahgunaan hak kekayaan intelektual (Download & Retransmission media berhak cipta) di luar tanggung jawab Maintainer.

Special Thanks To:

Special thanks to my friend who helped verify and improve the scraping logic for image boards! Your help made scraping much easier and less painful.

• wahyu apriliansyah