@ken-e/dataform-youtube

A Dataform package for processing YouTube analytics data from Google's BigQuery Data Transfer Service.

Installation

npm install @ken-e/dataform-youtube

Quick start

const youtube = require("@ken-e/dataform-youtube");

youtube({
  startDate: "2025-01-01",

  // Video and playlist title lookup tables
  titlesProject: "my-project",
  titlesDataset: "my_dataset",
  titlesTable: "stg_yt_video_titles",
  playlistTable: "stg_yt_playlist_titles",

  // YouTube data sources (one or more)
  sources: [{
    database: "my-bq-project",
    schema: "youtube_channel",
    suffix: "_"
  }],

  // Target BigQuery datasets
  target: {
    database: dataform.projectConfig.defaultDatabase,
    stagingSchema: "my_staging_dataset",
    outputSchema: "my_output_dataset"
  }
});

Output tables

Channel reports

| Table | Description | |---|---| | ytc_basic | Core video metrics: views, likes, comments, shares, watch time, subscribers | | ytc_combined | Metrics by all major dimensions (location, traffic source, device, OS) | | ytc_demographics | Age group and gender breakdowns | | ytc_demographics_views | Demographics with absolute view counts | | ytc_device_os | Device type and operating system breakdowns | | ytc_playback | Playback location breakdowns | | ytc_province | Province/state-level geographic breakdowns | | ytc_traffic_source | Traffic source breakdowns | | ytc_share_platform | Social sharing service breakdowns | | ytc_annotation | Annotation performance metrics | | ytc_cards | Card performance metrics | | ytc_end_screens | End screen element performance | | ytc_subtitles | Subtitle/caption analytics |

Playlist reports

| Table | Description | |---|---| | ytc_list_basic | Core playlist metrics | | ytc_list_combined | Playlist metrics by all dimensions | | ytc_list_device_os | Playlist device/OS breakdowns | | ytc_list_playback | Playlist playback location breakdowns | | ytc_list_province | Playlist province-level breakdowns | | ytc_list_traffic_source | Playlist traffic source breakdowns |

Data source migration (September 2025)

On September 22, 2025, Google deprecated several YouTube BQ Data Transfer table versions (a2 channel tables and a1 playlist tables) and replaced them with new versions (a3 and a2 respectively). This package handles the migration transparently by reading from both old and new tables with a date-based cutover.

Important: Google changed the view-counting methodology in the new tables. View counts after September 22, 2025 are significantly higher than before and are not directly comparable across the cutover boundary. See the BQ Data Transfer change log for details.

Documentation

See CLAUDE.md for detailed development documentation including architecture, configuration reference, and the source table migration guide.

Google documentation

• YouTube Channel Reports in BigQuery
• BigQuery Data Transfer change log