A/B Test Skill

One skill for the whole testing journey — brainstorm a hypothesis → set it up → get pacing updates in Slack / Gmail / Discord. Built to be simple and reliable: one happy path, few moving parts.

The flow (one journey)

1. Hypothesis — decide what to test (or get suggestions if unsure), and check it's worth testing against real account data (significance/power, not guesswork).
2. Setup — scaffold test.json + the monitor script into the client's 04-automations/.
3. Pacing — recurring updates to your channel on a schedule; interim = directional, final = verdict, then it self-disables.

It won't fire early

The skill interviews the user through a visible checklist (templates/intake-checklist.md) and only scaffolds + schedules once the intake is complete and the test passes the "does this make sense" gate (a real decision rule + a detectable effect). If it doesn't make sense, it stops and tells you why instead of setting up a half-baked test. The actual cron "fire" is the last step, after a dry-run you approve.

Try it in 10 seconds

python3 templates/monitor.py --demo

Prints a sample pacing message — no credentials needed. Then see QUICKSTART.md to install + run it for real.

Files

• QUICKSTART.md — zero-to-running in ~10 min (the monitor.py script + your account credentials).
• SKILL.md — the 3-phase flow + guardrails (start here).
• reference/
  • hypothesis.md — what makes a good hypothesis, how to suggest tests, and the significance/power check using real data.
  • setup.md — test.json schema + the default monitor pattern.
  • reporting.md — channels (Slack/Gmail/Discord), message cadence, scheduling.
• templates/
  • intake-checklist.md — the visible follow-along checklist + readiness gate (filled in during the interview; nothing fires until it's complete).
  • test.json — the test config.
  • monitor.py — the script that runs on a schedule: pull live Google Ads + Meta → compare to baseline → post → self-disable. Reports ROAS/CPA/CVR. Slack/Discord/Gmail output.

Design principle

Hypothesis-making used to be a separate skill; it's folded in here because for the user it's one process. Internally the hypothesis logic lives in its own reference file so it stays clean and reusable. Default to one pattern + one channel + one schedule — fewer levers, fewer things that break.

Tier: client-level. Run it inside a client brain, with your Google Ads + Meta credentials set up (see Setup/, or the online guides at adsrepo.com/setup).

Copyright (c) 2026 Tomas Kliment & Denis Capko / adsrepo.com. All rights reserved. See LICENSE for terms — https://www.adsrepo.com/license