@lytics/playwright-annotations

Generic annotation framework for Playwright tests + Contentstack/Lytics conventions

What This Package Provides

For Everyone 🌍

A generic annotation framework that any team can use:

• ✅ Type-safe annotation helpers
• ✅ Flexible validation framework
• ✅ Works with any annotation schema
• ✅ ESLint plugin (coming soon)

For Contentstack/Lytics Teams 🏢

Pre-built conventions for journey-driven testing:

• ✅ testSuiteName / journeyId / testCaseId pattern
• ✅ Validation rules for naming hierarchy
• ✅ Convenience helpers

You can use our pattern or define your own!

Installation

npm install @lytics/playwright-annotations
# or
pnpm add @lytics/playwright-annotations
# or
yarn add @lytics/playwright-annotations

Quick Start

Option A: Use Contentstack Conventions (Recommended)

import { test as base } from "@playwright/test";
import { pushSuiteAnnotation, pushTestAnnotations } from "@lytics/playwright-annotations";

test.describe("Account Security - View Tokens @smoke", () => {
  test.beforeEach(async ({}, testInfo) => {
    pushSuiteAnnotation(testInfo, "ACCOUNT_SECURITY");
  });

  test("allows user to view access tokens", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
      testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
    });

    // Your test implementation...
  });
});

Option B: Define Your Own Schema

import { pushAnnotations, createValidator } from "@lytics/playwright-annotations";
import type { TestAnnotations } from "@lytics/playwright-annotations";

// 1. Define your annotation schema
interface MyTeamAnnotations extends TestAnnotations {
  featureArea: string;
  ticketId: string;
  priority: 'high' | 'medium' | 'low';
}

// 2. Create a validator for your schema
const validateMyAnnotations = createValidator<MyTeamAnnotations>({
  rules: [
    (annotations) => ({
      valid: Boolean(annotations.featureArea),
      errors: annotations.featureArea ? [] : ['featureArea is required'],
      warnings: [],
    }),
    (annotations) => ({
      valid: /^[A-Z]+-\d+$/.test(annotations.ticketId),
      errors: /^[A-Z]+-\d+$/.test(annotations.ticketId) 
        ? [] 
        : ['ticketId must match pattern: PROJECT-123'],
      warnings: [],
    }),
  ]
});

// 3. Use in your tests
test("my feature", async ({}, testInfo) => {
  const annotations: MyTeamAnnotations = {
    featureArea: 'auth',
    ticketId: 'JIRA-123',
    priority: 'high'
  };

  const validation = validateMyAnnotations(annotations);
  if (!validation.valid) {
    throw new Error(`Invalid annotations: ${validation.errors.join(', ')}`);
  }

  pushAnnotations(testInfo, annotations);

  // Your test implementation...
});

Contentstack Conventions (Reference Implementation)

The Three Required Annotations

| Annotation | Scope | Purpose | Example | |-----------|-------|---------|---------| | testSuiteName | Suite (1:N journeys) | Organizational grouping | "ACCOUNT_SECURITY" | | journeyId | Journey (1:N test cases) | Unique user flow | "ACCOUNT_SECURITY_VIEW-TOKENS" | | testCaseId | Test Case (unique) | Specific scenario | "ACCOUNT_SECURITY_VIEW-TOKENS_VALID" |

Hierarchy

ACCOUNT_SECURITY (testSuiteName)
  ├── ACCOUNT_SECURITY_VIEW-TOKENS (journeyId)
  │   ├── ACCOUNT_SECURITY_VIEW-TOKENS_VALID (testCaseId)
  │   └── ACCOUNT_SECURITY_VIEW-TOKENS_EMPTY (testCaseId)
  └── ACCOUNT_SECURITY_CREATE-TOKEN (journeyId)
      └── ACCOUNT_SECURITY_CREATE-TOKEN_VALID (testCaseId)

Validation Rules

1. ✅ Required: All three annotations must be present
2. ✅ Hierarchy: testCaseId must start with journeyId (error)
3. ✅ Uniqueness: testCaseId cannot equal journeyId (must add suffix)
4. ⚠️ Recommendation: journeyId should start with testSuiteName (warning)

Flexible Delimiters

Hyphens and underscores are normalized for comparison:

// All valid:
testSuiteName: "HOME_DASHBOARD"
journeyId: "HOME-DASHBOARD_RECIPE"  // Hyphen variant OK
testCaseId: "HOME-DASHBOARD_RECIPE_VALID"

API Reference

Generic API

Types

interface TestAnnotations {
  [key: string]: string | string[] | undefined;
}

Functions

pushAnnotations<T>(testInfo, annotations)

Push any annotations to a test (generic).

pushAnnotations(testInfo, {
  myField: 'value',
  myOtherField: 'another value'
});

getAnnotations<T>(test)

Extract annotations from a test case (generic).

const annotations = getAnnotations<MyAnnotations>(test);

createValidator<T>(config)

Factory for creating custom validators.

const validator = createValidator<MyAnnotations>({
  rules: [
    (annotations) => ({
      valid: Boolean(annotations.requiredField),
      errors: annotations.requiredField ? [] : ['requiredField is required'],
      warnings: [],
    }),
  ]
});

Contentstack-Specific API

Types

interface ContentstackAnnotations {
  testSuiteName: string;
  journeyId: string;
  testCaseId: string;
  tags?: string[];
}

Functions

pushSuiteAnnotation(testInfo, testSuiteName)

Set suite annotation (use in beforeEach).

test.beforeEach(async ({}, testInfo) => {
  pushSuiteAnnotation(testInfo, "ACCOUNT_SECURITY");
});

pushTestAnnotations(testInfo, { journeyId, testCaseId })

Set test annotations with validation.

test("my test", async ({}, testInfo) => {
  pushTestAnnotations(testInfo, {
    journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
    testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
  });
});

pushContentstackAnnotations(testInfo, annotations)

Set all annotations at once.

pushContentstackAnnotations(testInfo, {
  testSuiteName: "ACCOUNT_SECURITY",
  journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
  testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
});

getContentstackAnnotations(test)

Extract Contentstack annotations from a test.

const annotations = getContentstackAnnotations(test);
// Returns: { testSuiteName, journeyId, testCaseId } | null

validateContentstackAnnotations(annotations)

Validate Contentstack annotation format.

const result = validateContentstackAnnotations({
  testSuiteName: "ACCOUNT_SECURITY",
  journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
  testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
});

if (!result.valid) {
  console.error(result.errors);
}

Why Annotations?

Annotations enable:

• 📊 Observability - Centralized dashboards showing test health
• 🔗 Traceability - Link tests to user journeys and requirements
• 📈 Trend Analysis - Track test stability over time
• 🎯 Coverage Tracking - Identify gaps in test coverage
• 🔍 Querying - Find all tests for a journey or suite

Integration with Other Packages

With Reporter

Annotations are consumed by @lytics/playwright-reporter:

import { CoreReporter } from '@lytics/playwright-reporter';
import { FirestoreAdapter } from '@lytics/playwright-adapters/firestore';

export default new CoreReporter({
  adapters: [
    new FirestoreAdapter({
      projectId: 'my-project',
      // Adapter uses annotations for querying and grouping
    }),
  ],
});

With Journey Tools

Journey metadata drives test generation:

import { TestGenerator } from '@lytics/playwright-journey';

const generator = new TestGenerator({
  // Generates tests with proper annotations from journey metadata
});

Examples

Example 1: Basic Contentstack Usage

import { test } from "@playwright/test";
import { pushSuiteAnnotation, pushTestAnnotations } from "@lytics/playwright-annotations";

test.describe("User Management @smoke", () => {
  test.beforeEach(async ({}, testInfo) => {
    pushSuiteAnnotation(testInfo, "USER-MANAGEMENT");
  });

  test("admin can create user", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "USER-MANAGEMENT_CREATE-USER",
      testCaseId: "USER-MANAGEMENT_CREATE-USER_VALID",
    });

    // Test implementation...
  });

  test("validates required fields", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "USER-MANAGEMENT_CREATE-USER",
      testCaseId: "USER-MANAGEMENT_CREATE-USER_INVALID",
    });

    // Test implementation...
  });
});

Example 2: Custom Schema

import { pushAnnotations, createValidator } from "@lytics/playwright-annotations";

interface MyAnnotations {
  epic: string;
  story: string;
  severity: 'blocker' | 'critical' | 'major' | 'minor';
}

const validateMyAnnotations = createValidator<MyAnnotations>({
  rules: [
    (a) => ({
      valid: Boolean(a.epic && a.story),
      errors: !a.epic || !a.story ? ['epic and story are required'] : [],
      warnings: [],
    }),
  ]
});

test("my test", async ({}, testInfo) => {
  const annotations: MyAnnotations = {
    epic: 'AUTH-2023',
    story: 'AUTH-2023-01',
    severity: 'critical'
  };

  const result = validateMyAnnotations(annotations);
  if (!result.valid) {
    throw new Error(result.errors.join(', '));
  }

  pushAnnotations(testInfo, annotations);
});

Contributing

See CONTRIBUTING.md in the repository root.

License

MIT

Related Packages

• @lytics/playwright-reporter - Adapter-based reporter
• @lytics/playwright-adapters - Storage adapters
• @lytics/playwright-journey - Journey-driven testing (internal)