Sungen - AI-Native E2E Test Generator

Version: 3.2.0 (Unreleased) Priority-Based E2E Test Generation with Deterministic Selector Mapping

🎯 What is Sungen?

Sungen is a deterministic-first E2E test automation framework that generates Playwright tests from Gherkin features using a priority-based approach:

1. Strict UI Scanning - Filters only interactive elements (input, button, a, select, textarea)
2. Priority-Based Selector Mapping:
   • 1️⃣ Direct ID Match - Exact data-testid matching (instant, free, deterministic)
   • 2️⃣ Accessibility Match - Semantic role + aria-label matching
   • 3️⃣ AI Fallback - Claude Sonnet as last resort (not primary solution)
3. Auto-Tagging Tool - Inject stable data-testid into source code proactively
4. Authentication Support - Built-in Playwright storage states via @auth:{role} tags
5. Test Generation - Create production-ready Playwright test code

Key Achievement: 90%+ Direct ID match rate with minimal AI dependency!

🚀 Quick Start

Installation

npm install -g @sun-asterisk/sungen@latest

Or use locally in your project:

npm install --save-dev @sun-asterisk/sungen

Setup

1. Initialize your Sungen project:

sungen init

This creates:

• qa/screens/ - Screen definitions directory (use sungen add --screen to create)
• spec/generated/ - Generated test code output directory
• playwright.config.ts - Playwright configuration
• .gitignore - Updated with Sungen patterns

Note: Use sungen add --screen <name> to create screen definitions with proper structure.

2. Create your first screen definition:

sungen add --screen login --path /auth/login

This creates a screen definition with:

• qa/screens/login/features/login.feature - Gherkin scenarios
• qa/screens/login/selectors/login.yaml - Element selectors
• qa/screens/login/test-data/login.yaml - Test data variables

3. Edit the generated feature file and run the pipeline:

Usage

# Create a new screen definition
sungen add --screen login --path /auth/login

# Step by step:
sungen map --screen login          # 2. Map selectors with AI
sungen generate --screen login     # 3. Generate Playwright tests

# Clear cache
sungen cache-clear

📁 Project Structure

your-project/
├── app/                          # Your React source
│   ├── auth/
│   │   └── page.tsx
│   └── chat/
│       └── page.tsx
│
├── qa/                           # Input artifacts
│   └── screens/                  # Screen definitions (NEW structure)
│       └── auth/
│           ├── features/         # Gherkin scenarios
│           │   └── login.feature
│           │   └── logout.feature
│           ├── selectors/        # Element mappings
│           │   └── login.yaml
│           │   └── logout.yaml
│           └── test-data/        # Test variables
│               └── login.yaml
│               └── logout.yaml
│
├── specs/                        # Output artifacts (sibling to qa/)
│   ├── .auth/                    # Authentication storage states (gitignored)
│   │   ├── .gitignore
│   │   ├── admin.json
│   │   └── user.json
│   ├── auth.setup.ts             # Playwright auth setup (auto-generated)
│   └── generated/                # Generated Playwright tests
│       └── auth
│           ├── login.spec.ts
│           └── logout.spec.ts
│
└── sungen.config.yaml            # Framework configuration

Key Changes in v3.2:

• Organized by screens: Each screen has its own directory
• Self-contained: Features, selectors, and test data grouped together
• Use sungen add --screen <name> to create new screen definitions

🎨 Gherkin Syntax

Sungen uses a simple, natural language grammar for Gherkin:

Pattern: User + action + [element] + with {{data}}

Example Feature File

Feature: Login Screen

  As a user
  I want to interact with the Login screen
  So that I can access my account
  Path: /auth/login

  @high
  Scenario: User logs in successfully
    Given User open page
    When User fill [Email] field with {{valid_email}}
    And User fill [Password] field with {{valid_password}}
    And User click [Login] button
    Then User see [Welcome message !] text with {{success_message}}

Grammar Rules

| Component | Syntax | Example | |-----------|--------|---------| | Actor | User | User open page | | Action | click, fill, see | User click [button] | | Element | [element name] | [Email] field, [submit] button | | Data | {{variable}} | {{valid_email}}, {{success.message}} | | Path | Path: /route | Path: /auth/login |

Common Actions

# Navigation
Given User open page

# Input
When User fill [element] field with {{data}}

# Click
When User click [element] button

# Verification
Then User see [element] text with {{expected_text}}

Test Data

Define variables in test-data/<screen>.yaml:

# test-data/login.yaml
valid_email: "[email protected]"
valid_password: "SecurePassword123"
success:
  message: "Welcome back!"

Authentication with Storage States ⭐ NEW

Sungen supports Playwright's authentication storage states via @auth:{role} tags, allowing you to reuse authenticated browser contexts across tests.

Basic Usage

Tag your features or scenarios:

@auth:admin
Feature: User Management
  
  Scenario: Delete user
    Given User navigate to "/users"
    When User click [delete-button]
    Then User see [success-message]

@auth:user
Feature: Profile Settings
  
  Scenario: Update email
    Given User navigate to "/profile"
    When User fill [email] field with {{new_email}}

Generated Playwright code automatically includes storage state:

// user-management.spec.ts
test.use({ storageState: 'specs/.auth/admin.json' });

test('Delete user', async ({ page }) => {
  // Already authenticated as admin!
  await page.goto('/users');
  // ...
});

Tag Inheritance

Tags can be applied at feature level (inherited by all scenarios) or scenario level (overrides feature):

@auth:admin
Feature: Admin Dashboard
  
  Scenario: View analytics
    # Inherits @auth:admin from feature
    
  @auth:moderator
  Scenario: Review flagged content
    # Overrides with @auth:moderator
    
  @no-auth
  Scenario: View public help page
    # Explicitly disables authentication

Precedence: Scenario > Feature > None

Setting Up Authentication

On first sungen generate with @auth tags, an auth.setup.ts scaffold is auto-generated:

sungen generate --screen admin

# Output:
✓ Generated specs/auth.setup.ts - Update TODOs before running tests
✓ Created specs/.auth/ directory for storage states
  Detected roles: admin, user, moderator

Customize the generated setup file:

// specs/auth.setup.ts
import { test as setup } from '@playwright/test';

const authFile = (role: string) => `specs/.auth/${role}.json`;

setup('authenticate as admin', async ({ page }) => {
  // TODO: Navigate to your login page
  await page.goto('/login');
  
  // TODO: Fill in credentials for admin role
  await page.getByLabel('Email').fill('[email protected]');
  await page.getByLabel('Password').fill('admin123');
  
  // TODO: Click login button
  await page.getByRole('button', { name: 'Sign in' }).click();
  
  // TODO: Wait for authentication to complete
  await page.waitForURL('/dashboard');
  
  // Save authentication state
  await page.context().storageState({ path: authFile('admin') });
});

// Additional setup functions for other roles...

Features

• ✅ Feature-level tags - All scenarios inherit authentication
• ✅ Scenario-level tags - Override feature authentication
• ✅ @no-auth tag - Explicitly disable inherited authentication
• ✅ Auto-generation - Setup scaffold created on first use
• ✅ Multiple roles - Support unlimited authentication roles
• ✅ Storage reuse - Fast tests by reusing authenticated contexts

Notes

• Storage state files (specs/.auth/*.json) are gitignored by default
• Setup runs before tests via Playwright's global setup
• New roles detected after initial setup trigger a warning with manual update instructions

Element Selectors

Map elements in selectors/<screen>.yaml:

# selectors/login.yaml
email-input:
  selector: ""
  type: "testid"
  
submit-button:
  selector: ""
  type: "role"
  value: "button"

🏗️ Architecture

📖 CLI Commands

sungen init

Initialize Sungen project structure

sungen init

Creates:

• qa/screens/ directory for screen definitions
• spec/generated/ directory for generated tests
• playwright.config.ts if it doesn't exist
• Updates .gitignore with Sungen patterns

sungen add --screen ⭐ NEW

Create a new screen definition with scaffolded files. Supports multiple test scenarios per screen.

sungen add --screen <name> [options]

Options:
  --screen <name>        Screen name (required)
  -p, --path <path>      Screen route path - also controls filename
  -d, --description      Screen description

Examples:
  # Create screen with default file
  sungen add --screen login
  
  # Create screen with custom path/filename
  sungen add --screen login --path /login-valid --description "Valid login"
  
  # Add additional test scenarios to existing screen
  sungen add --screen login --path /login-invalid
  sungen add --screen login --path /login-mfa

What it does:

• First call: Creates qa/screens/<name>/ directory structure
• Subsequent calls: Adds additional feature files to existing screen
• Generates features/<filename>.feature with Gherkin template
• Generates selectors/<filename>.yaml with selector structure
• Generates test-data/<filename>.yaml for test variables
• Path option: Controls both filename and Path metadata in feature file

Multi-file workflow (organizing test scenarios):

# Create base screen
sungen add --screen login

# Add valid login scenario
sungen add --screen login --path /login-valid

# Add invalid credentials scenario
sungen add --screen login --path /login-invalid

# Add 2FA scenario
sungen add --screen login --path /login-mfa

Result:

qa/screens/login/
  ├── features/
  │   ├── login.feature
  │   ├── login-valid.feature
  │   ├── login-invalid.feature
  │   └── login-mfa.feature
  ├── selectors/
  │   ├── login.yaml
  │   ├── login-valid.yaml
  │   ├── login-invalid.yaml
  │   └── login-mfa.yaml
  └── test-data/
      ├── login.yaml
      ├── login-valid.yaml
      ├── login-invalid.yaml
      └── login-mfa.yaml

Backward compatible: Default behavior (no --path) creates single file named after screen.

sungen map

Map Gherkin references to actual selectors using AI

sungen map --screen <screen-name> [options]

Options:
  --screen <name>   Screen name to map (required)
  --all             Map all screens
  --force           Force regeneration, bypass cache, overwrite existing files
  --verbose         Detailed output

Output: Processes each feature file individually, generating separate YAML files for each:

Mapping screen: login

✓ Mapped login.feature
   → selectors/login.yaml (5 elements)
   → test-data/login.yaml (3 variables)
   
✓ Mapped login-valid.feature
   → selectors/login-valid.yaml (3 elements)
   → test-data/login-valid.yaml (2 variables)

Next step: sungen generate --screen login

Multi-file support: Each .feature file generates matching .yaml files, maintaining clean separation between test scenarios.

sungen generate

Generate Playwright test code from features

sungen generate [options]

Options:
  --ai-mapper          Enable AI fallback for unknown Gherkin patterns
  --framework <name>   Test framework (playwright, appium, integration) - default: playwright
  -s, --screen <name>  Filter generation to specific screen (recommended)
  -f, --force          Force regeneration, bypass cache

Examples:
  sungen generate --screen login     # Generate tests for login screen
  sungen generate -s login --force   # Force regenerate login screen tests
  sungen generate                    # Generate all tests (legacy)

Output: Shows per-file results with step counts:

Generating tests: login

✓ Generated login.spec.ts
   → spec/generated/login/login.spec.ts (8 steps)

✓ Generated login-valid.spec.ts
   → spec/generated/login/login-valid.spec.ts (5 steps)

✓ Generated login-invalid.spec.ts
   → spec/generated/login/login-invalid.spec.ts (3 steps)

Next step: npx playwright test

Selector Merging: Supports optional base + feature-specific selector files:

• Base file: selectors/<screen>.yaml - Shared selectors for all scenarios
• Feature file: selectors/<feature-name>.yaml - Scenario-specific selectors (required)
• Feature-specific selectors override base selectors when keys conflict

Validation: Checks for missing selector files before generation and suggests running map command if needed.

sungen full

Run complete pipeline (discover + map + generate)

sungen full [options]

Options:
  --screen <name>   Process specific screen (optional)
  --depth <num>     Max component depth
  --force           Force regeneration
  --verbose         Detailed output

sungen cache-clear

Clear all cached data

sungen cache-clear

sungen makeauth ⭐ NEW

Generate authentication state for login sessions (supports SSO and manual login)

sungen makeauth <name> [options]

Options:
  -u, --url <url>          Base URL of the application (auto-detected from playwright.config.ts)
  -p, --path <path>        Login page path (default: /login)
  -o, --output <dir>       Output directory (default: specs/.auth)
  -t, --timeout <ms>       Overall timeout in milliseconds (default: 180000)
  --nav-timeout <ms>       Navigation timeout in milliseconds (default: 180000)
  --stability-wait <ms>    URL stability wait in milliseconds (default: 5000)
  --headless               Run browser in headless mode
  --verify                 Verify existing auth state is still valid
  --list                   List all existing auth states
  --export                 Export auth state as base64 for CI

Examples:
  sungen makeauth user               # Create auth state for 'user' role
  sungen makeauth admin --url http://localhost:3000
  sungen makeauth admin --path /auth/signin
  sungen makeauth admin --list       # List all auth states
  sungen makeauth admin --verify     # Check if auth is still valid

What it does:

• Opens a browser window for manual login (including SSO providers like Google, GitHub, etc.)
• Saves authentication state (cookies, localStorage, sessionStorage) to specs/.auth/<name>.json
• Can be used in Playwright tests to skip login process and maintain sessions
• Supports multiple auth roles (admin, user, guest, etc.)

Using auth state in tests:

// playwright.config.ts
export default defineConfig({
  projects: [
    {
      name: 'authenticated',
      use: {
        storageState: 'specs/.auth/user.json',
      },
    },
  ],
});

See docs/makeauth.md for complete documentation.

sungen auto-tag ⭐ NEW

Auto-inject stable data-testid attributes into source code

sungen auto-tag [options]

Options:
  --screen <name>   Tag specific screen (optional, defaults to all)
  --dry-run         Preview changes without modifying files
  --force           Overwrite existing data-testid (use with caution)
  --verbose         Show detailed changes

Examples:
  sungen auto-tag --screen login --dry-run   # Preview changes
  sungen auto-tag --screen login             # Apply changes
  sungen auto-tag                            # Tag all screens

What it does:

• Scans React/Vue/HTML source code for interactive elements
• Injects data-testid="screen-element" attributes
• Idempotent (skips if data-testid already exists)
• Preserves code formatting (Prettier/ESLint compatible)

Before:

<input type="email" name="email" />
<button onClick={handleSubmit}>Login</button>

After:

<input type="email" name="email" data-testid="login-email-input" />
<button onClick={handleSubmit} data-testid="login-submit-btn">Login</button>

🎓 Examples

See the /examples folder for complete working examples:

• ai-chat - Chat application with authentication
• More examples coming soon!

📚 Documentation

Getting Started

• Documentation Index - Start here for all documentation
• Implementation Guide ⭐ Core reference
  • Kim chỉ nam (North Star Principles)
  • Architecture overview
  • Implementation roadmap
  • Best practices

Core Guides

• System Overview - High-level architecture
• Gherkin Standards - Gherkin syntax and conventions
• Selector Override Guide - How to fix AI mapping errors
• Debugging Guide - Troubleshooting common issues

Recent Updates

• Improvement Summary - Latest fixes and improvements (Dec 2025)
  • Test results: 1 → 6 passed (+500%)
  • Selector override mechanism
  • Heuristic mapper implementation
  • Complete test-data structure

Quick Links

• For QA: Start with Gherkin Standards
• For Developers: Read Implementation Guide
• Fix AI Errors: Use Selector Override

🤝 Contributing

Contributions are welcome! Please read our contributing guidelines and follow the Kim chỉ nam (North Star Principles) in the Implementation Guide.

📝 License

MIT License - see LICENSE file for details

🙏 Credits

• Built with Claude Sonnet 4.5
• Uses @anthropic-ai/sdk
• Powered by Playwright
• Gherkin parsing by @cucumber/gherkin

Made with ❤️ by Bach Ngoc Hoai