webflow-engine

v1.0.0

Published

7 months ago

Declarative workflows for complex website interactions - The missing piece for AI agent automation

0High
0Medium
0Low

hal-crackbot

ai-agents browser-automation mcp workflows web-scraping steipete-style declarative-automation

WebFlow Engine 🌊

Declarative workflows for complex website interactions. The missing piece that turns AI agents from simple browser automation to sophisticated multi-step task execution.

The Problem

AI agents can navigate to websites and click buttons, but struggle with complex multi-step workflows:

"Apply to 10 jobs on different sites with customized applications"
"Find apartments, contact landlords, and schedule viewings"
"Research products, compare prices, and handle checkout processes"

WebFlow Engine solves this. Define workflows once, execute them across different sites with intelligent adaptation.

Features

🎯 Declarative YAML Workflows - Define complex interactions as readable configs
🤖 Smart Element Discovery - Find elements across different sites with fuzzy matching
🔄 Cross-Site Adaptation - Same workflow works on LinkedIn, Indeed, Monster, etc.
🧠 AI-Powered Pattern Recognition - Automatically discover interaction patterns
🔗 SessionKeeper Integration - Works with persistent browser sessions
📝 Built-in Workflows - Job applications, apartment hunting, e-commerce
🛠️ Step-by-Step Execution - Debug and control workflow execution
🎨 Custom Step Handlers - Extend with JavaScript for complex logic

Installation

npm install -g webflow-engine
# or with SessionKeeper for full AI agent stack
npm install -g sessionkeeper webflow-engine

Quick Start

1. Start the MCP Server

webflow
# Server running on stdio, ready for MCP clients

2. Configure Your AI Client

Add to your MCP configuration:

{
  "mcpServers": {
    "webflow": {
      "command": "webflow",
      "args": []
    }
  }
}

3. Use From Your AI

AI: I'll help you apply to frontend jobs. Let me run the job application workflow.

[workflow_run workflowName="job-application" sessionId="job-search" variables={
  "job_site_url": "https://linkedin.com/jobs",
  "search_query": "Frontend Developer Berlin",
  "personal_info": {
    "fullName": "John Doe",
    "email": "[email protected]"
  },
  "resume_file": "./resume.pdf"
}]

Applied to 5 matching positions with customized cover letters!

MCP Tools

`workflow_list`

List all available workflows and their descriptions.

`workflow_run`

Execute a complete workflow with variables.

{
  "workflowName": "job-application",
  "sessionId": "browser-session",
  "variables": {
    "search_query": "Frontend Developer",
    "personal_info": {
      "fullName": "Your Name",
      "email": "[email protected]"
    }
  },
  "stepByStep": false
}

`workflow_step`

Execute workflows step-by-step for debugging.

{
  "contextId": "session_workflow_12345",
  "action": "next" // next, retry, skip, abort
}

`pattern_discover`

AI-powered element pattern discovery for new sites.

{
  "sessionId": "browser-session",
  "url": "https://newjobsite.com",
  "intent": "find search box"
}

`workflow_create`

Create custom workflows from YAML definitions.

Built-in Workflows

🎯 Job Application

Automatically apply to jobs across multiple platforms:

Search for positions matching criteria
Fill application forms with personal info
Upload resume and cover letter
Handle multi-step application processes

name: job-application
description: Apply to jobs on various job sites
variables:
  search_query: "Frontend Developer"
  personal_info:
    fullName: "Your Name"
    email: "[email protected]"
  resume_file: "./resume.pdf"
steps:
  - name: navigate_to_jobs
    type: navigate
    target: "{job_site_url}"
  - name: search_positions
    type: type
    target: job_search_input
    value: "{search_query}"
  # ... more steps

🏠 Apartment Search

Find and inquire about rental properties:

Search by location and budget
Extract listing details
Send personalized inquiries
Schedule viewings

🛒 E-commerce Automation

Handle online shopping workflows:

Product research and comparison
Add items to cart
Apply coupon codes
Complete checkout process

Creating Custom Workflows

Basic Workflow Structure

name: my-workflow
description: What this workflow does
version: 1.0.0
variables:
  # Variables that can be passed in
  site_url: ""
  search_term: ""

steps:
  - name: navigate
    type: navigate
    target: "{site_url}"
    
  - name: search
    type: type
    target: search_input_pattern
    value: "{search_term}"
    
  - name: submit
    type: click
    target: search_button_pattern

patterns:
  search_input_pattern:
    selectors:
      - 'input[type="search"]'
      - 'input[placeholder*="search"]'
    text: ["search", "find"]
    
  search_button_pattern:
    selectors:
      - 'button[type="submit"]'
      - 'input[type="submit"]'
    text: ["search", "go", "find"]

Step Types

navigate - Go to a URL
click - Click an element
type - Type text into a field
select - Select from dropdown
wait - Wait for element or time
extract - Extract data from page
condition - Conditional logic
loop - Repeat steps
custom - Execute JavaScript handler

Smart Element Discovery

WebFlow uses multiple strategies to find elements:

Direct CSS Selectors - Standard CSS selector matching
Text Content Matching - Find elements by visible text
Attribute Fuzzy Matching - Match placeholder, name, id attributes
AI-Powered Scoring - ML-style heuristics for element ranking
Cross-Site Learning - Learn patterns from successful interactions

Pattern Definitions

Patterns make workflows portable across different sites:

patterns:
  login_button:
    selectors:
      - 'button[data-action="login"]'
      - '.login-button'
      - '#login'
    text: ["login", "sign in", "log in"]
    attributes:
      class: ["btn-primary", "login"]
    position: "first"  # first, last, or number
    scoring:
      visible: 5       # boost for visible elements
      interactive: 3   # boost for clickable elements

Advanced Features

Conditional Logic

- name: check_login_required
  type: condition
  condition: "login_form.length > 0"
  
- name: handle_login
  type: click
  target: login_button
  condition: "login_required"

Loops and Iteration

- name: apply_to_all_jobs
  type: loop
  condition: "job_listings.length > 0"
  maxIterations: 10
  steps:
    - name: click_job
      type: click
      target: job_link
    - name: apply
      type: custom
      target: application_form

Custom Step Handlers

Extend workflows with JavaScript:

// Custom handler for complex form filling
async function fillJobApplication(context: WorkflowContext): Promise<any> {
  const { page, variables } = context;
  
  // Smart form detection and filling
  await detectAndFillForm(page, variables.personal_info);
  
  // Handle file uploads
  if (variables.resume_file) {
    await uploadResume(page, variables.resume_file);
  }
  
  return { status: 'application_completed' };
}

Integration with SessionKeeper

WebFlow works perfectly with SessionKeeper for persistent sessions:

# Start both servers
sessionkeeper &
webflow &

# Use together in AI workflows

// AI can maintain login sessions and execute complex workflows
await sessionKeeper.navigate("gmail-session", "https://gmail.com");
await sessionKeeper.login("gmail-session", credentials);

await webFlow.run("email-management", "gmail-session", {
  action: "send_follow_ups",
  template: "conference_follow_up"
});

Use Cases

🎯 Automated Job Applications

"Apply to 20 frontend developer positions in Berlin, customizing cover letters based on company research"

Search multiple job boards
Extract job requirements
Generate customized applications
Track application status

🏠 Apartment Hunting Assistant

"Find apartments under €1200 in Kreuzberg, send inquiries, and schedule viewings for this weekend"

Search rental sites
Filter by criteria
Send personalized inquiries
Coordinate viewing schedules

💰 Investment Research

"Research these 10 stocks, compare P/E ratios, and create a summary report"

Navigate financial sites
Extract financial data
Compare metrics
Generate reports

🛒 Smart Shopping

"Find the best deal on a MacBook Pro, check reviews, and complete purchase with best payment method"

Price comparison across sites
Review aggregation
Cart optimization
Payment processing

Error Handling & Recovery

WebFlow includes robust error handling:

Automatic Retries - Configurable retry logic for failed steps
Fallback Selectors - Multiple ways to find the same element
Graceful Degradation - Continue workflow even if some steps fail
Visual Debugging - Screenshots and page state capture
Session Recovery - Resume workflows after interruptions

Architecture

┌─────────────┐    ┌──────────────┐    ┌─────────────┐
│     AI      │───▶│   WebFlow    │───▶│  Website    │
│   Agent     │    │   Engine     │    │ (Dynamic)   │
│            │◀───│              │◀───│             │
└─────────────┘    └──────────────┘    └─────────────┘
                           │
                           ▼
                   ┌──────────────┐
                   │  Workflow    │
                   │  Patterns    │
                   │  (YAML)      │
                   └──────────────┘

Configuration

Environment Variables

WEBFLOW_HEADLESS - Run headless browser (default: true)
WEBFLOW_DATA_DIR - Storage directory (default: ./workflow-data)
WEBFLOW_WORKFLOWS_DIR - Workflow definitions (default: ./workflows)
WEBFLOW_TIMEOUT - Default timeout in ms (default: 30000)

Advanced Configuration

# webflow.config.yaml
browser:
  headless: true
  timeout: 30000
  viewport:
    width: 1920
    height: 1080

patterns:
  discovery:
    fuzzy_threshold: 0.7
    max_candidates: 10
    
workflows:
  retry_attempts: 3
  step_delay: 1000
  auto_screenshot: true

Development

git clone https://github.com/hal-crackbot/webflow-engine
cd webflow-engine
npm install
npm run dev

Testing Workflows

# Test workflow syntax
webflow validate ./workflows/job-application.yaml

# Debug step-by-step
webflow run job-application --step-by-step --debug

# Pattern discovery
webflow discover https://newsite.com --intent "find login form"

Roadmap

[ ] Visual Workflow Designer - GUI for creating workflows
[ ] Machine Learning Patterns - Auto-improve element discovery
[ ] Workflow Marketplace - Share and discover workflows
[ ] Multi-Site Coordination - Workflows spanning multiple sites
[ ] A/B Testing - Test different workflow variations
[ ] Performance Analytics - Track workflow success rates

Contributing

Built following the steipete philosophy: focused infrastructure that solves real problems for AI agents.

Fork the repository
Create a feature branch
Add tests for new functionality
Submit PR with clear description

License

Why This Matters

AI agents need to perform complex, multi-step tasks on the web, not just simple click-and-type operations. WebFlow bridges the gap between basic browser automation and intelligent task execution.

Before WebFlow: "I can help you navigate to LinkedIn, but you'll need to manually apply to each job."

After WebFlow: "I've applied to 15 relevant positions with customized cover letters. Here's the summary of applications sent."

This is the difference between a browser remote control and a genuine AI assistant.

Building the future of AI agent capabilities. 🚀