Desktop Driver Schemas

Type-safe schemas for desktop automation driver capabilities. This package provides shared schema definitions for both TypeScript and Python projects.

Features

• 🎯 Type Safety: Full TypeScript and Python type definitions
• 🔄 Auto-generated: Schemas generated from central capabilities.json
• 📦 Multi-language: Works with TypeScript, Python, and Node.js
• ✅ Validation: Built-in validation with Zod (TS) and Pydantic (Python)
• 🚀 Easy to use: Simple imports and clear API

Installation

TypeScript/Node.js

npm install @trigger_mesh/desktop-driver-schemas
# or
yarn add @trigger_mesh/desktop-driver-schemas

Python

pip install trigger-mesh-desktop-driver-schemas

Usage

TypeScript

import { ActionSchema, ClickAtAction } from '@trigger_mesh/desktop-driver-schemas';
import { z } from 'zod';

// Validate action
const action: ClickAtAction = {
  type: 'click_at',
  x: 100,
  y: 200
};

const validated = ActionSchema.parse(action);
console.log(validated); // { type: 'click_at', x: 100, y: 200 }

Python

from trigger_mesh_desktop_driver_schemas import Action, ClickAtAction, CAPABILITIES

# Validate action
action = ClickAtAction(
    type="click_at",
    x=100,
    y=200
)

print(action)  # type='click_at' x=100 y=200

# Access capabilities
print(CAPABILITIES.keys())

Raw JSON

import capabilities from '@trigger_mesh/desktop-driver-schemas/capabilities';

console.log(capabilities.focus_window);
// { description: "Focus a window by title substring", args: {...} }

Available Capabilities

The package includes schemas for 29+ desktop automation capabilities:

• Mouse Actions: click_at, double_click, right_click, drag, scroll
• Keyboard Actions: type_text, key_press, key_combination, xdo_sequence
• Window Management: focus_window, wait_for_window, launch_app
• Screenshots: screenshot, screenshot_base64, screenshot_region_base64
• Image Detection: find_image, wait_for_image, click_image
• File System: fs_list, fs_read, fs_write
• OCR: ocr_read_region, ocr_find_text, ocr_wait_text
• Accessibility: ax_wait, ax_click
• System: shell_exec, wait

Development

Prerequisites

• Node.js 18+
• Python 3.8+
• npm or yarn

Setup

# Install dependencies
npm install

# Generate schemas
npm run generate

# Build packages
npm run build

# Validate driver implementation
npm run validate

Scripts

• npm run generate - Generate all schemas from capabilities.json
• npm run build - Build TypeScript and prepare for publishing
• npm run validate - Validate driver implementation
• npm run test - Run tests

Publishing

NPM (TypeScript/Node.js)

# Login to npm
npm login

# Publish
npm publish

PyPI (Python)

# Install build tools
pip install build twine

# Build Python package
python -m build

# Upload to PyPI
twine upload dist/*

API Reference

TypeScript Exports

• ActionSchema - Zod schema for validating actions
• ResultSchema - Zod schema for validating results
• ClickAtAction, ScreenshotAction, etc. - Individual action types
• capabilities - Raw capabilities JSON

Python Exports

• Action - Pydantic base model for actions
• ClickAtAction, ScreenshotAction, etc. - Individual action models
• CAPABILITIES - Raw capabilities dictionary
• get_capability(name) - Get specific capability definition
• validate_capability(name, args) - Validate capability arguments

Contributing

1. Update capabilities.json with new capabilities
2. Run npm run generate to update schemas
3. Run npm run validate to ensure driver compatibility
4. Test with both TypeScript and Python
5. Submit a pull request

License

MIT License - see LICENSE file for details.