react-native-agentkit

Convert React Native app flows to CLI for AI agent automation.

What it does

react-native-agentkit lets AI agents and CLI tools control your React Native app programmatically. Wrap your app with <AgentKitProvider>, and the library automatically discovers all interactive elements (buttons, inputs, scrollviews, switches) and exposes them through a command interface.

Installation

npm install react-native-agentkit
# or
yarn add react-native-agentkit

Quick Start

1. Wrap your app

import { AgentKitProvider } from 'react-native-agentkit';

export default function App() {
  return (
    <AgentKitProvider debug={__DEV__} relayUrl="ws://localhost:8347">
      <YourApp />
    </AgentKitProvider>
  );
}

That's it! The library automatically:

• Scans your component tree for interactive elements
• Starts a command bridge on the device
• Accepts commands from CLI tools

2. Start the relay server

The CLI communicates with the app through a lightweight WebSocket relay:

npx react-native-agentkit-relay
# Listening on port 8347

3. Control via CLI

# Interactive REPL
npx react-native-agentkit connect --relay=ws://localhost:8347

# Execute single commands
npx react-native-agentkit exec list --relay=ws://localhost:8347
npx react-native-agentkit exec tap submit-btn --relay=ws://localhost:8347
npx react-native-agentkit exec type email-input "[email protected]" --relay=ws://localhost:8347

# Pipe mode for AI agents
echo '{"cmd":"list"}' | npx react-native-agentkit pipe --relay=ws://localhost:8347

Commands

| Command | Description | Example | | ---------- | ----------------------------- | ----------------------------------------------------------------- | | list | List all interactive elements | {"cmd": "list"} | | tap | Tap an element | {"cmd": "tap", "target": "submit-btn"} | | toggle | Toggle a switch/checkbox | {"cmd": "toggle", "target": "agree-switch"} | | setValue | Set slider/numeric value | {"cmd": "setValue", "target": "brightness", "value": 75} | | type | Type text into input | {"cmd": "type", "target": "email", "text": "[email protected]"} | | clear | Clear input text | {"cmd": "clear", "target": "email"} | | scroll | Scroll a view | {"cmd": "scroll", "target": "main-scroll", "direction": "down"} | | swipe | Swipe an element | {"cmd": "swipe", "target": "item-1", "direction": "left"} | | read | Read element value | {"cmd": "read", "target": "header"} | | state | Full screen state snapshot | {"cmd": "state"} | | find | Search elements by text | {"cmd": "find", "filterText": "login"} | | back | Navigate back | {"cmd": "back"} | | wait | Wait for element to appear | {"cmd": "wait", "target": "success-msg", "timeout": 5000} | | ping | Check connection | {"cmd": "ping"} |

Response Format

All commands return structured JSON:

{
  "success": true,
  "command": "tap",
  "target": "submit-btn",
  "result": { "elementTapped": true },
  "screenState": { "elements": [...], "count": 5 },
  "timestamp": 1710000000000
}

Props

| Prop | Type | Default | Description | | ----------------------- | --------- | ----------- | ---------------------------------------- | | port | number | 8347 | Server port | | debug | boolean | false | Enable debug logging | | scanInterval | number | 1000 | Element scan interval (ms) | | includeNonInteractive | boolean | false | Include non-interactive elements | | includeScreenState | boolean | true | Include screen state in responses | | enabled | boolean | true | Enable/disable the bridge | | relayUrl | string | — | Cloud relay WebSocket URL for production | | channelId | string | "default" | Channel ID for relay pairing | | channelSecret | string | — | Shared secret for relay authentication |

Production Usage (Cloud Relay)

In production builds, Metro isn't available. The cloud relay enables AI agents to control the app over the network.

Architecture

┌──────────┐        ┌───────────────┐        ┌──────────────┐
│  CLI /   │◄──WS──►│  Relay Server │◄──WS──►│   RN App     │
│ AI Agent │        │  (Node.js)    │        │ (production) │
└──────────┘        └───────────────┘        └──────────────┘

1. Start the relay server

# Default port 8347
npx react-native-agentkit-relay

# Custom port
npx react-native-agentkit-relay --port=9000

2. Configure the app

<AgentKitProvider
  relayUrl="ws://your-relay-host:8347"
  channelId="my-app"
  channelSecret="your-shared-secret"
>
  <YourApp />
</AgentKitProvider>

3. Connect via CLI

# Interactive REPL
npx react-native-agentkit connect --relay=ws://your-relay-host:8347 --channel=my-app --secret=your-shared-secret

# Execute a command
npx react-native-agentkit exec list --relay=ws://your-relay-host:8347 --channel=my-app --secret=your-shared-secret

# Pipe mode (for AI agents)
echo '{"cmd":"list"}' | npx react-native-agentkit pipe --relay=ws://your-relay-host:8347 --secret=your-shared-secret

Authentication

The relay supports shared-secret authentication to prevent unauthorized access:

• Dev mode (default): Auth is optional. If no secret is set, anyone can connect to the channel.
• Production: Set channelSecret in the app and --secret in the CLI. The first client to join a channel with a secret "locks" that channel — subsequent clients must present matching secrets.
• Enforced auth: Start the relay with --require-auth to reject all unauthenticated connections.

# Start relay with enforced auth
npx react-native-agentkit-relay --require-auth

💡 Tip: Use a unique channelId + channelSecret per app/device for isolation and security.

Hooks

import {
  useAgentKit, // Full context access
  useAgentKitElement, // Manually register an element
  useAgentKitRescan, // Trigger a re-scan
  useAgentKitStatus, // Monitor bridge status
} from 'react-native-agentkit';

Manual Element Registration

For custom components that aren't auto-discovered:

function CustomButton({ onPress, label }) {
  const ref = useRef(null);

  useAgentKitElement(
    'my-btn',
    {
      type: 'button',
      label,
      state: { disabled: false, selected: false },
      position: { x: 0, y: 0, width: 0, height: 0 },
      children: [],
      actions: ['tap'],
    },
    ref,
    { onPress }
  );

  return (
    <Pressable ref={ref} onPress={onPress}>
      <Text>{label}</Text>
    </Pressable>
  );
}

Element Discovery

Elements are discovered via:

1. testID prop — Always included, highest priority
2. Accessibility props — accessibilityLabel, accessibilityRole
3. Interactive handlers — onPress, onChangeText, onValueChange
4. Component type — Pressable, TextInput, Switch, ScrollView, etc.

💡 Tip: Use testID on key elements for reliable AI agent targeting.

Native Dialog Handling

AgentKit automatically intercepts native Alert.alert(), PermissionsAndroid, and react-native-permissions dialogs:

• Conditional suppression — Native dialogs are only suppressed when an agent is connected. When no agent is connected, they display normally.
• Alert buttons: alert-{title}-{button-text}
• Permission buttons: permission-{name}-grant / permission-{name}-deny / permission-{name}-block
• Buttons auto-clean up after being tapped or after a timeout

# After an action triggers Alert.alert('Confirm', '...', [{text:'Cancel'}, {text:'Delete'}])
list                              # Shows: alert-confirm-cancel, alert-confirm-delete
tap alert-confirm-delete          # Executes the Delete button's onPress handler

# Permission request (via PermissionsAndroid or react-native-permissions)
list                              # Shows: permission-camera-grant, permission-camera-deny, permission-camera-block
tap permission-camera-grant       # Grants the permission

💡 Tip: Apps using react-native-permissions get automatic interception for both iOS and Android.

AI Agent Prompts

Copy-paste these prompts to give an AI agent (Claude Code, Cursor, etc.) the ability to control your app.

Local Development

Use this when your app and relay are running locally:

You can control a React Native app using `react-native-agentkit`. The app is running
on a local device/simulator with a relay server at ws://localhost:8347.

**Read the skill file first:**
Read the file at `node_modules/react-native-agentkit/SKILL.md` for the full command
reference and workflow patterns.

**To send commands, use pipe mode:**
echo '{"cmd":"state"}' | npx react-native-agentkit pipe --relay=ws://localhost:8347

**Workflow:** Always start with `state` to see what's on screen, then use commands
like `tap`, `type`, `scroll`, `swipe`, `toggle`, `setValue`, etc. to interact.
Each response includes the updated screen state.

Now please: [describe what you want the agent to do]

Production / Remote Device

Use this when connecting to a deployed relay with authentication:

You can control a React Native app using `react-native-agentkit`. The app is connected
to a relay server.

**Read the skill file first:**
Read the file at `node_modules/react-native-agentkit/SKILL.md` for the full command
reference and workflow patterns.

**To send commands, use pipe mode with authentication:**
echo '{"cmd":"state"}' | npx react-native-agentkit pipe --relay=ws://your-relay-host:8347 --channel=your-channel --secret=your-secret

**Workflow:** Always start with `state` to see what's on screen, then use commands
like `tap`, `type`, `scroll`, `swipe`, `toggle`, `setValue`, etc. to interact.
Each response includes the updated screen state.

Now please: [describe what you want the agent to do]

Tip: The key is having the agent read SKILL.md first — it contains all command patterns, element discovery strategies, and best practices the agent needs.

Running the Example

# 1. Install dependencies
yarn install

# 2. Start the relay server (in a separate terminal)
npx react-native-agentkit-relay

# 3. Start the example app on iOS simulator
cd example
npx expo start --ios

# 4. Connect the CLI (in a separate terminal)
npx react-native-agentkit connect --relay=ws://localhost:8347

# Or execute a single command
npx react-native-agentkit exec list --relay=ws://localhost:8347

License

MIT