Handit CLI

Setting up your autonomous engineer

Stop being your AI's on-call engineer. Your autonomous engineer monitors your AI 24/7, detects issues, creates fixes, and ships them as pull requests—automatically.

The Problem: You're Your AI's On-Call Engineer

2am failures wake you up. Your AI starts giving bad responses, customers complain, and you're debugging blind. Did the model change? Is a tool broken? Is there a logic error? Without visibility, you're playing whack-a-mole with quality issues.

Manual fixes don't scale. You spot-check responses, find problems, and wonder about the thousands of interactions you didn't see. By the time you notice issues, they've already impacted users.

Your autonomous engineer solves this by monitoring every interaction, detecting quality drops, analyzing root causes, and shipping proven fixes—all automatically.

The Solution: Your Autonomous Engineer

🔍 Monitors your AI interactions and detects quality issues
🛠️ Generates fixes and tests them against real data
📝 Creates pull requests with proven improvements

Time required: Under 5 minutes
Prerequisites: Handit.ai Account, Node.js, and admin access to your GitHub repository

Set up your autonomous engineer

Your autonomous engineer setup happens in one seamless flow through the Handit CLI. Here's what will happen:

Step 1: Install the Handit CLI

npm install -g @handit.ai/cli

Step 2: Start the Setup Process

Navigate to your AI project directory and run:

handit-cli setup

The CLI will guide you through connecting your autonomous engineer to your AI system:

🔧 Initial Connection

• Connect your Handit.ai account
• Install the Handit SDK in your project
• Configure your API key for monitoring

📱 Test Your Connection

• The CLI will ask you to run your app to verify everything works
• Your autonomous engineer immediately starts monitoring your AI
• You'll see real-time data flowing in your dashboard

Example of code the CLI generates for you:

The CLI adds tracing only to your main agent entry points - the functions that start agent execution:

Python:

# Auto-generated by handit-cli setup
from handit_ai import tracing, configure
import os

configure(HANDIT_API_KEY=os.getenv("HANDIT_API_KEY"))

# Tracing added to your main agent function (entry point)
@tracing(agent="customer-service-agent")
async def process_customer_request(user_message: str):
    # Your existing agent logic (unchanged)
    intent = await classify_intent(user_message)      # Not traced individually
    context = await search_knowledge(intent)          # Not traced individually  
    response = await generate_response(context)       # Not traced individually
    return response

JavaScript:

// Auto-generated by handit-cli setup
import { configure, startTracing, endTracing } from '@handit.ai/handit-ai';

configure({
  HANDIT_API_KEY: process.env.HANDIT_API_KEY
});

// Tracing added to your main agent function (entry point)
export const processCustomerRequest = async (userMessage) => {
  startTracing({ agent: "customer-service-agent" });
  try {
    // Your existing agent logic (unchanged)
    const intent = await classifyIntent(userMessage);     // Not traced individually
    const context = await searchKnowledge(intent);       // Not traced individually
    const response = await generateResponse(context);     // Not traced individually
    return response;
  } finally {
    endTracing();
  }
};

Key point: The CLI only adds tracing to your main agent functions, not to every individual function. Everything inside the traced function gets captured automatically.

Your Autonomous Engineer is Now Active

✅ Check your dashboard: Go to dashboard.handit.ai to see your AI's baseline metrics.

🛠️ Automatic fixes: As soon as Handit detects your first issue, it will automatically create a pull request with the fix.

How it works:

1. Detects quality issues in your AI responses
2. Analyzes the root cause automatically
3. Generates a fix and tests it against real data
4. Creates a pull request with the proven improvement

🔍 Continuous monitoring: Your AI is monitored 24/7 - when issues are detected, fixes are shipped automatically.

Commands

setup - Initial Agent Setup

Sets up Handit instrumentation for your agent.

npx @handit.ai/cli setup

Options:

• --test - Use test environment (localhost)

Examples:

# Production setup
npx @handit.ai/cli setup

# Test with localhost  
npx @handit.ai/cli setup --test

github - GitHub Integration

Connect your repository to Handit for automatic PR creation when new prompts are detected.

npx @handit.ai/cli github

What it does:

• Authenticates with your Handit account
• Detects your Git repository
• Installs the Handit GitHub App for your repositories
• Enables automatic PR creation for prompt optimizations

Requirements:

• Git repository (any remote or no remote)
• GitHub account with repository access
• Handit account with company setup

Coming Soon

The following commands are planned for future releases:

• monitor - Collect execution traces from your agent
• evaluate - Analyze traces and provide optimization insights

Supported Languages

JavaScript/TypeScript

Supported Patterns:

• Express.js route handlers
• Async/await functions
• Arrow functions
• Class methods
• Function declarations

Example Entry Point:

// Express route handler
app.post('/process-document', async (req, res) => {
  const result = await processDocument(req.body);
  res.json(result);
});

// Regular function
async function processDocument(data) {
  const result = await analyzeDocument(data);
  return result;
}

Python

Supported Patterns:

• FastAPI endpoints
• Async functions
• Class methods
• Regular functions
• Decorators

Example Entry Point:

# FastAPI endpoint
@app.post("/process-document")
async def process_document(request: DocumentRequest):
    result = await analyze_document(request.data)
    return result

# Class method
class DocumentProcessor:
    async def process_document(self, data):
        result = await self.analyze_document(data)
        return result

Configuration

Environment Variables

# Required for AI-powered analysis
export OPENAI_API_KEY="your-openai-api-key"

# Handit.ai API key (auto-configured during setup)
export HANDIT_API_KEY="your-handit-api-key"

Configuration Files

handit.config.json (auto-generated):

{
  "agentName": "my-document-processor",
  "entryFile": "server.js",
  "entryFunction": "processDocument",
  "language": "javascript",
  "projectRoot": "/path/to/project"
}

Project Structure

After setup, your project will include:

your-project/
├── handit.config.json          # Configuration file
├── handit_service.js           # JavaScript service (auto-generated)
├── handit_service.py           # Python service (auto-generated)
├── server.js                   # Your instrumented entry point
└── services/
    └── documentProcessor.js    # Instrumented functions

Troubleshooting

Common Issues

❌ CLI command not found? Install Node.js first: node --version (should show v16+)

❌ "Authentication failed" during setup? Check your Handit.ai account credentials at dashboard.handit.ai

❌ No traces appearing in dashboard? Run handit-cli setup again and verify your API key: echo $HANDIT_API_KEY

❌ "Could not detect entry point"

• The AI will help you find the correct entry point interactively

❌ "No functions detected"

• Ensure your entry function calls other functions
• Check that functions are properly exported/imported

❌ "OpenAI API key missing"

• Set environment variable: export OPENAI_API_KEY="your-key"

Debug Mode

# Enable verbose logging
[email protected]/cli:* npx @handit.ai/cli setup

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

git clone https://github.com/Handit-AI/handit-cli.git
cd handit-cli
npm install
npm run dev

License

MIT License - see LICENSE file for details.

Support

• Documentation: docs.handit.ai
• Issues: GitHub Issues
• Discord: Join our community
• Email: [email protected]

Acknowledgments

• Built with OpenAI GPT-4 for intelligent code analysis
• Powered by Handit.ai for agent monitoring
• Inspired by the need for better AI agent observability

Made with ❤️ by the Handit Team

Website • Twitter • GitHub