Type Extraction Framework

A powerful TypeScript type extraction system that uses the TypeScript Compiler API to generate type definitions from third-party APIs that don't provide official TypeScript clients.

Overview

This framework solves a common problem: many third-party APIs (like Guesty, Stripe, Twilio) provide comprehensive REST APIs but lack official TypeScript type definitions. Rather than manually maintaining types or using error-prone regex patterns, this framework uses the TypeScript AST to extract accurate type definitions from existing code.

Key Features

• ESM-first - Built for modern React 19 and Next.js projects
• AST-based extraction - Uses TypeScript Compiler API for accurate parsing
• Handles complex types - Nested objects, generics, unions, intersections
• API-specific adapters - Extensible architecture for different APIs
• Type transformations - Rename, add/remove properties, create discriminated unions
• JSDoc preservation - Maintains documentation and comments
• Zero any usage - Converts any to unknown for type safety
• Performance optimized - Sub-second extraction for large codebases

Architecture

app/build/types/
├── core/                      # Core framework
│   ├── types.ts              # Type definitions
│   ├── extractor.ts          # Base extractor class
│   ├── generator.ts          # File generator
│   └── index.ts              # Public API
├── adapters/                  # API-specific implementations
│   ├── guesty/               # Guesty adapter
│   ├── stripe/               # Stripe adapter (future)
│   └── twilio/               # Twilio adapter (future)
└── README.md                 # This file

Installation

# Install as a dev dependency (when published to npm)
npm install --save-dev @invisiblecities/type-extraction

# Or copy the framework to your project
cp -r app/build/types ./type-extraction

Quick Start

1. Create Configuration File

Create type-extraction.config.js in your project root:

export default {
  api: 'myapi',
  source: {
    root: './lib/myapi',
    patterns: ['**/*.ts']
  },
  output: {
    directory: './src/types/generated',
    filename: '{api}.types.ts'
  }
};

2. Run Extraction

# Using config file
type-extract

# Or with CLI options
type-extract --api myapi --source ./lib/myapi

# This generates:
# - src/types/generated/myapi.types.ts
# - src/types/generated/index.ts

Configuration

Configuration File

The framework looks for configuration in these locations:

• type-extraction.config.js
• type-extraction.config.mjs
• type-extraction.config.json
• .type-extractionrc.js
• .type-extractionrc.json

Default Output Locations

{
  output: {
    // Generated types go here (relative to project root)
    directory: './src/types/generated',
    
    // Filename pattern
    filename: '{api}.types.ts',
    
    // Also generates index.ts
    generateIndex: true
  }
}

Important: Generated files should be:

• ✅ Committed to version control
• ✅ Added to .prettierignore (they're auto-formatted)
• ❌ NOT in node_modules (not persistent)
• ❌ NOT in dist/ or build/ (often gitignored)

Plugin Architecture

The type extraction framework uses a flexible plugin architecture that allows you to keep API-specific adapters in your project while using the framework from npm.

Adapter Loading Order

When you run type-extract --api guesty, the CLI searches for adapters in this order:

1. Built-in adapters (inside the npm package)

   node_modules/@invisiblecities/type-extraction/adapters/guesty/

2. Separate npm packages

   node_modules/@invisiblecities/type-extraction-guesty/

3. Local project adapters (recommended for client-specific APIs)

   ./type-extraction/adapters/guesty/extractor.js

Recommended Setup

After installing the framework as a dev dependency:

npm install --save-dev @invisiblecities/type-extraction

Create your adapter in your project:

your-project/
├── type-extraction/
│   └── adapters/
│       └── guesty/
│           ├── extractor.ts    # Your Guesty adapter
│           └── index.ts        # Exports
├── src/
└── package.json

This keeps your proprietary API logic in your codebase while leveraging the framework's infrastructure.

Creating a New API Adapter

1. Create adapter directory

   mkdir -p type-extraction/adapters/myapi

2. Implement the extractor

   // type-extraction/adapters/myapi/extractor.ts
   import { BaseTypeExtractor } from '@invisiblecities/type-extraction';
   import type { ExtractionRules } from '@invisiblecities/type-extraction';
   
   export class MyAPIExtractor extends BaseTypeExtractor {
     constructor() {
       const rules: ExtractionRules = {
         apiId: 'myapi',
            
         // Define transformations
         transforms: {
           'Order': {
             // Add discriminated union
             discriminate: 'status',
             variants: {
               'pending': 'PendingOrder',
               'completed': 'CompletedOrder'
             }
           }
         },
            
         // Exclude internal types
         excludeTypes: ['InternalConfig'],
            
         // Custom naming
         naming: {
           prefix: 'MyAPI',
           transform: (name: string) => `MyAPI${name}`
         }
       };
          
       super(rules);
     }
   
     protected async applyTransformations(): Promise<void> {
       // Apply API-specific transformations
       // The base class handles standard transforms
     }
   
     protected async validateTypes(): Promise<void> {
       // Validate extracted types
       // Add custom validation logic
     }
   }

3. Create extraction script

   // scripts/extract-myapi-types.mjs
   import { MyAPIExtractor } from '../app/build/types/adapters/myapi/extractor.js';
   import { TypeGenerator } from '../app/build/types/core/generator.js';
   import { glob } from 'glob';
   
   async function main() {
     // Discover source files
     const files = await glob('lib/myapi/**/*.ts', {
       absolute: true,
       ignore: ['**/*.test.ts']
     });
   
     // Extract types
     const extractor = new MyAPIExtractor();
     const context = await extractor.extract(files);
   
     // Generate output
     const generator = new TypeGenerator({
       outputDir: './src/generated',
       filePattern: '{api}-types.ts',
       generateIndex: true,
       splitTypes: false
     });
   
     await generator.generate(context);
   }
   
   main();

API-Specific Rules

Transformations

Transform extracted types to match your application's conventions:

transforms: {
  'User': {
    // Rename the type
    rename: 'AppUser',
    
    // Add properties
    addProperties: [{
      name: 'fullName',
      type: 'string',
      optional: true,
      readonly: false
    }],
    
    // Remove properties
    removeProperties: ['internalId'],
    
    // Transform properties
    transformProperties: {
      'email': {
        type: 'Email', // Use branded type
        optional: false
      }
    }
  }
}

Discriminated Unions

Convert regular types to discriminated unions for better type safety:

transforms: {
  'Payment': {
    discriminate: 'method',
    variants: {
      'card': 'CardPayment',
      'bank': 'BankPayment',
      'crypto': 'CryptoPayment'
    }
  }
}

This generates:

interface PaymentBase {
  amount: number;
  currency: string;
}

interface CardPayment extends PaymentBase {
  method: 'card';
  last4: string;
}

interface BankPayment extends PaymentBase {
  method: 'bank';
  accountNumber: string;
}

type Payment = CardPayment | BankPayment | CryptoPayment;

Validation

Add custom validators to ensure type quality:

validators: {
  'Order': (type: ExtractedType) => {
    const required = ['id', 'customerId', 'items'];
    const props = type.properties?.map(p => p.name) || [];
    
    const missing = required.filter(r => !props.includes(r));
    
    return {
      valid: missing.length === 0,
      errors: missing.map(m => `Missing required property: ${m}`)
    };
  }
}

Output Options

Unified File (Default)

All types in a single file:

const generator = new TypeGenerator({
  outputDir: './generated',
  filePattern: '{api}-types.ts',
  splitTypes: false
});

Split Files

One file per type:

const generator = new TypeGenerator({
  outputDir: './generated/types',
  splitTypes: true,
  generateIndex: true
});

Best Practices

1. Source Organization

Keep API implementation code organized:

lib/myapi/
├── client.ts         # API client
├── endpoints/        # Endpoint implementations
├── types/           # Manual type definitions
└── utils/           # Helper functions

2. Type Safety

• Never use any - the framework converts to unknown
• Use branded types for primitive values (emails, IDs, etc.)
• Implement proper validation for all external data

3. Documentation

• Preserve JSDoc comments in source files
• Document transformations and their rationale
• Keep extraction rules well-commented

4. Performance

• Target < 500ms extraction time
• Use file patterns to limit scope
• Cache results when appropriate

Common Issues

Missing Closing Braces

Problem: Regex-based extraction truncates at first }

Solution: This framework uses AST parsing which handles nested objects correctly

Duplicate Exports

Problem: JSDoc comments between export keywords

Solution: The framework intelligently merges JSDoc with declarations

Type Conflicts

Problem: Same type name from multiple sources

Solution: Use naming rules to add prefixes/suffixes

Performance Metrics

The framework tracks:

• Files parsed
• Types extracted
• Transformations applied
• Validation results
• Extraction time

Example output:

📊 Extraction Results:
   Files parsed: 54
   Types extracted: 633
   Transforms applied: 12
   Validations passed: 621
   Validations failed: 12
   Extraction time: 505ms

CLI Usage

# Using configuration file
type-extract

# Specify API directly
type-extract --api stripe --source ./lib/stripe

# Custom output directory
type-extract --api twilio --output ./types/twilio

# Verbose mode
type-extract --verbose

# Show help
type-extract --help

Project Structure Conventions

my-project/
├── type-extraction.config.js    # Configuration file
├── src/
│   ├── types/
│   │   └── generated/           # Default output location
│   │       ├── myapi.types.ts   # Generated types
│   │       └── index.ts         # Index file
│   └── lib/
│       └── myapi/               # API implementation
└── package.json

Future Enhancements

• [x] Configuration system
• [x] CLI interface
• [ ] Watch mode for continuous extraction
• [ ] Incremental extraction (only changed files)
• [ ] Type dependency graph visualization
• [ ] Automatic discriminated union detection
• [ ] Integration with OpenAPI specs
• [ ] Type compatibility checking
• [ ] NPM package publication

Contributing

To add support for a new API:

1. Create an adapter in app/build/types/adapters/[api-name]/
2. Implement extraction rules and transformations
3. Add comprehensive tests
4. Document the adapter's specific features
5. Submit a PR with examples

License

MIT