Language Style Profiler

Extract comprehensive language style profiles from text of any length using OpenAI-compatible APIs and the intext library.

Features

• Long Text Support: Process arbitrarily long documents using sliding-window chunking
• OpenAI-Compatible: Works with OpenAI, Anthropic, Azure, local models, and any OpenAI-compatible API
• Flexible Schemas: Use the built-in language style profile schema or provide custom JSON schemas
• Type-Safe: Full TypeScript support with generics for strongly-typed results
• Provenance Tracking: Understand which parts of the text influenced each extracted field

Installation

npm install language-style-profiler

Quick Start

import { StyleProfiler, createOpenAIClient } from 'language-style-profiler';

const openai = createOpenAIClient(process.env.OPENAI_API_KEY!);
const profiler = new StyleProfiler(openai);

const text = "Your long document text here...";
const result = await profiler.extract(text);

console.log(result.profile.lexicalChoice.formality); // "formal" | "neutral" | "colloquial"
console.log(result.metadata.chunkCount); // Number of chunks processed
console.log(result.metadata.perFieldProvenance); // Which chunks influenced each field

Usage

Basic Usage with Default Schema

The package includes a comprehensive language style profile schema that analyzes:

• Lexical Choice: Formality, complexity, concreteness, jargon usage
• Syntactic Structure: Sentence length, parallelism, voice, complexity
• Rhetorical Devices: Metaphor usage, narrative approach, questioning style
• Persuasive Appeal: Emotional tone, logical flow, evidence type
• Interactivity: Audience engagement, personal disclosure, call to action
• Information Density: Content pacing, conceptual difficulty, repetition style

import { StyleProfiler, createOpenAIClient } from 'language-style-profiler';

const openai = createOpenAIClient(process.env.OPENAI_API_KEY!);
const profiler = new StyleProfiler(openai, {
  model: 'gpt-4o-mini',
  temperature: 0.1,
  chunkTokens: 500,
  overlapTokens: 50,
  concurrency: 8
});

const result = await profiler.extract(longText);

// Access extracted dimensions
console.log(result.profile.lexicalChoice.formality);
console.log(result.profile.syntacticStructure.sentenceLength);
console.log(result.profile.rhetoricalDevices.metaphorUsage);

Custom Schema Extraction

Provide your own JSON schema for custom extraction tasks:

const customSchema = {
  type: "object",
  properties: {
    sentiment: {
      type: "string",
      description: "Overall sentiment of the text",
      enum: ["positive", "negative", "neutral", "mixed"]
    },
    urgency: {
      type: "string",
      description: "Level of urgency conveyed",
      enum: ["high", "medium", "low"]
    },
    topics: {
      type: "array",
      description: "Main topics discussed",
      items: { type: "string" }
    }
  }
};

const result = await profiler.extract<{
  sentiment: "positive" | "negative" | "neutral" | "mixed";
  urgency: "high" | "medium" | "low";
  topics: string[];
}>(longText, { schema: customSchema });

console.log(result.profile.sentiment); // Type-safe access

Runtime Overrides

const profiler = new StyleProfiler(openai);

// Override options for specific extraction
const result = await profiler.extract(text, {
  overrides: {
    temperature: 0.0,  // More deterministic
    concurrency: 1,    // Lower rate limiting
    debug: true        // Enable debug logging
  }
});

Using and Customizing the Default Schema

The default language style schema is exported for direct use or customization:

import { StyleProfiler, createOpenAIClient, defaultSchema } from 'language-style-profiler';

const openai = createOpenAIClient(process.env.OPENAI_API_KEY!);
const profiler = new StyleProfiler(openai);

// Use the default schema directly
const result1 = await profiler.extract(text, { schema: defaultSchema });

// Or extend it with custom fields
const extendedSchema = {
  ...defaultSchema,
  properties: {
    ...defaultSchema.properties,
    customDimension: {
      type: "object",
      description: "Your custom analysis dimension",
      properties: {
        customField: {
          type: "string",
          description: "Custom field description",
          enum: ["option1", "option2", "option3"]
        }
      }
    }
  }
};

const result2 = await profiler.extract(text, { schema: extendedSchema });

// Alternative: get schema via profiler instance
const schema = profiler.getDefaultSchema();

API Reference

StyleProfiler

Constructor

new StyleProfiler(client: OpenAICompatibleClient, options?: ProfilerOptions)

Parameters:

• client: Any OpenAI-compatible client instance (must expose chat.completions.create)
• options: Optional configuration object

Options:

• chunkTokens (number, default: 2000): Tokens per chunk
• overlapTokens (number, default: 400): Overlap between chunks
• concurrency (number, default: 3): Parallel chunk processing
• model (string, default: 'gpt-4'): Model identifier
• temperature (number, default: 0.1): Sampling temperature
• maxTokens (number): Max tokens per LLM call
• debug (boolean): Enable debug logging

Methods

extract<T>(text: string, options?: ExtractOptions): Promise<ExtractionResult<T>>

Extract style profile from text.

Parameters:

• text: The text to analyze
• options: Optional extraction configuration
  • schema: Custom JSON schema (uses default if not provided)
  • overrides: Override constructor options for this extraction

Returns:

{
  profile: T,  // Extracted data matching your schema
  metadata: {
    chunkCount: number,
    perFieldProvenance: Record<string, { sourceChunks: number[] }>,
    rawChunkResults: Array<{ chunkId: number; parsed: Record<string, unknown> }>
  }
}

getDefaultSchema(): SchemaField

Get a copy of the default language style profile schema.

License

MIT

Contributing

Contributions welcome! Please open an issue or PR on GitHub.