nodepyx

Run Python libraries from Node.js as if they were native — embed CPython in-process with full TypeScript types, Proxy-based API, async/await support, and Next.js integration.

Why nodepyx?

Traditional Python↔Node.js bridges spawn a separate Python process and communicate via stdin/stdout or HTTP — introducing serialization overhead, process startup latency, and complex lifecycle management.

nodepyx embeds CPython directly inside the Node.js process via a native N-API addon. Every Python call happens in-process: no IPC, no child processes, no HTTP servers.

| Feature | nodepyx | python-shell / pythonia | gRPC/HTTP | |---------|--------|------------------------|-----------| | In-process (zero IPC) | ✅ | ❌ | ❌ | | TypeScript types | ✅ Full .d.ts | Partial | Manual | | await on Python objects | ✅ Proxy thenable | ❌ | ❌ | | NumPy zero-copy | ✅ SharedArrayBuffer | ❌ | ❌ | | DataFrame native | ✅ DataFrameResult | ❌ | ❌ | | Next.js integration | ✅ withnodepyx() | ❌ | ❌ | | Auto venv/conda | ✅ | Partial | ❌ |

Quick Start

npm install nodepyx

import { init, py } from 'nodepyx';

await init({
  virtualenv: {
    path:        './.venv',
    packages:    ['pandas', 'numpy', 'scikit-learn'],
    autoInstall: true,
  },
});

// Import Python modules — works exactly like Python `import`
const pd  = await py.import('pandas');
const np  = await py.import('numpy');

// Call functions with await
const df  = await pd.read_csv('data.csv');
const arr = await np.arange(100);          // → Float64Array

// Chain attribute access
const mean = await df.describe().loc['mean'];

// Iterate Python generators
for await (const row of df.iterrows()) {
  console.log(row);
}

Installation

Prerequisites

• Node.js ≥ 18.0.0
• Python 3.8 – 3.12 (auto-detected; or set nodepyx_PYTHON)
• C++ build tools (for native addon)
  • Linux: gcc, g++, make, python3
  • macOS: Xcode Command Line Tools (xcode-select --install)
  • Windows: Visual Studio Build Tools + Python

# Install from npm
npm install nodepyx

# Or with pre-built binary (skips compilation)
nodepyx_SKIP_BUILD=1 npm install nodepyx && npm run download-prebuilds

Configuration

import { init } from 'nodepyx';

await init({
  // Python executable (default: auto-detect)
  pythonExecutable: '/usr/bin/python3.11',

  // Virtualenv (recommended)
  virtualenv: {
    path:        './.venv',
    packages:    ['pandas>=2.0', 'numpy>=1.24', 'scikit-learn'],
    autoInstall: true,    // install missing packages automatically
    autoCreate:  true,    // create venv if it does not exist
  },

  // OR use Conda
  conda: {
    envName:  'my-env',
    packages: ['pytorch', 'torchvision'],
  },

  // Thread pool for GIL management
  threadPoolSize: 4,        // default: 4

  // Logging
  logLevel: 'info',         // 'debug' | 'info' | 'warn' | 'error'

  // Memory limits
  memory: {
    heapLimitMB: 2048,
    gcThresholdMB: 512,
  },
});

API Reference

Lifecycle

import { init, shutdown, isInitialized } from 'nodepyx';

await init(config?);      // Initialize Python runtime
await shutdown();         // Release all Python resources
isInitialized();          // → boolean

py — Main Proxy Object

import { py } from 'nodepyx';

// Import a module
const np = await py.import('numpy');

// Evaluate a Python expression
const result = await py.eval('1 + 2 + 3');  // → 6

// Execute statements
await py.exec(`
  import sys
  print(sys.version)
`);

// Run a .py file
await py.runFile('./my_script.py');

// Install packages at runtime
await py.installPackages(['requests', 'httpx']);

Proxy API

Every Python object returned by nodepyx is a JavaScript Proxy with full await support:

const pd = await py.import('pandas');

// Attribute access
const version = await pd.__version__;      // string

// Method calls
const df = await pd.DataFrame({ a: [1,2,3], b: [4,5,6] });
const shape = await df.shape;              // [3, 2]
const desc  = await df.describe();         // DataFrameResult

// Chained calls
const top5 = await df.sort_values('a', { ascending: false }).head(5);

// Indexing  (df['column'])
const col = await df['a'];                 // SeriesResult

NumPy Arrays

const np  = await py.import('numpy');
const arr = await np.linspace(0, 1, 100);  // NumPyArrayResult

// NumPyArrayResult interface:
arr.data     // Float64Array (or Int32Array, Uint8Array, etc.)
arr.shape    // number[]  — e.g. [100] or [3, 4]
arr.dtype    // 'float64' | 'float32' | 'int32' | ...
arr.ndim     // number
arr.size     // number — total element count

// Send TypedArrays back to Python
const NumpyBridge = new NumpyBridge();
const wire = NumpyBridge.serializeTypedArray(new Float64Array([1,2,3]), { dtype:'float64', itemsize:8 }, [3]);

Pandas DataFrames

const pd = await py.import('pandas');
const df = await pd.read_csv('data.csv');  // DataFrameResult

// DataFrameResult interface:
df.columns   // string[]
df.records   // Record<string, unknown>[]  — one object per row
df.index     // unknown[]
df.dtypes    // Record<string, string>
df.shape     // [rows, cols]

// Static helpers
import { DataFrameBridge } from 'nodepyx';
const cols = DataFrameBridge.toColumnArrays(df);  // column-keyed arrays
const filtered = DataFrameBridge.filterRows(df, row => row.score > 0.9);
const stats = DataFrameBridge.describeColumn(df, 'price');  // {count,mean,std,min,max}

Error Handling

import { PythonError, isPythonError, isPythonErrorOfType } from 'nodepyx';

try {
  await py.eval('1 / 0');
} catch (err) {
  if (isPythonError(err)) {
    console.log(err.pythonType);   // 'ZeroDivisionError'
    console.log(err.message);      // '[Python ZeroDivisionError] division by zero'
    console.log(err.traceback);    // Full Python traceback
  }
}

// Type-specific check
if (isPythonErrorOfType(err, 'ImportError')) { ... }

Environment Management

Virtualenv (recommended)

import { VenvManager } from 'nodepyx';

const mgr = new VenvManager({
  venvPath:   './.venv',
  packages:   ['pandas', 'numpy', 'scikit-learn'],
  autoCreate: true,
});

const result = await mgr.setup();
console.log('Installed:', result.installedPackages);
console.log('Venv python:', result.pythonExecutable);

Conda

import { CondaManager } from 'nodepyx';

const conda = new CondaManager({ envName: 'ml-env', packages: ['pytorch', 'pip::some-pip-package'] });
const result = await conda.setup();

PackageInstaller

import { PackageInstaller } from 'nodepyx';

const pip = new PackageInstaller({ pythonExecutable: '/usr/bin/python3' });

// Install with progress events
pip.on('progress', (ev) => console.log(ev.package, ev.status));

const result = await pip.install(['pandas>=2.0', 'numpy>=1.24']);
console.log('installed:', result.installed);
console.log('already present:', result.alreadyInstalled);

TypeScript Stubs

# Generate .d.ts stubs for Python modules
npx nodepyx-stubs pandas numpy sklearn torch

# Regenerate all cached stubs
npx nodepyx-stubs --all

# List cached stubs
npx nodepyx-stubs --list

Then in your TypeScript code:

import type { DataFrame } from 'nodepyx/pandas';   // full IDE autocomplete
import type { ndarray }   from 'nodepyx/numpy';

Next.js Integration

// next.config.js
const { withnodepyx } = require('nodepyx/next');

module.exports = withnodepyx({
  // your Next.js config
}, {
  virtualenv: { path: './.venv', packages: ['pandas'], autoInstall: true },
});

// app/api/python/route.ts  (Server Route — Node.js runtime only)
import { NextResponse } from 'next/server';
import { init, evalPython } from 'nodepyx';

let ready = false;
async function ensureReady() {
  if (!ready) { await init(); ready = true; }
}

export async function GET() {
  await ensureReady();
  const result = await evalPython('list(range(10))');
  return NextResponse.json(result);
}

// Client component
'use client';
import { usePython } from 'nodepyx/next';

export function Chart() {
  const { data, loading, run } = usePython(async (py) => {
    const np = await py.import('numpy');
    return (np as any).random.randn(100).tolist();
  });

  return <button onClick={run}>{loading ? '…' : 'Generate'}</button>;
}

Plugins

import { init } from 'nodepyx';
import type { nodepyxPlugin } from 'nodepyx';

const myPlugin: nodepyxPlugin = {
  name: 'my-plugin',
  version: '1.0.0',
  supportedModules: ['my_lib'],
  handledTypes: ['MySpecialType'],

  async onInit(ctx) {
    ctx.registerTypeHandler('MySpecialType', (raw, typeHint) => {
      // Transform raw wire dict into a JS value
      return { specialData: JSON.parse(raw.data as string) };
    });
  },
};

await init({ plugins: [myPlugin] });

Supported Python Versions

| Python | Status | |--------|--------| | 3.12 | ✅ Recommended | | 3.11 | ✅ Supported | | 3.10 | ✅ Supported | | 3.9 | ✅ Supported | | 3.8 | ✅ Supported (EOL) | | 3.7 | ❌ Not supported |

License

MIT © nodepyx contributors