cjs-module-lexer-rs

v0.2.6

Published

3 years ago

[![npm](https://img.shields.io/npm/v/cjs-module-lexer-rs)](https://www.npmjs.com/package/cjs-module-lexer-rs) ![NPM](https://img.shields.io/npm/l/cjs-module-lexer-rs)

Downloads

0High
0Medium
0Low

chenyuwang

NPM

CJS Module Lexer (Rust)

This is a rewrite of cjs-module-lexer in Rust. It is a CommonJS lexer used to detect the most likely list of named exports of a CommonJS module.

Online Playground

CJS Module Lexer Playground

Installation

WebAssembly JS Wrapper

With Package Managers

npm i cjs-module-lexer-rs

yarn add cjs-module-lexer-rs

pnpm add cjs-module-lexer-rs

With esm.sh

import { init, parse } from "https://esm.sh/cjs-module-lexer-rs";

Rust

Coming soon...

Get Started

Node

// example.js
const { init, parse } = require("cjs-module-lexer-rs");

const code = `
    module.exports.asdf = 'asdf';
    exports = 'asdf';
    module.exports = require('./asdf');
    if (maybe)
    module.exports = require("./another");
`;

init().then(() => console.log(parse(code, 'filename.js')));

{
  imports: [ './asdf', './another'],
  exports: [ 'asdf' ],
  reexports: [ './another' ],
  errors: []
}

Web

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Lexer Example</title>
    </head>
    <body>
        <script type="module">
            import { init, parse } from "https://esm.sh/cjs-module-lexer-rs";

            const code = `
                module.exports.asdf = 'asdf';
                exports = 'asdf';
                module.exports = require('./asdf');
                if (maybe)
                module.exports = require("./another");
            `;

            init().then(() => console.log(parse(code, 'filename.js')));
        </script>
    </body>
</html>

{
    "imports": [
        "./asdf",
        "./another",
    ],
    "exports": [
        "asdf"
    ],
    "reexports": [
        "./another"
    ],
    "errors": []
}

The Why

The frontend tooling has migrated to usage of moderm native languages to accelerate development and improve developer experience. cjs-module-lexer remains useful in many scenes, but it assumes the code is UTF16 and only allows single-thread. Making cjs-module-lexer working with Rust requires FFI and unsafe code. Hopefully this Rust library will be useful for Rust tooling authors to interop between CJS and ESM.

Features

| Feature | Status | Since | Note | |---|---|---|---| | exports.asdf = x | 👌 | 0.1.0 | | exports['asdf'] = x | 👌 | 0.1.0 | | module.exports = { ... } | 👌 | 0.1.0 | { ... } is like { a, b, c: d }, where d is Literal or Identifier | | require('module') | 👌 | 0.1.0 | | Object.defineProperty(exports, 'q', { enumerable: true, get() { return q } }) | 👷 | | TypeScript: export {colorFactory} from './color-factory'; | __export, __exportStar | 👷 | | TypeScript: export * from 'external' | | Skip StringLiteral | 👌 | 0.1.0 | | Skip RegularExpressionLiteral | 👌 | 0.1.0 | Skip Template | 👌 | 0.1.0 | Non-unicode Named Export | ❌ | | Not supported due to std::str only allows unicode strings

Reference

https://babeljs.io/docs/en/babel-plugin-transform-modules-commonjs

Benchmarks

Native

cargo bench

test tests::bench_angular          ... bench:   5,176,437 ns/iter (+/- 260,598)
test tests::bench_angular_min      ... bench:   2,321,468 ns/iter (+/- 939,693)
test tests::bench_d3               ... bench:   3,155,154 ns/iter (+/- 182,490)
test tests::bench_d3_min           ... bench:   1,897,146 ns/iter (+/- 100,576)
test tests::bench_magic_string     ... bench:     209,630 ns/iter (+/- 11,577)
test tests::bench_magic_string_min ... bench:     145,834 ns/iter (+/- 7,527)
test tests::bench_rollup           ... bench:   3,918,989 ns/iter (+/- 130,726)
test tests::bench_rollup_min       ... bench:   2,637,478 ns/iter (+/- 74,449)