@shadowdara/linguist-js
v1.0.0
Published
A Node.js reimplementation of github/linguist with full .gitattributes support (linguist-language, linguist-vendored, linguist-documentation, linguist-generated, linguist-detectable, linguist-ignore).
Readme
linguist-js
A standalone Node.js tool that computes a repository's language statistics
like github/linguist and
respects all relevant .gitattributes directives:
linguist-language=<Name>— force the language for matching fileslinguist-vendored[=true|false]— mark/unmark as vendored codelinguist-documentation[=true|false]— mark/unmark as documentationlinguist-generated[=true|false]— mark/unmark as generatedlinguist-detectable[=true|false]— override the default visibility in the statisticslinguist-ignore— ignore the file completely (also in--files)
Nested .gitattributes files are handled correctly: per path, the
nearest .gitattributes file always wins for an attribute (same
behavior as Git itself); within a file, the last matching line wins.
Usage
node linguist.js [Path] # Print language statistics (default: current directory)
node linguist.js [Path] --files # Additionally list each file with its detected language
node linguist.js [Path] --breakdown # Also print a per-language file list
node linguist.js [Path] --json # Output as JSON
node linguist.js [Path] --all # Without git: scan the entire directory tree (instead of `git ls-files`)By default, git ls-files is used, so .gitignore is respected
automatically (untracked/ignored files do not appear there). Outside a Git
repository or with --all, the complete file tree is scanned instead
(only .git/ is excluded).
Example
With a .gitattributes like:
samfile linguist-language=Shell
go.sum linguist-language=go
docker-compose.yml linguist-language=Dockerfile
/cmd/findergen/frontend/** linguist-vendored
/.obsidian/** linguist-vendored
*.json linguist-detectable=truelinguist.js detects:
samfileas Shell, even though it has no extensiongo.sumas go (override), regardless of the default detectiondocker-compose.ymlas Dockerfile instead of YAML- everything under
/cmd/findergen/frontend/and/.obsidian/is excluded from the statistics (vendored) *.jsonfiles are included in the statistics (unlike the default)
Language database (languages.json)
Contains a hand-picked subset (~90 entries) of Linguist's real
languages.yml: type (programming/markup/data/prose), extensions,
filenames, interpreters (for shebang detection) and optional
detectable (default: true for programming/markup, false for
data/prose — analogous to Linguist, see e.g. JSON/YAML/Markdown).
You can extend the file freely to add more languages, extensions or filenames.
Known limitations (deliberate simplifications)
Compared to the real Ruby gem github-linguist:
- No heuristics/Bayes classifier for ambiguous extensions (e.g.
.hC vs. C++,.mObjective-C vs. MATLAB) — the first match fromlanguages.jsonis always taken. For such cases, simply setlinguist-languagein.gitattributes. - Binary detection is a simple null-byte scan of the first 8 KB (no
charlock_holmes/encoding detection). - Vendor/documentation/generated defaults are a compact subset of the
real
vendor.yml/documentation.ymlrules — the most important cases (node_modules, dist, build, vendor, lockfiles, README/LICENSE/CHANGELOG) are covered, but not complete. - With an explicit
linguist-languageoverride, the file is by default automatically treated asdetectable(unlesslinguist-vendored/-documentation/-generated/-detectable=falseis also explicitly set) — this matches the common practical purpose of such an override (e.g.go.sum linguist-language=go, so that it shows up in the statistics).
For the absolute reference implementation, the official github-linguist gem remains authoritative — this tool is intended for Node-only environments/CI without Ruby.
