npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

mmap-utils-cherry

v1.0.0

Published

node.js mmap bindings.

Downloads

4

Readme

Mmap for Node.js

mmap(2) / madvise(2) / msync(2) / mincore(2) for Node.js.

NOTE: this is a fork of https://github.com/ozra/mmap-io as that repo is unmaintained.

mmap-ing is by nature potentially blocking, and should not be used in concurrent serving/processing applications, but rather has it's niche where multiple processes are working on the same giant sets of data (thereby sparing physical memory, and load times if the kernel does it's job for read ahead), preferably multiple readers and single or none concurrent writer, to not spoil the gains by loads of spin-locks, mutexes or such.

News and Updates

2021-11-29: version 2.0.0

  • basic cleanups over docs/code.
  • fixed bug where mmap wouldn't work with lengths or offsets greater than 2GB; this was due to some bad type-casting.

2019-07-09/B: version 1.1.3, ..., 1.1.6

  • rewritten the C++-code to catch up with V8/Nan breaking changes for node.js 12.*, which also removes all warnings in earlier versions.
  • refactored in to wrapper functions for extracting values, so should new breaking changes come in later versions, it will be quicker to adjust.
  • major "package.json" mistakes. Unless running build manually, the js-files where never packaged, nor built. Now whitelisted in package.json.
  • major mistake 2: when they finally where packaged, they we're killed off when the C++ module was rebuilt on installment. So. Finally: js-files are now built to "dist", and the binary into "build". This way the TypeScript and LiveScript aren't required for end user

2019-07-09/A: version 1.1.1

  • when replacing GNU Make, for some reason I used yarn in "package.json" — which may have failed builds for those not having it installed (and then not building "es-release"), and completely missing the point of getting rid of Make.
  • updated README to reflect new build command (npm run build) (should only ever be needed if you clone from git and contribute)
  • added back the "main" entry in package.json. Hell of a blunder! Tests changed to import from root so they fail without it.
  • the never before tested example code here in the README, has been ran and corrected, thanks to @LiamKarlMitchell

2019-03-08: version 1.1.0

  • rewrote the es part of the lib code from LiveScript to TypeScript. The prudent thing to do in a lib. The test remains in LS.
  • offs_t changed to size_t because of bitwidth goofyness. Thanks to @bmarkiv
  • removed dependency on GNU Make by adding build commands to "package.json". Might help those on windows platform who didn't have it installed. However they still rely on a horde of "common posix utils", so your setup might be lacking anyway then.
  • note that there are some compile warnings because of changes in C++ API's in NAN/V8, ignored for now in contemplation whether to switch to node-addon-api (napi for C++), since call overhead isn't an issue in this library, everything considered.

2018-01-16: version 1.0.0

  • bumped the version to 1.0.0 since, well, why not.
  • changed deprecated calls from ForceSet to DefineOwnProperty
  • general source noise cleanups
  • fixed compilation errors for newer nodejs. Thanks to @djulien
  • windows-specific problems (#5, #6), and more, fixed. Thanks to @bkmartinjr
  • updated README clarifying Contribution Guidelines

2017-03-08: version 0.11.1

  • compilation fixes 10.8 OSX fix. Thanks to @arrayjam

2016-07-21: version 0.10.1

  • incore fix for Mac OS. Thanks to @rustyconover

2016-07-14: version 0.10.0

  • incore added. Thanks to @rustyconover

2015-10-10: version 0.9.4

  • Compilation on Mac should work now. Thanks to @aeickhoff

2015-10-01: version 0.9.3

  • Windows compatibility added. Thanks to @toxicwolf
  • Rewrote the bindings to Nan 2.0.9 API version (V8/io/Node hell...)
    • Had to remove the error codes to get it working in the time I had available (or rather - didn't have..) — error messages are still there — with code in message instead. Though, usually nothing goes wrong, so only the test cares ;-)
  • Added some helpful targets in Makefile human_errors, ls only, build only, etc. (useful if you wanna hack the module)
  • Since all functionality that can possibly be is in place, I bumped all the way to 0.8. Not battle tested enough to warrant higher.
  • Commented away experimental async read-ahead caching when readahead hint was on. It hasn't broken, but it's an unnecessary risk. Plays safe. You can toy with it yourself if you want to try to milk out some ms performance.

2015-03-04: version 0.1.3

  • This is the first public commit, and the code has one day of development put into it as of now. More tests are needed so don't count on it being production ready just yet (but soon).

Install

Use npm or git.

$ npm install mmap-utils
$ git clone https://github.com/ipinfo/mmap-utils.git
$ cd mmap-utils
$ npm install
$ npm build

Usage

Note: All code in examples are in LiveScript

# Following code is plastic fruit; not t[ae]sted...

mmap = require "mmap-utils"
fs = require "fs"

some-file = "./foo.bar"

fd = fs.open-sync some-file, "r"
fd-w = fs.open-sync some-file, "r+"

# In the following comments:
# - `[blah]` denotes optional argument
# - `foo = x` denotes default value for argument

size = fs.fstat-sync(fd).size
rx-prot = mmap.PROT_READ .|. mmap.PROT_EXECUTE
priv = mmap.MAP_SHARED

# map( size, protection, privacy, fd [, offset = 0 [, advise = 0]] ) -> Buffer
buffer = mmap.map size, rx-prot, priv, fd
buffer2 = mmap.map size, mmap.PROT_READ, priv, fd, 0, mmap.MADV_SEQUENTIAL
w-buffer = mmap.map size, mmap.PROT_WRITE, priv, fd-w

# advise( buffer, advise ) -> void
# advise( buffer, offset, length, advise ) -> void
mmap.advise w-buffer, mmap.MADV_RANDOM

# sync( buffer ) -> void
# sync( buffer, offset, length ) -> void
# sync( buffer, is-blocking-sync[, do-page-invalidation = false] ) -> void
# sync( buffer, offset = 0, length = buffer.length [, is-blocking-sync = false [, do-page-invalidation = false]] ) -> void

mmap.sync w-buffer
mmap.sync w-buffer, true
mmap.sync w-buffer, 0, size
mmap.sync w-buffer, 0, size, true
mmap.sync w-buffer, 0, size, true, false

# incore( buffer ) -> [ unmapped-pages Int, mapped-pages Int ]
core-stats = mmap.incore buffer

Misc

  • Checkout man pages mmap(2), madvise(2), msync(2), mincore(2) for more detailed intell.
  • The mappings are automatically unmapped when the buffer is garbage collected.
  • Write-mappings need the fd to be opened with "r+", or you'll get a permission error (13).
  • If you make a read-only mapping and then ignorantly set a value in the buffer, all hell previously unknown to a JS'er breaks loose (segmentation fault). It is possible to write some devilous code to intercept the SIGSEGV and throw an exception, but let's not do that!
  • Offset, and in some cases length needs to be a multiple of PAGESIZE (which commonly is 4096)
  • Huge pages are only supported for anonymous / private mappings (well, in Linux), so I didn't throw in flags for that since I found no use.
  • As Ben Noordhuis previously has stated: Supplying hint for a fixed virtual memory adress is kinda moot point in JS, so not supported.

Tests

$ npm test