npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

pg-anonymize

v1.3.0

Published

Export your postgreSQL database anonymized

Downloads

6

Readme

pg-anonymize

Export your PostgreSQL database anonymized. Replace all sensitive data thanks to faker. Output to a file that you can easily import with psql.

oclif Version Downloads License

Usage

Run this command by giving a connexion string and an output file name (no need to install first thanks to npx):

npx pg-anonymize postgres://user:secret@localhost:1234/mydb -o dump.sql

☝️ This command requires pg_dump. It may already be installed as soon as PostgreSQL is installed.

Output can also be stdout ('-') so you can pipe the output to zip, gz, or to psql:

npx pg-anonymize postgres://user:secret@localhost:1234/mydb -o - | psql DATABASE_URL

API

--columns | -c

Specify list of columns to anonymize

Use --columns option with a comma separated list of column name:

npx pg-anonymize postgres://localhost/mydb \
  --columns=email,firstName,lastName,phone

Specifying another list via --columns replace the default automatically anonymized values:

email,name,description,address,city,country,phone,comment,birthdate

You can also specify the table for a column using the dot notation:

public.user.email,public.product.description,email,name

Customize replacements

You can also choose which faker function you want to use to replace data (default is faker.random.word):

npx pg-anonymize postgres://localhost/mydb \
  --columns=firstName:faker.name.firstName,lastName:faker.name.lastName

:point_right: You don't need to specify faker function since the command will try to find correct function via column name.

You can use plain text too for static replacements:

npx pg-anonymize postgres://localhost/mydb \
  --columns=textcol:hello,jsoncol:{},intcol:12

--extension

Use an extension file to create your own custom replacements

Create an extension file, written in javascript

// myExtension.js
module.exports = {
  maskEmail: (email) => {
   const [name, domain] = email.split('@');
   const { length: len } = name;
   const maskedName = name[0] + '...' + name[len - 1];
   const maskedEmail = maskedName + '@' + domain;
   return maskedEmail;
  }
};

Pass the path to --extension and use the module exports in --columns

npx pg-anonymize postgres://localhost/mydb \
  --extension ./myExtension.js \
  --columns=email:extension.maskEmail

--config | -f

Use a configuration file

You can use the --config option to specify a file with a list of column names and optional replacements, one per line:

Create a configuration file:

name
email
password:faker.random.word

Pass the path to the file into --config

npx pg-anonymize postgres://localhost/mydb \
  --config /path/to/file

--skip

Skip tables

Use --skip to skip anonymizing entire tables

npx pg-anonymize postgres://localhost/mydb --skip public.posts

--preserve-null | -n

Preserve NULL values

Use --preserve-null to skip anonymization on fields with NULL values.

npx pg-anonymize postgres://localhost/mydb --preserve-null

--faker-locale

Set fakers locale (i18n)

Use --faker-locale to change the locale used by faker (default: en)

Import the anonymized file

The anonymized output file is plain SQL text, you can import it with psql.

psql -d mylocaldb < output.sql

Why

There are a bunch of competitors, still I failed to use them:

  • postgresql_anonymizer may be hard to setup and may be cumbersome for simple usage. Still, I guess it's the best solution.
  • pganonymize fails when it does not use public schema or columns have uppercase characters
  • pganonymizer also fails with simple cases. Errors are not explicit and silent.