Tricoteuses Droits et démarches

Library, Web API & site to browse Service-public.fr "Vos droits et démarches" open data from French Direction de l'information légale et administrative (Dila)

Tricoteuses Droits et démarches is free and open source software.

• software repository
• GNU Affero General Public License version 3 or greater

Vos droits et démarches data sources:

• https://www.dila.premier-ministre.gouv.fr/repertoire-des-informations-publiques/les-donnees-administratives
• https://www.data.gouv.fr/fr/datasets/service-public-fr-guide-vos-droits-et-demarches-particuliers/
• https://www.data.gouv.fr/fr/datasets/service-public-fr-guide-vos-droits-et-demarches-entreprendre/
• https://lecomarquage.service-public.fr/vdd/3.5/complet/
• https://lecomarquage.service-public.fr/vdd/3.5/part/
• https://lecomarquage.service-public.fr/vdd/3.5/pro/
• Evolutions COPERIA 3.4 to 3.5
• XSD Schemas of version 3.5 (copied from the DILA Schemas/3.5 directory into static/schemas_xsd)

Installation

Create database

Using Debian GNU/Linux, install PostgreSQL, then:

sudo su - postgres
createuser canutes -P # and enter the password
createdb -O canutes canutes
psql canutes
  CREATE SCHEMA IF NOT EXISTS droits_et_demarches AUTHORIZATION canutes;
  CREATE EXTENSION IF NOT EXISTS pg_trgm WITH SCHEMA droits_et_demarches;
  \q
exit

Install dependencies

npm install

Server Configuration

Create a .env file to set configuration variables (you can use example.env as a template). Then:

npm run configure

Datasets Initialization

mkdir -p ../dila-data/vos_droits_et_demarches/complet
cd ../dila-data/vos_droits_et_demarches/complet
git init
git branch -m main
cd -

mkdir -p ../dila-data/vos_droits_et_demarches/part
cd ../dila-data/vos_droits_et_demarches/part
git init
git branch -m main
cd -

mkdir -p ../dila-data/vos_droits_et_demarches/pro
cd ../dila-data/vos_droits_et_demarches/pro
git init
git branch -m main
cd -

Datasets Update, Cleaning, Repair & Conversion From XML To JSON

Full (aka "complet") Dataset Update, Cleaning, Repair & Conversion From XML To JSON

Full (aka "complet") Dataset Update

cd ../dila-data/vos_droits_et_demarches/complet/
git switch main
git pull

# Download XML.
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/download_droits_et_demarches.ts complet --commit --push
cd ../dila-data/vos_droits_et_demarches/complet/

Full (aka "complet") Dataset Cleaning

git branch -d nettoyage
git switch -c nettoyage
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/clean_droits_et_demarches.ts complet
cd ../dila-data/vos_droits_et_demarches/complet/
git add .
git commit -m "Nettoyage automatique par script clean_droits_et_demarches"
git push -f origin nettoyage

Full (aka "complet") Dataset Automatic Repair

git branch -d corrections_auto
git switch -c corrections_auto
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/repair_droits_et_demarches.ts complet
cd ../dila-data/vos_droits_et_demarches/complet/
git add .
git commit -m "Corrections automatique par script repair_droits_et_demarches"
git push -f origin corrections_auto

Full (aka "complet") Dataset Manual Repair

git switch corrections
git rebase corrections_auto
# Manually handle eventual conflicts, then:
git push -f origin corrections

Full (aka "complet") Dataset Conversion From XML To JSON

Full (aka "complet") Dataset Conversion From XML To JSON Before Cleaning & Repairs

cd ../dila-data/vos_droits_et_demarches/complet/
git switch main
git pull
cd ../complet_json/
git switch main
git pull
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/convert_droits_et_demarches_to_json.ts complet --all-new-commits --commit --push
cd ../dila-data/vos_droits_et_demarches/complet/

Full (aka "complet") Dataset Conversion From XML To JSON After Cleaning & Repairs

cd ../dila-data/vos_droits_et_demarches/complet/
git switch corrections
git pull
cd ../complet_json/
git switch corrections
git pull
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/convert_droits_et_demarches_to_json.ts complet --commit --push --strict
cd ../dila-data/vos_droits_et_demarches/complet/

Particuliers Dataset Update

cd ../dila-data/vos_droits_et_demarches/part/
git switch main
git pull

# Download XML.
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/download_droits_et_demarches.ts part --commit --push
cd ../dila-data/vos_droits_et_demarches/part/

Professionnels Dataset Update

cd ../dila-data/vos_droits_et_demarches/pro/
git switch main
git pull

# Download XML.
cd ../../../tricoteuses-droits-et-demarches/
npx tsx src/scripts/download_droits_et_demarches.ts pro --commit --push
cd ../dila-data/vos_droits_et_demarches/pro/

Database Update From Full (aka "complet") Dataset

npx tsx src/scripts/import_droits_et_demarches.ts

Server Launch

In development mode:

npm run dev

In production mode:

npm run build
npm run preview

Development

Public library schemas

The package root exports browser-compatible Zod schemas for the converted JSON data, for example PublicationSchema. These schemas validate JSON objects used by the website and by downstream front-end applications.

XML parsing and XML-to-JSON validation are server-only concerns. XML schemas are kept under src/lib/server/schemas and use explicit *XmlSchema names, for example PublicationXmlSchema, so they cannot be imported indirectly by browser code.

JSON Schemas Generation

npx tsx src/scripts/generate_droits_et_demarches_json_schemas.ts
# Prettify generated JSON file.
npm run format

Note: JSON schemas are generated in file static/schemas.json