GenIRSim

Flexible simulation and evaluation framework for generative IR. [demo] [docs]

Tested in node 22.15.0 with npm 11.3.0.

Quickstart

If you are not within the Webis network, you first need to replace all occurrences of the URL https://llm.srv.webis.de/api/chat and the model (default) in the configuration with the values for a server you have access to (OpenAI-compatible API). Then,

npm install
node bin/genirsim static/configurations/discussion.json > eval.json

To run the web server:

npm install
node bin/genirsim-server

or

GENIRSIM_VERSION=$(jq -r '.version' package.json)
docker run --rm -it -p 8000:8000 ghcr.io/webis-de/genirsim:${GENIRSIM_VERSION}