ai-compare-candidates

Compare and rank multiple candidate objects using artificial intelligence retrieval augmented generation, providing the rationale

This package allows you to rank multiple candidate objects in a customised manner by providing a user-supplied function that converts each such object into a candidate document in the form of a string. It then uses a vector database and embedding to perform a similarity search retrieval of an initial number of candidates. This search is then refined into the same or a smaller number of candidates by feeding them into a large language model which will rank the candidates. The candidate documents or part thereof can also be summarised into a defined word limit.

Cloning of package

After performing a git clone:

0. This assumes that you have a suitable node version installed. If you don't already have yarn installed globally:

npm install -g yarn

1. Remove the line "packageManager": "[email protected]" and the preceding comma in package.json
2. Remove the line yarnPath: .yarn/releases/yarn-4.12.0.cjs from .yarnrc.yml

yarn set version 4.12.0
yarn --immutable

Running of example

0. If you don't already have @quasar/cli installed globally

npm install -g @quasar/cli

cd example
yarn workspaces focus
yarn dev

Usage

AICompareCandidates.compareCandidates<Candidate>({
	candidates,
	problemDescription='',
	generateSearchAreasInstruction=this.defaultGenerateSearchAreasInstruction.bind(this),
	parseSearchAreasResponse=this.defaultParseSearchAreasResponse.bind(this),
	convertCandidateToDocument=this.defaultConvertCandidateToDocument.bind(this),
	candidatesForInitialSelection=2,
	candidatesForFinalSelection=1,
	generateRankingInstruction=this.defaultGenerateRankingInstruction.bind(this),
	extractIdentifiersFromRationale=this.defaultExtractIdentifiersFromRationale.bind(this),
	extractIdentifierFromCandidateDocument=this.defaultExtractIdentifierFromCandidateDocument.bind(this),
	candidateIdentifierField=undefined,
	findCandidateFromIdentifier=this.defaultFindCandidateFromIdentifier.bind(this),
	getSummarisableSubstringIndices,
	generatePromptTemplate=this.defaultGeneratePromptTemplate.bind(this),
	skipRationale=false
}:AICompareCandidates.CompareArguments<Candidate>=<AICompareCandidates.CompareArguments<Candidate>>{}):Promise<AICompareCandidates.CompareCandidatesReturn<Candidate>|void>

candidates: Candidate objects as you define them

problemDescription: The prompt for description of the problem, i.e. how you would like the AI model to select the candidates.

generateSearchAreasInstruction: A function for generating the instruction for determining the relevant subject areas to use to search the vector database.

parseSearchAreasResponse: A function to parse the search areas response returned by the generator.

convertCandidateToDocument: A function to convert a candidate object to a candidate document, i.e. a string, which can be embedded in the vector database.

candidatesForInitialSelection: The number of candidates which should be returned by the initial vector database search.

candidatesForFinalSelection: The number of candidates which should be finally selected by the text generation ranking model.

generateRankingInstruction: A function for generating the instruction for ranking the candidates returned by the initial vector database search.

extractIdentifiersFromRationale: A function for extracting the correct candidate object identifiers from the entire rationale generated by the ranking model.

extractIdentifierFromCandidateDocument: A function for extracting the correct candidate object identifier from a candidate document string.

candidateIdentifierField: The field in the candidate object which is used to identify it, e.g. name or ID number.

findCandidateFromIdentifier: A function for finding a candidate object from a provided identifier field.

getSummarisableSubstringIndices: A function for identifying the substring indices of a candidate document string which can be summarised by the summarisation model.

generatePromptTemplate: A function for generating the template for the prompt to the generator.

skipRationale: Whether to skip rationale generation.