Zync NestJS AI Module

A NestJS library that provides AI-powered OCR and data extraction using OpenAI and Google Gemini, with automatic deduplication and token usage tracking via MongoDB.

📦 Features

• OpenAI image understanding: Extract structured data from images using gpt-4o
• Google Gemini OCR: Extract structured JSON from ID-like documents using gemini-2.5-flash
• Smart caching: Images are hashed; repeated uploads return cached results from MongoDB
• Token accounting: Stores prompt and completion token usage for later analysis
• NestJS-ready modules: Plug-and-play OpenaiModule and GeminiModule
• Typed interfaces: Strongly typed interfaces for extraction responses

🚀 Installation

Local Development

# Clone the repository
git clone https://github.com/zynctech/zync-nest-ai-module.git
cd zync-nest-ai-module

# Install dependencies
pnpm install

# Build the library
pnpm run build

Using in Other Projects

Method 1: File Path Dependency (Recommended for pnpm)

In your project's package.json:

{
  "dependencies": {
    "zync-nest-ai-module": "file:../../zync-library/zync-nest-ai-module"
  }
}

Then run:

pnpm install

Method 2: npm link (For npm users)

# In the library directory
npm link

# In your consuming project
npm link zync-nest-ai-module

Method 3: pnpm link (Alternative for pnpm)

# Setup pnpm global bin directory (one-time setup)
pnpm setup

# In the library directory
pnpm link --global

# In your consuming project
pnpm link --global zync-nest-ai-module

📋 Peer Dependencies

This library uses peer dependencies to avoid version conflicts with your project. You must install the following in your consuming project:

Required Peer Dependencies

pnpm add \
  @nestjs/common@^11.1.6 \
  @nestjs/core@^11.1.6 \
  @nestjs/config@^4.0.2 \
  @nestjs/axios@^4.0.1 \
  @nestjs/mongoose@^11.0.3 \
  @nestjs/graphql@^13.1.0 \
  axios@^1.11.0 \
  graphql@^16.11.0 \
  moment@^2.30.1 \
  mongoose@^8.19.2 \
  mongoose-delete@^1.0.6 \
  multer@^2.0.2 \
  openai@^6.9.0 \
  randomatic@^3.1.1 \
  rxjs@^7.8.2 \
  sharp@^0.34.5 \
  zync-nest-data-module@^1.1.28

Or with npm:

npm install \
  @nestjs/common@^11.1.6 \
  @nestjs/core@^11.1.6 \
  @nestjs/config@^4.0.2 \
  @nestjs/axios@^4.0.1 \
  @nestjs/mongoose@^11.0.3 \
  @nestjs/graphql@^13.1.0 \
  axios@^1.11.0 \
  graphql@^16.11.0 \
  moment@^2.30.1 \
  mongoose@^8.19.2 \
  mongoose-delete@^1.0.6 \
  multer@^2.0.2 \
  openai@^6.9.0 \
  randomatic@^3.1.1 \
  rxjs@^7.8.2 \
  sharp@^0.34.5 \
  zync-nest-data-module@^1.1.28

This ensures:

• Single source of truth for framework and DB packages
• No duplicate installs between your app and the library
• Smaller bundle size and fewer version conflicts

🔧 Configuration

Environment Variables

# OpenAI configuration
open_ai_key=your_openai_api_key

# Google Gemini configuration
gemini_api_key=your_gemini_api_key

# MongoDB configuration for zync-nest-data-module
mongodb_uri=mongodb://localhost:27017/your_database

You must also configure the openaiDbConnection connection name in your zync-nest-data-module setup so that both OpenaiModule and GeminiModule can register their schemas against that connection.

📚 Usage

Module Setup

The library exposes two namespaces from the main entry:

import { Module } from '@nestjs/common';
import { Openai, Gemini } from 'zync-nest-ai-module';

@Module({
  imports: [
    Openai.OpenaiModule,
    Gemini.GeminiModule,
    // ...other modules
  ],
})
export class AppModule {}

Using OpenaiService for Extraction

OpenaiService accepts a file (with a buffer property) and returns sanitized structured data extracted by OpenAI.

import {
  Controller,
  Post,
  UploadedFile,
  UseInterceptors,
} from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { Openai } from 'zync-nest-ai-module';

@Controller('openai')
export class OpenaiController {
  constructor(
    private readonly openaiService: Openai.OpenaiService,
  ) {}

  @Post('extract')
  @UseInterceptors(FileInterceptor('file'))
  async extract(@UploadedFile() file: Express.Multer.File) {
    return this.openaiService.extractDataFromFile(file);
  }
}

Behavior:

• The image is preprocessed with sharp (resize + grayscale)
• A SHA-256 hash of the processed image is computed
• If a record with that hash already exists in MongoDB, the cached result is returned
• Otherwise, OpenAI gpt-4o is called and the result (plus token usage) is stored

Using GeminiService for Extraction

GeminiService works similarly, but uses Google Gemini and returns JSON for common ID fields.

import {
  Controller,
  Post,
  UploadedFile,
  UseInterceptors,
} from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { Gemini } from 'zync-nest-ai-module';

@Controller('gemini')
export class GeminiController {
  constructor(
    private readonly geminiService: Gemini.GeminiService,
  ) {}

  @Post('extract')
  @UseInterceptors(FileInterceptor('file'))
  async extract(@UploadedFile() file: Express.Multer.File) {
    return this.geminiService.extractDataFromFile(file);
  }
}

The Gemini prompt expects and returns JSON in this shape:

{
  "idNumber": "",
  "name": "",
  "address": "",
  "dateOfBirth": "",
  "gender": "",
  "religion": ""
}

Missing values should be returned as null by the model; the service will persist the raw JSON string and token usage.

Utility Helpers

The utils export includes helpers for working with files:

• readFileToBase64(filePath: string): read a file and return a base64 string
• readFileToBuffer(filePath: string): read a file into a Buffer
• deleteFile(filePath: string): delete a file from disk
• sanitizeExtractedText(text: string): convert line-based key: value text into a JSON object

These are already used internally by OpenaiService, but you can also use them in your own modules.

🛠️ Development

Scripts

# Build the library
pnpm run build

# Build in watch mode
pnpm run build:watch

# Clean build artifacts
pnpm run clean

# Clean and rebuild
pnpm run clean && pnpm run build

# Update library links in dependent projects
pnpm run update-links

# Start development server (example app)
pnpm run start:dev

Project Structure (Relevant Parts)

src/
├── gemini/
│   ├── gemini.module.ts      # Gemini NestJS module
│   ├── gemini.repository.ts  # MongoDB repository
│   ├── gemini.schema.ts      # Mongoose schema (collection: ai_gemini)
│   └── gemini.service.ts     # Google Gemini integration
├── openai/
│   ├── openai.interface.ts   # IExtractResult interface
│   ├── openai.module.ts      # OpenAI NestJS module
│   ├── openai.repository.ts  # MongoDB repository
│   ├── openai.schema.ts      # Mongoose schema
│   └── openai.service.ts     # OpenAI integration
├── app.module.ts             # Example application module
├── app.controller.ts         # Example controller
├── index.ts                  # Library entry (exports Openai, Gemini)
└── main.ts                   # Example bootstrap

🔍 Common Issues

• Missing Mongo connection: Ensure zync-nest-data-module is configured with an openaiDbConnection using mongodb_uri.
• Invalid or missing API keys: Check open_ai_key and gemini_api_key in your environment; 401/403 errors usually mean keys or scopes are wrong.
• Large or corrupt images: Very large files or non-image buffers can cause sharp to throw; validate uploads on the API boundary.
• Rate limits: OpenAI or Gemini may throttle; consider adding retry logic or queueing at the caller level if needed.

📝 License

ISC

👥 Author

sabiridwan

🔗 Repository

https://github.com/zynctech/zync-nest-ai-module

🐛 Issues

If you encounter any issues or have questions, please file an issue on the project repository.