BatchJS-Data

Extension of Batch JS adding data storage support for databases.

• BatchJS-Data
• Download
  • NPM
  • Yarn
• Usage
• Documentation
• Collaborators welcome!

Download

NPM

npm install batchjs-data --no-optional
npm install sqlite sqlite3  #For SQLite implementation
npm install mariadb         #For MariaDB implementation
npm install mysql2          #For MySQL implementation
npm install pg @types/pg    #For PostgreSQL implementation

Yarn

yarn add batchjs-data --no-optional
yarn add mariadb         #For MariaDB implementation
yarn add mysql2          #For MySQL implementation
yarn add pg @types/pg    #For PostgreSQL implementation
yarn add sqlite sqlite3  #For SQLite implementation

Usage

1. Create your reader

   import sqlite3 from "sqlite3";
   import  {open} from "sqlite";
   import { SqliteBatchEntityReader } from "batchjs-data/sqlite";
   import { UserDTO } from "./UserDTO";
   
   export class UserBatchReader extends SqliteBatchEntityReader<UserDTO,UserDTO> {
       constructor(options:{batchSize:number,query?:string}) {
           super({
               batchSize: options.batchSize,
               dbConnectionFactory: () => { return open({filename: './database.db', driver: sqlite3.Database});},
               query: options.query || "SELECT id, username FROM users",
               rowToEntity: (row: UserDTO) => row
           });
       }
   }

2. Create your writer

   import sqlite3 from "sqlite3";
   import sqlite, {open} from "sqlite";
   import { SqliteBatchEntityWriter } from "batchjs-data/sqlite";
   import { UserDTO } from "./UserDTO";
   
   export class UserBatchWriter extends SqliteBatchEntityWriter<UserDTO> {
       constructor(options:{batchSize:number}){
           super({
               batchSize: options.batchSize,
               dbConnectionFactory: () => { return open({filename: './database.db', driver: sqlite3.Database});},
               prepareStatement: "INSERT INTO users (id, username) VALUES (@id, @username)",
               saveEntity:(entity: UserDTO, stmt: sqlite.Statement)=>stmt.all<void>({'@id': entity.id, '@username': entity.username})
           });
       }
   }

3. Use them in your BatchJS Job

   import { Job, Step } from "batchjs";
   
   // Implement a step
   class StepImplementation extends Step {
       // Set a name to the step
       constructor(name: string = "DemoStep") {
           super(name);
       }
   
       // Implement the reader to load step data source
       protected _reader() {
           return new UserBatchReader({batchSize:2});
       }
   
       // Implement the processors to transform data sequently using our streams or your own streams
       protected _processors() {
           const opts: TransformOptions = {
               objectMode: true,
               transform(
                   chunk: unknown,
                   encoding: BufferEncoding,
                   callback: TransformCallback
               ) {
                   this.push(chunk);
                   callback();
               },
           };
           return [new Transform(opts), new Transform(opts)];
       }
   
       // Implement the write to stock final step data
       protected _writer() {
           return new UserBatchWriter({batchSize:2})
       }
   }
   
   // Implement a Job
   class JobImplementation extends Job {
       // Implement to set the steps to be sequently executed.
       protected _steps() {
           return [new StepImplementation(), new StepImplementation()];
       }
   }
   
   // Instance the Job
   const job = new JobImplementation("My job");
   
   // Set events listener
   job.on("stepStart", (step: step) => {
       console.log(`Starting step ${step.name}`);
   });
   
   // Launch the job
   job.run()
       .then(() => {
           console.log("Job completed successfully");
       })
       .catch((error) => {
           console.log("Job completed with errors");
       });

Documentation

• Core API
• MariaDB API
• MySQL API
• PostgreSQL API
• SQLite API

Collaborators welcome!

• ¿Do you like the project? Give us a :star: in GitHub.
• :sos: ¿Do you need some help? Open a discussion in GitHub help wanted
• :bug: ¿Do you find a bug? Open a issue in GitHub bug report
• :bulb: ¿Do you have a great idea? Open a issue in GitHub feature request
• :computer: ¿Do you know how to fix a bug? Open a pull request in GitHub pull request.
• ¿Do you know a security issue? Take a read to our security strategy.

Subscribe our code of conduct and follow the Contribution Guidelines.