Syncables

This repository contains a sync engine that can be used to download (and in the future also update) a collection of objects from an API. It can be configured declaratively by extending the OpenAPI spec.

Find the path that can be used to fetch a (paged) collection of items. This will typically look something like this:

paths:
 /widgets:
    get:
      parameters:
        - description: Token specifying which result page to return. Optional.
          in: query
          name: pageToken
          schema:
            type: string
      responses:
        "200":
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/CalendarList"
components:
  schemas:
    CalendarList:
      properties:
        results:
          items:
            properties:
              backgroundColor:
                type: string
          type: array
        nextPageToken:
          description: Token used to access the next page of this result. Omitted if no further results are available.
          type: string
      type: object

Under paths['/widgets']['get']['responses']['200']['content']['application/json'], add an object syncable, in which you can specify:

• name: a string descriptor of the collection, e.g. "widgets"
• pagingStrategy: one of pageNumber, offset, pageToken or dateRange
• query: an object containing query parameters to add in addition to the paging-related ones
• for a pageNumber paging strategy, you can add pageNumberParamInQuery if it's not page.
• for a offset paging strategy, you can add offsetParamInQuery if it's not offset.
• for a pageToken paging strategy, you can add pageTokenParamInQuery if it's not pageToken and pageTokenParamInResponse if it's not pageToken
• for a dateRange paging strategy, you can add startDateParamInQuery if it's not startDate, endDateParamInQuery if it's not endDate, startDate if it's not '20000101000000', and endDate if it's not '99990101000000'

Usage

Create the OAD

You start with an OAD, for instance one from here. Probably the API you want to use is not specifying syncables yet, so you'll need to add those yourself using an overlay, something like the ones you see here. To compute the effect of the overlay you can run a command like:

./node_modules/.bin/overlayjs --openapi ./openapi/oad/acube-peppol.yaml --overlay ./openapi/overlay/acube-peppol-overlay.yaml > openapi/generated/acube.yaml

Generate the type

Assuming you're working in TypeScript, you can benefit from generating types from the OAD, like so:

npx openapi-typescript openapi/generated/acube.yaml -o ./src/types/acube.d.ts

Install syncables

Depending on your preferred package manager, you can run something like this to install syncables from npm:

pnpm install syncables

Write your code

Now you have the AOD with the definition of the syncable, and the type for the items you will sync, you can write code like this:

import { readFileSync } from 'fs';
import { components } from './src/types/google-calendar.js';
import { Syncable } from 'syncable';

type Entry = components['schemas']['CalendarListEntry'];
const specStr = readFileSync('./openapi/generated/google-calendar.yaml').toString();

const syncable = new Syncable<Entry>(specStr, {
  Authorization: `Bearer ${process.env.GOOGLE_BEARER_TOKEN}`
});
const data = await syncable.fullFetch();
console.log(data);

You can use the showcase-google-calendar branch of this repo to run a simple OAuth client that can obtain a value for the GOOGLE_BEARER_TOKEN environment variable.

Development

git clone https://github.com/tubsproject/syncables
cd syncables
pnpm install
pnpm generate
pnpm build
pnpm test
pnpm prettier
pnpm login
pnpm publish