sv-pubsub-client

Client for communicating with the Google Pub/Sub service.

Google Pub/Sub is the message broker typically used by microservices to communicate events that occur in one system to the other. These events are typically changes in data, e.g. accounts-update.

The most important concepts in Google Pub/Sub are Topics, Subscriptions and Messages.

Publishing a Message to a Topic without any Subscriptions will result in that message being lost. A Topic can and should be created by both Producers and Consumers of that Topic, to ensure that it exists before either end attempt to interact with it. Subscriptions are created from Topics.

These hold Messages, which Consumers consume via the persistent bidirectional connection it maintains with the Google Pub/Sub service. This is called a "streaming pull", not to be confused with polling/pull, which relies on the Consumer polling for new Messages on an interval.

Authorising with Google Pub/Sub

Authorisation should be done through service accounts. Each microservice typically has a service account. A service account will need permissions to create topics, subscriptions, attach to subscriptions and publish messages in order to authorise.

The credentials required are:

• projectId, e.g. sv-shared-231700
• credentials, i.e. the Application Default Credentials of your Google Service Account

Conventions

Topics should be named after the environment, producer, and event name of the events that pass through it. For example, if a Camber user updates an Account on the live app, we would be broadcasting that events to all Consumers of a Topic named live-camber-accounts-updated.

Subscriptions should be named after the consuming app, e.g. live-auth-live-camber-accounts-updated and set an appropriate retention period. Subscriptions are never automatically cleaned up, however the Messages stored within them are deleted after an hour, by default, although this can be changed.