@itentialopensource/adapter-dbt_cloud
v0.3.4
Published
This adapter integrates with system described as: dbtCloudApi.
Downloads
282
Readme
Adapter for Dbt_cloud
Table of Contents
Some of the page links in this document and links to other GitLab files do not work in Confluence however, the information is available in other sections of the Confluence material.
Specific to this Adapter
Authentication
Sample Properties
Sample Properties can be used to help you configure the adapter in the Itential Automation Platform. You will need to update connectivity information such as the host, port, protocol and credentials.
Available Calls
Swagger
Generic Adapter Information
Overview
Versioning
Itential Product and opensource adapters utilize SemVer for versioning. The current version of the adapter can be found in the package.json
file or viewed in the IAP GUI on the System page. All Itential opensource adapters can be found in the Itential OpenSource Repository.
Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to Dbt_cloud works appropriately.
Release notes can be viewed in CHANGELOG.md.
Supported IAP Versions
Itential Product adapters are built for particular versions of IAP and packaged with the versions they work with.
Itential opensource adapter as well as custom adapters built with the Itential Adapter Builder work acoss many releases of IAP. As a result, it is not often necessary to modify an adapter when upgrading IAP. If IAP has changes that impact the pronghorn.json, like adding a new required section, this will most likely require changes to all adapters when upgrading IAP.
Many of the scripts that come with all adapters built using the Itential Adapter Builder do have some dependencies on IAP or the IAP database schema and so it is possible these scripts could stop working in different versions of IAP. If you notify Itential of any issues, the Adapter Team will attempt to fix the scripts for newer releases of IAP.
Getting Started
These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.
Helpful Background Information
There is Adapter documentation available on the Itential Documentation Site. This documentation includes information and examples that are helpful for:
Authentication
IAP Service Instance Configuration
Code Files
Endpoint Configuration (Action & Schema)
Mock Data
Adapter Generic Methods
Headers
Security
Linting and Testing
Build an Adapter
Troubleshooting an Adapter
Others will be added over time. Want to build a new adapter? Use the Itential Adapter Builder
Prerequisites
The following is a list of required packages for installation on the system the adapter will run on:
Node.js
npm
Git
The following list of packages are required for Itential opensource adapters or custom adapters that have been built utilizing the Itential Adapter Builder. You can install these packages by running npm install inside the adapter directory.
If you are developing and testing a custom adapter, or have testing capabilities on an Itential opensource adapter, you will need to install these packages as well.
chai
eslint
eslint-config-airbnb-base
eslint-plugin-import
eslint-plugin-json
testdouble
How to Install
- Set up the name space location in your IAP node_modules.
cd /opt/pronghorn/current/node_modules (* could be in a different place)
if the @itentialopensource directory does not exist, create it:
mkdir @itentialopensource
- Clone/unzip/tar the adapter into your IAP environment.
cd @itentialopensource
git clone [email protected]:@itentialopensource/adapters/adapter-dbt_cloud
or
unzip adapter-dbt_cloud.zip
or
tar -xvf adapter-dbt_cloud.tar
- Run the adapter install script.
cd adapter-dbt_cloud
npm install
npm run lint:errors
npm run test
- Restart IAP
systemctl restart pronghorn
Create an adapter service instance configuration in IAP Admin Essentials GUI
Copy the properties from the sampleProperties.json and paste them into the service instance configuration in the inner/second properties field.
Change the adapter service instance configuration (host, port, credentials, etc) in IAP Admin Essentials GUI
Testing
Mocha is generally used to test all Itential Opensource Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.
Unit Testing
Unit Testing includes testing basic adapter functionality as well as error conditions that are triggered in the adapter prior to any integration. There are two ways to run unit tests. The prefered method is to use the testRunner script; however, both methods are provided here.
node utils/testRunner --unit
npm run test:unit
npm run test:baseunit
To add new unit tests, edit the test/unit/adapterTestUnit.js
file. The tests that are already in this file should provide guidance for adding additional tests.
Integration Testing - Standalone
Standalone Integration Testing requires mock data to be provided with the entities. If this data is not provided, standalone integration testing will fail. When the adapter is set to run in stub mode (setting the stub property to true), the adapter will run through its code up to the point of making the request. It will then retrieve the mock data and return that as if it had received that data as the response from Dbt_cloud. It will then translate the data so that the adapter can return the expected response to the rest of the Itential software. Standalone is the default integration test.
Similar to unit testing, there are two ways to run integration tests. Using the testRunner script is better because it prevents you from having to edit the test script; it will also resets information after testing is complete so that credentials are not saved in the file.
node utils/testRunner
answer no at the first prompt
npm run test:integration
To add new integration tests, edit the test/integration/adapterTestIntegration.js
file. The tests that are already in this file should provide guidance for adding additional tests.
Integration Testing
Integration Testing requires connectivity to Dbt_cloud. By using the testRunner script it prevents you from having to edit the integration test. It also resets the integration test after the test is complete so that credentials are not saved in the file.
Note: These tests have been written as a best effort to make them work in most environments. However, the Adapter Builder often does not have the necessary information that is required to set up valid integration tests. For example, the order of the requests can be very important and data is often required for
creates
andupdates
. Hence, integration tests may have to be enhanced before they will work (integrate) with Dbt_cloud. Even after tests have been set up properly, it is possible there are environmental constraints that could result in test failures. Some examples of possible environmental issues are customizations that have been made within Dbt_cloud which change order dependencies or required data.
node utils/testRunner
answer yes at the first prompt
answer all other questions on connectivity and credentials
Test should also be written to clean up after themselves. However, it is important to understand that in some cases this may not be possible. In addition, whenever exceptions occur, test execution may be stopped, which will prevent cleanup actions from running. It is recommended that tests be utilized in dev and test labs only.
Reminder: Do not check in code with actual credentials to systems.
Configuration
Additional Information
Enhancements
Contributing
Helpful Links
Adapter Technical Resources
Node Scripts
There are several node scripts that now accompany the adapter. These scripts are provided to make several activities easier. Many of these scripts can have issues with different versions of IAP as they have dependencies on IAP and Mongo. If you have issues with the scripts please report them to the Itential Adapter Team. Each of these scripts are described below.
Troubleshoot
License and Maintainers
Itential Product Adapters are maintained by the Itential Product Team.
Itential OpenSource Adapters are maintained by the Itential Adapter Team and the community at large.
Custom Adapters are maintained by other sources.