mabl Tools for Playwright

Introduction

Leverage the power of the mabl cloud from within their own Playwright tests. By using mabl Tools for Playwright, developers can easily introduce advanced test functionality such as email, database, generative AI testing, and more.

Email testing

mabl Tools for Playwright gives developers the ability to create and use temporary emails as well as long-lived emails for use within their Playwright tests. With these abilities, developers are able to create full end-to-end functional tests that validate sign-up and login functionality without the overhead of managing their own email servers and accounts.

Generative AI testing

With the generative AI testing functionality of mabl Tools for Playwright, developers can easily leverage the power of AI to do more complex validations of their test environments without complex test code. Using the tools, developers can implement complex assertions in their tests that validate images, analyze text (translations, tone, sorting, etc), validate chatbot interactions, and a whole lot more.

Multi-factor authentication and credentials testing

mabl Tools for Playwright allows developers to create functional end-to-end tests that rely on securely stored and managed passwords in the mabl cloud. With these tools, developers have a central place to manage credentials securely outside their codebase and leverage more sophisticated login flows that require generating two-factor authentication codes to complete a login. This functionality also works alongside the email testing tools for cases where login flows require opening emails to extract MFA codes or follow login links.

Database testing

Mabl’s database connectivity tool gives developers secure access to test databases connected to the mabl cloud, allowing them to manage test environment DB state as well as validate it all without having to install database drivers and manage test environment networking. Developers can write setup and teardown code from within their Playwright tests that ensure the database state is correct for executing tests. They can also assert during their tests that the correct state has been updated as expected, offering more depth to the validations their Playwright tests are capable of.

PDF Testing

mabl Tools for Playwright allows developers to load PDF documents downloaded during test execution into a Playwright tab as testable HTML. This allows for Playwright tests to not only validate the PDF downloaded but also validate the contents of the document are what is expected.

Quickstart

Before you begin

• Sign up or log in to your mabl workspace
• Create an API key to use in your test suite
  • Navigate to Settings > APIs and click “Create API key”
  • Select “Command Line Interface” as the key option, give it a name, and click “Save”

Prerequisites

To leverage mabl Tools for Playwright, you will need a developer environment with:

• NodeJS >= 18.0.0
• Playwright test suite

Installing mabl Tools for Playwright

npm install @mablhq/playwright-tools

Using the mabl tools

You can set up mabl Tools for Playwright inside your Playwright Test Fixtures for use across the entire test suite by importing and calling the createMablFixtures function inside the Playwright extend function.

import {test as base} from '@playwright/test';

// Import the initialization function for mabl Tools for Playwright
import {createMablFixtures, MablFixtures} from '@mablhq/playwright-tools';

// For TypeScript you can include the MablFixtures typing when extending
export const test = base.extend<MablFixtures>({

  // existing fixture configuration can remain in place

  // this function will generate all the mabl Fixtures and configure them to use the supplied API key 
  ...createMablFixtures({
    apiKey: process.env.MABL_PLAYWRIGHT_TESTS_KEY,
    test: base,
  }),
});

export {expect} from '@playwright/test';

With the fixtures configured, you will now have access to the mabl object inside your test cases, which allows you to access any of the mabl tools.

Examples

Email testing

import {SandboxPage} from '../utils/SandboxPage';
import {expect, test} from '../utils/fixtures';

test('Validate emails can be received using mabl sandbox app', async ({
  page,
  mabl,
}) => {
  await page.goto('https://sandbox.mabl.com');
  const sandbox = new SandboxPage(page);
  sandbox.maybeWaitForReady();
  const mailboxPage = await sandbox.goToMailboxPage();
  // Create a temporary mailbox for the test case using the mabl tools
  const emailAddress = await mabl.createMailBoxAddress();
  const subject = 'Test email';
  const body = 'This is a test email';
  // This uses the email as a destination address inside the test case 
  await mailboxPage.fillOutAndSendEmail(emailAddress, subject, body);

  // Leverage the mabl Tooset to wait for an email to appear in the temporary mailbox generated earlier in the test.
  const mablEmail = await mabl.waitForEmail(emailAddress, page, {subject});
  // Open the email in the supplied Playwright Page
  await mablEmail.open();

  // The mabl tools returns a MablMailboxEmail object that allows for easy access to all the received email attributes for faster validation 
  expect(await mablEmail.getToAddress()).toBe(emailAddress);
  expect(await mablEmail.getSubject()).toBe(subject);
  expect(await mablEmail.getBodyText()).toBe(body);
  expect(await mablEmail.getFromAddress()).toBe('[email protected]');
});

Generative AI testing

Validating page state

import {expect, test} from '../utils/fixtures';

test('Evaluate a GenAI assertion to validate the state of a page', async ({
  page,
  mabl,
}) => {
  await page.goto('https://www.mabl.com');
  // Create a prompt to supply to the Gen AI tool
  const prompt =
    'Please validate that the web page you are looking at is a marketing page for a company named mabl which does automated UI testing. Also validate that there are no broken images on the page';

  // Supply the Playwright Page to the evaluate function so mabl can extract a screenshot and other information to use with the prompt 
  const result = await mabl.evaluateGenAiAssertion(page, prompt);
  
  // The result returned includes both a success (true/false) attribute as well as an explanation for the evaluation
  console.log(`GEN AI EXPLANATION: [${result.explanation}]`);
  expect(result.success).toBe(true);
});

Validating downloaded files

import {expect, test} from '../utils/fixtures';

test('Evaluate a GenAI assertion with a downloaded file', async ({
  page,
  mabl,
}) => {
  await page.goto('https://sandbox.mabl.com/downloads-pdfs');

  // Wait for the download button and trigger download
  const downloadPromise = page.waitForEvent('download');
  await page.getByText('Download CSV').click();
  const download = await downloadPromise;

  // Use the mabl tools to evaluate the downloaded file content with a GenAI assertion
  // This works with various file types including CSV, text files, and more
  const result = await mabl.evaluateGenAiAssertionInFile(
    page,
    'Outdoor Patio Chair is in stock',
    download,
  );

  // The result includes both success status and an explanation of the evaluation
  console.log(`GEN AI FILE EXPLANATION: [${result.explanation}]`);
  expect(result.success).toBe(true);
});

Database testing

import {expect, test} from '../utils/fixtures';

// Database connection ID from your mabl workspace
const MABL_DATABASE_CONNECTION = 'grm9yi7lFN36WMjCQ4K2jg-dc';

test('Evaluate a mabl database query', async ({page, mabl}) => {
  const dbQuery = 'SELECT * from pets LIMIT 15';
  const result = await mabl.executeQuery(MABL_DATABASE_CONNECTION, dbQuery, []);
  expect(result.metadata?.status).toBe('success');
  expect(result.columns?.length).toEqual(5);
  expect(result.rows?.length).toEqual(15);
});

PDF testing

test('Validate PDFs downloaded from the app', async ({
  browser,
  page,
  mabl,
}) => {
  await page.goto('https://sandbox.mabl.com');
  await page.waitForSelector("a:has-text('downloads and pdfs')");
  // click the downloads and pdfs button
  await page.click("a:has-text('downloads and pdfs')");

  // get ready for the download
  const downloadPromise = page.waitForEvent('download');
  
  // click the download button/link
  await page.click("a:has-text('download pdf')");

  // wait for download to complete
  const download = await downloadPromise;
  const pdfPath = await download.path();

  // create a Page for the downloaded PDF to open in
  const pageForPdf = await browser.newPage();

  // use the mabl tools to open the file in a new Page 
  await mabl.openPdfFile(pdfPath, pageForPdf);

  // validate the contents of the PDF
  const element = pageForPdf.locator('text=Journey run export');
  expect(await element.isVisible()).toBeTruthy();
});

Multi-factor Authentication and credentials testing

You will need the IDs for any credentials you’d like to use in your tests. These can be found in the mabl app by going to Configuration > Credentials and clicking on the credential you’d like to use. The View mabl CLI IDs utility can be found in the upper left of the credentials details UI.

You can also use the mabl CLI from your terminal to list the credentials and their IDs using the command $ mabl credentials list

import {expect, test} from '../utils/fixtures';
import {SandboxPage} from '../utils/SandboxPage';

// This is the ID of the mabl credentials to use in a test
const MFA_CREDENTIALS_ID = 'L7HamCgVXVY4UbelLPTRzA-c';

test('Validate that we can get an MFA token from mabl', async ({
  page,
  mabl,
}) => {
  await page.goto('https://sandbox.mabl.com');
  const mfaToken = await mabl.getMfaCode(MFA_CREDENTIALS_ID);
  // now use the MFA token inside your test
});

test('Validate that we can get and use credentials from mabl', async ({
  page,
  mabl,
}) => {
  await page.goto('https://sandbox.mabl.com');
  const sandbox = new SandboxPage(page);
  sandbox.maybeWaitForReady();
  const loginPage = await sandbox.goToLoginPage();

  // fetches the latest version of the credentials from mabl
  const creds = await mabl.getCredentials(MFA_CREDENTIALS_ID);
  await loginPage.fillOutAndSubmitLoginForm(creds.username, creds.password);
});