Building a Scalable Test Automation Framework for Large Applications: TypeScript, Playwright, Screenplay & Serenity BDD
Testing large, complex web applications is a monumental challenge. As applications grow, so does the test suite. Traditional test automation frameworks, often built on the Page Object Model (POM), can become brittle, hard to maintain, and difficult for non-technical stakeholders to understand.
What if we could build a framework that was not only robust and scalable but also produced living documentation that the entire team—from developers to business analysts—could use?
In this post, we'll build a production-grade test automation framework from the ground up. We'll combine the power of modern tools and patterns to create a solution that is maintainable, readable, and ready for the demands of large-scale applications.
Table of Contents
The Dream Team: Our Technology Stack
We've carefully selected a stack where each component solves a specific problem, creating a synergistic whole.
- TypeScript: A statically-typed superset of JavaScript. It brings type safety, improved autocompletion, and easier refactoring to our framework, which is crucial for large codebases.
- Playwright: A modern, fast, and reliable browser automation library from Microsoft. Its auto-waits, powerful tooling, and cross-browser support make it a top choice for testing today's dynamic web apps.
- The Screenplay Pattern: A user-centric approach to modeling automated tests. Instead of focusing on pages, it focuses on Actors who have Abilities (like browsing the web) and perform Tasks (like logging in). This makes tests more readable and highly composable.
- Serenity/JS: An open-source framework that elegantly implements the Screenplay Pattern and integrates seamlessly with tools like Playwright and Cucumber. Its killer feature is its ability to generate rich, illustrated reports that act as living documentation.
- CSV Data-Driven Testing: A straightforward method to separate test data from test logic. This allows us to run the same test scenario with multiple data sets, increasing coverage without duplicating code.
Why This Stack? The Core Concepts
Traditional frameworks often mix what the user does with how the code interacts with the page. The Screenplay Pattern, facilitated by Serenity/JS, solves this by creating clear layers of abstraction:
- Actors: Represent users with different roles (e.g.,
Admin,StandardUser). - Abilities: What an Actor can do (e.g.,
BrowseTheWebusing Playwright, orCallAnAPI). - Tasks: High-level, business-focused actions (e.g.,
LoginWithValidCredentials). Tasks are composed of other Tasks or low-level Interactions. - Interactions: Low-level actions that directly use an Ability (e.g.,
Click,Enter). - Questions: Ask for information about the state of the application (e.g.,
Text.of(Header)).
This composition-over-inheritance model prevents the deep, brittle inheritance chains often seen in POMs, making the framework far more flexible and maintainable.
Setting Up the Project: The Blueprint
Let's get our hands dirty and set up the project structure.
Prerequisites:
- Node.js (v16 or newer)
- An IDE like Visual Studio Code
Step 1: Initialize the Project
Create a new directory and initialize a Node.js project.
mkdir serenity-playwright-framework
cd serenity-playwright-framework
npm init -yStep 2: Install Dependencies
We'll install all the necessary Serenity/JS modules, Playwright, Cucumber, and TypeScript tooling.
npm install --save-dev \
@serenity-js/core \
@serenity-js/cucumber \
@serenity-js/playwright \
@serenity-js/assertions \
@serenity-js/console-reporter \
@serenity-js/serenity-bdd \
@cucumber/cucumber \
playwright \
typescript \
ts-node \
rimrafrimraf: A utility to clean directories, useful for our report generation.
Step 3: Configure TypeScript (tsconfig.json)
Create a tsconfig.json file in the root of your project. This tells the TypeScript compiler how to translate our .ts files into JavaScript.
{
"compilerOptions": {
"target": "ES2021",
"module": "commonjs",
"lib": ["es2021", "dom"],
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"strict": true,
"skipLibCheck": true,
"sourceMap": true,
"outDir": "target/ts"
},
"include": [
"src/**/*.ts",
"features/**/*.ts",
"serenity.config.ts"
]
}Step 4: Configure Serenity/JS (serenity.config.ts)
This is the heart of our framework's configuration. Create a serenity.config.ts file. It tells Serenity/JS which test runner to use (Cucumber), what reports to generate, and how to configure our actors.
// serenity.config.ts
import { ConsoleReporter } from '@serenity-js/console-reporter';
import { ArtifactArchiver, serenity } from '@serenity-js/core';
import { SerenityBDDReporter } from '@serenity-js/serenity-bdd';
import { photographer, PlaywrightNeeded } from '@serenity-js/playwright';
import { resolve } from 'path';
import { Actors } from './src/actors';
export const config = {
// Specify the test runner configuration
testRunner: require.resolve('@serenity-js/cucumber'),
// Configure the test runner
cucumber: {
// Paths to your feature files
spec: ['features/**/*.feature'],
// Paths to your step definitions
'require': [
'features/step-definitions/**/*.ts'
],
// Tell Cucumber to use ts-node to run TypeScript files
'require-module': ['ts-node/register'],
},
// Define the crew of reporters
crew: [
// Produce Serenity BDD JSON reports
new SerenityBDDReporter(),
// Produce human-readable output to the console
ConsoleReporter.forDarkTerminals(),
// Capture artifacts like screenshots and browser logs
ArtifactArchiver.storingArtifactsAt(resolve(__dirname, 'target/site/serenity')),
],
// Configure actors
actors: new Actors(),
// Define the abilities actors will have
spawns: async (actor) => actor.whoCan(
PlaywrightNeeded.usingAnExistingBrowser(),
),
};Step 5: Add Scripts to package.json
Let's add some scripts to our package.json to make running tests and generating reports easy.
"scripts": {
"clean": "rimraf target",
"test:execute": "serenity-js run --config serenity.config.ts",
"test:report": "serenity-js report",
"test": "npm run clean && npm run test:execute && npm run test:report"
}Building the Framework: A Production-Grade Example
We'll automate a common scenario on the popular practice site, SauceDemo: logging in and verifying that products are displayed. We'll use a data-driven approach to test both a standard user and a locked-out user.
Final Directory Structure
Here’s what our project will look like:
.
├── features
│ ├── step-definitions
│ │ └── login.steps.ts
│ └── login.feature
├── node_modules
├── src
│ ├── actors.ts
│ ├── data
│ │ └── users.csv
│ ├── screen
│ │ ├── LoginPage.ts
│ │ └── ProductsPage.ts
│ ├── tasks
│ │ ├── Login.ts
│ │ └── SeeProducts.ts
│ └── questions
│ └── ErrorMessage.ts
├── package.json
├── serenity.config.ts
└── tsconfig.jsonThe Feature File: Defining Behavior
Create features/login.feature. We use a Scenario Outline with an Examples table to define our data-driven test.
# features/login.feature
Feature: User Login
As a user of the SauceDemo website,
I want to be able to log in
So that I can access the product inventory.
Scenario Outline: User attempts to log in with their credentials
Given is a registered user
When attempts to log in with their credentials from the row
Then should
Examples: User Data
| row | actor | outcome |
| 1 | Stan | see the product inventory |
| 2 | Lucy | be told that they are locked out |
The Actors: Who is Performing the Actions?
Create src/actors.ts. This class will be responsible for providing actors to our tests, giving them the ability to browse the web.
// src/actors.ts
import { Actor, Cast } from '@serenity-js/core';
import { BrowseTheWebWithPlaywright } from '@serenity-js/playwright';
import { CallAnApi } from '@serenity-js/rest'; // Example for future API tests
import { Browser, chromium } from 'playwright';
export class Actors implements Cast {
private browser: Browser;
constructor() {
// We launch the browser once for all actors
this.browser = chromium.launch({ headless: true });
}
// This method is called by Serenity/JS for each actor from the feature file
prepare(actor: Actor): Actor {
return actor.whoCan(
BrowseTheWebWithPlaywright.using(this.browser),
// Example of how you'd add an API ability
// CallAnApi.at('https://api.example.com')
);
}
}UI Targets: Locating Elements Intelligently
Let's define our page element locators. This keeps them separate from our test logic.
Create src/screen/LoginPage.ts:
// src/screen/LoginPage.ts
import { By, PageElement } from '@serenity-js/web';
export const LoginPage = {
usernameField: () => PageElement.located(By.id('user-name')).describedAs('username field'),
passwordField: () => PageElement.located(By.id('password')).describedAs('password field'),
loginButton: () => PageElement.located(By.id('login-button')).describedAs('login button'),
};Create src/screen/ProductsPage.ts:
// src/screen/ProductsPage.ts
import { By, PageElement } from '@serenity-js/web';
export const ProductsPage = {
header: () => PageElement.located(By.css('.title')).describedAs('products page header'),
inventory: () => PageElement.located(By.id('inventory_container')).describedAs('product inventory'),
};Tasks: The "How" - Composable User Actions
Tasks encapsulate the steps a user takes to achieve a goal.
Create src/tasks/Login.ts:
// src/tasks/Login.ts
import { Task } from '@serenity-js/core';
import { Click, Enter, Navigate } from '@serenity-js/web';
import { LoginPage } from '../screen/LoginPage';
export const Login = {
withCredentials: (username: string, password: string): Task =>
Task.where(`#actor logs in with username "${username}"`,
Navigate.to('https://www.saucedemo.com/'),
Enter.theValue(username).into(LoginPage.usernameField()),
Enter.theValue(password).into(LoginPage.passwordField()),
Click.on(LoginPage.loginButton()),
),
};Create src/tasks/SeeProducts.ts:
// src/tasks/SeeProducts.ts
import { Ensure, isPresent } from '@serenity-js/assertions';
import { Task } from '@serenity-js/core';
import { ProductsPage } from '../screen/ProductsPage';
export const SeeProducts = (): Task =>
Task.where(`#actor sees the product inventory`,
Ensure.that(ProductsPage.inventory(), isPresent()),
Ensure.that(ProductsPage.header(), isPresent()),
);Questions: The "What" - Verifying the State
Questions retrieve information from the application for assertions.
Create src/questions/ErrorMessage.ts:
// src/questions/ErrorMessage.ts
import { Question } from '@serenity-js/core';
import { By, PageElement, Text } from '@serenity-js/web';
export const ErrorMessage = {
displayed: (): Question> =>
Text.of(PageElement.located(By.css('[data-test="error"]')))
.describedAs('the error message'),
}; The Data Source: Driving Tests with CSV
Create src/data/users.csv. The headers (username, password) must match the variable names we will use in our step definitions.
username,password
standard_user,secret_sauce
locked_out_user,secret_sauceThe Step Definitions: Gluing It All Together
This is where we connect our Gherkin feature file to our Tasks and Questions.
Create features/step-definitions/login.steps.ts:
// features/step-definitions/login.steps.ts
import { Given, When, Then } from '@cucumber/cucumber';
import { actorCalled, actorInTheSpotlight } from '@serenity-js/core';
import { Ensure, equals } from '@serenity-js/assertions';
import { readFileSync } from 'fs';
import { parse } from 'csv-parse/sync';
import { Login } from '../../src/tasks/Login';
import { SeeProducts } from '../../src/tasks/SeeProducts';
import { ErrorMessage } from '../../src/questions/ErrorMessage';
// Load and parse the CSV data
const users = parse(readFileSync('src/data/users.csv', 'utf-8'), {
columns: true,
skip_empty_lines: true
});
Given('{actor} is a registered user', (actor: string) => {
return actorCalled(actor).attemptsTo(
// This is a great place for setup tasks, like ensuring a user exists via an API call
);
});
When('{actor} attempts to log in with their credentials from the row {int}', (actor: string, rowIndex: number) => {
const user = users[rowIndex - 1]; // CSV is 0-indexed, row is 1-indexed
return actorCalled(actor).attemptsTo(
Login.withCredentials(user.username, user.password)
);
});
Then('{actor} should see the product inventory', () =>
actorInTheSpotlight().attemptsTo(
SeeProducts()
)
);
Then('{actor} should be told that they are locked out', () =>
actorInTheSpotlight().attemptsTo(
Ensure.that(ErrorMessage.displayed(), equals('Epic sadface: Sorry, this user has been locked out.'))
)
);Running the Tests and Generating Reports
You're all set! Run the tests from your terminal:
npm testAfter the run completes, a target directory will be created. Inside target/site/serenity, you'll find an index.html file. Open it in your browser.
You will be greeted by a stunning, detailed report:
- Dashboard: An overview of test results, execution times, and feature coverage.
- Living Documentation: Each feature file is rendered as a readable document.
- Drill-Down Details: Click on any test scenario to see a step-by-step breakdown. Each step shows the actions performed, the data used, execution time, and a screenshot taken automatically at the end of the step!
This report is invaluable. It provides absolute clarity on what was tested, how it was tested, and why it failed, complete with visual proof.
Scaling Up: Best Practices for Large Projects
This framework is built for scale. Here's how to keep it clean as your project grows:
- Organize by Domain: Instead of a single
tasksorscreenfolder, structure yoursrcdirectory by business capability or feature domain (e.g.,src/authentication,src/product_search,src/checkout). Each domain folder would contain its own tasks, questions, and UI definitions. - Use Custom Abilities: What if an actor needs to interact with a REST API to set up test data? You can create a custom
CallAnApiability. The Screenplay Pattern is not limited to web UIs. - CI/CD Integration: The
npm testcommand can be directly plugged into any CI/CD pipeline (GitHub Actions, Jenkins, CircleCI). The reports can be archived as build artifacts for easy access. - Parallel Execution: Serenity/JS and Playwright are designed for parallel execution. You can configure your test runner to run features in parallel, dramatically reducing feedback time.
Conclusion
We've moved beyond simple scripts and built a true automation framework. By combining TypeScript's safety, Playwright's power, the Screenplay Pattern's clarity, and Serenity/JS's reporting brilliance, we've created a system that is:
- Scalable: The compositional nature of Tasks allows you to build complex user journeys from simple, reusable blocks.
- Maintainable: Separation of concerns is at its core. Changing a UI locator doesn't require changing your business logic.
- Readable: Tests are written in a business-centric language that everyone on the team can understand.
- Trustworthy: The detailed, screenshot-rich reports provide undeniable proof of the application's behavior, bridging the gap between development and business.
This blueprint provides a powerful foundation. It empowers teams to build tests that are not a maintenance burden but a valuable asset, providing clarity and confidence with every single run.
Comments