This post is about how I’ve setup PHPUnit on the local development environment and on GitHub Actions for Willow CMS. At the time of writing, Willow has 116 tests and 414 assertions. This gives me a reasonable level of confidence to ensure the reliability and quality of the codebase. Here are the key components of my testing setup how they contribute to what is on the way to being a well-tested application.

1. PHPUnit Configuration (phpunit.xml.dist)

The phpunit.xml.dist file is the main PHPUnit configuration. It defines important settings such as memory limits, error reporting, and the test suite structure. It’s configured to include the tests/TestCase/ directory, where all my test cases live. It also configures PHPUnit to use the CakePHP fixture extension, which simplifies database setup for testing. You get this for free with a CakePHP installation. One thing I have customised is the randomisation of tests with executionOrder="random". Randomizing the test execution order is a good way to ensure you don’t have hidden dependencies between tests. When tests are run in a fixed order, it’s possible for one test to inadvertently rely on the state or side effects of another test, and you’d never know this when executing the tests in the same order on every run. Good tests are truly independent and self-contained.

Another customisation (but one since removed in recent commits) is the inclusion of a clean-up class I’d written in PHP that ran after test execution. I was having issues with permissions between the Docker development environment and my host file system after running tests. Suffice to say, it is possible to use the phpunit.xml.dist file to configure PHP code that you want to run at various points of PHPUnit execution.

You can take a look at the configuration and code I had at the time to do this, and read more about PHPUnit extenions in the PHPUnit Documentation. Here’s a link to the phpunit.xml.dist file as it is today.

2. Continuous Integration with GitHub Actions (.github/workflows/ci.yml)

To automate my testing process, I’ve set up a continuous integration (CI) workflow using GitHub Actions. I found GitHub Actions quite fiddly at first as it required multiple rounds of making a configuration change, committing that change to the repository and then pushing up to GitHub to see if it worked. If you don’t know about GitHub actions read more here, it is brilliant and I’m really only scratching the surface.

The .github/workflows/ci.yml file defines the steps to be executed on a virtual machine in the cloud whenever I push changes to specific branches or create a pull request. The steps include installing software on the machine as well as configuration. Take a look at the file to see how I’m doing things like installing and starting MySQL, installing PHP dependencies with composer and copying CakePHP configuration files into place.

You only get one virtual machine and only so many minutes of execution time per month so you want this stage to be as lightweight and fast as possible. Unlike the Docker development environment (future blog post) I’m not setting up nice to haves like redis, PHPMyAdmin etc - this is just a single virtual machine with the bare essentials to enable setting up a MySQL database, installing PHP dependencies, running PHPUnit tests, and performing static analysis with tools like PHP CodeSniffer and PHPStan.

The end goal is that my code should be thoroughly tested and adhere to coding standards before it gets merged into the main branch.

3. Shared Test Case Code (tests/TestCase/AppControllerTestCase.php)

To keep my test code DRY (Don’t Repeat Yourself), I’ve created a reusable AppControllerTestCase class that extends the CakePHP TestCase. This class provides common functionality for controller tests, which for the moment is just simulating user authentication methods via the loginUser method. Take a look at the file.

Although I do have tests for the login method, I don’t want to have to perform that ‘proper’ login (literally posting test user credentials to the login action) every time I want to test a feature in the admin area. Test code should hone in on the specific thing being tested and not introduce code before or after that which we’re trying to test and in this case, that requires a mechanism to easily have a starting point with a logged in user.

Take a look at some tests in the user controller that simply call the loginUser method and then get straight into the actual test.

4. Controller Tests for the Imporant Stuff (tests/TestCase/Controller/UsersControllerTest.php)

Two critical controlelrs to to test are the UsersController and the ArticlesController for both the admin and non-admin site of Willow. The UsersControllerTest class covers a wide range of scenarios, including login, logout, registration, email confirmation, and user editing in both the admin area and front-end site for registered users.

Take a look at the files here and here. One improvement for the future would be to break these out into seperate test cases for the the admin and non-admin controllers. It’s good practice to test both successful and unsuccessful scenarios to ensure that the application behaves correctly - don’t just test the happy path. For example, I verify that both a non-admin user and admin user can log in, as well as testing that a non-admin user or non-logged in user is denied access to the admin area.

5. Code Coverage Report

You need to know where you are to better understand where you need to get to with testing. To gain insights into the coverage of my tests, I generate a code coverage report using PHPUnit. I have aliases set up in my dev_aliases.txt file to easily run PHPUnit with coverage options. Take a look here.

The phpunit_cov alias generates a text-based coverage report shown in the terminal, while phpunit_cov_html produces an HTML report in the webroot/coverage directory. These reports help me identify areas of the codebase that lack sufficient test coverage and guide me in writing additional tests.

By implementing these testing practices, I’ve achieved several benefits:

• Increased confidence in the stability and reliability of Willow CMS
• Early detection of bugs and regressions through automated testing - I know very quickly if my latest code has broken core functionality
• Improved code quality by enforcing coding standards and best practices
• Faster development cycles by catching issues early in the development process
• Easier collaboration with (hopefully) other developers through a well-defined testing strategy

Looking ahead, there are a few improvements I plan to make:

• Break out the test cases between admin and non admin features for key controllers
• Expand the test suite to cover more edge cases and complex scenarios as I add features
• Make more use of the local Jenkins Docker container that is part of the developer environment to run a suite of automated front-end tests via different browsers

So, current me absolutely thanks past me for putting the effort in with these tests, and I’m sure future me will thank current me for continuing to add to them. Testing is not just about catching bugs; it’s about building confidence in your code and enabling faster, more reliable development cycles.

Further reading:

• More about testing CakePHP applications with PHPUnit.
• More about PHPUnit

Happy testing!

Additional

Now you’re up to speed, how about reading on for the more detailed breakdown of the GitHub Actions ci.yml file.

Deep Dive - ci.yml for GitHub Actions

The ci.yml file has all the magic for automating the continuous integration process with GitHub Actions. Let’s break down its structure and functionality in detail. You can take a look at an actual run of the workflow here

Name and Trigger Events

The workflow is named CI, which stands for Continuous Integration. It is designed to run automatically on specific events:

• Push Events: The workflow triggers on pushes to the main, development, staging, and any branches that follow the feature/* naming convention. This ensures that any changes pushed to these branches are automatically tested.
• Pull Request Events: It also triggers on pull requests targeting the main, development, and staging branches. This helps in validating the changes before they are merged into these critical branches.

name: CI

on:
  push:
    branches:
      - main
      - development
      - staging
      - 'feature/*'
  pull_request:
    branches:
      - main
      - development
      - staging

Jobs and Environment

The workflow defines a single job named test, which runs on the ubuntu-latest environment. This ensures that the tests are executed in a consistent and up-to-date Linux environment.

jobs:
  test:
    runs-on: ubuntu-latest

Matrix Strategy

I’ve chosen a strategy matrix to test the application across multiple PHP versions (8.1, 8.2, 8.3). This just means the steps below are run against each version of PHP and this handled via GitHub Actions for us. We could add PHP 8.4 to the array if wanted.

strategy:
  matrix:
    php-version: ['8.1', '8.2', '8.3']

Checkout Code:

The actions/checkout@v4 action is used to clone the repository code into the runner environment. This is usually the first step in any CI process as it provides the codebase to work with.

steps:
- uses: actions/checkout@v4

Setup MySQL:

MySQL service is started, and a test database cms_test is created. Note that we don’t load any SQL to create database tables - that is handled by CakePHP running the migrations when we execute the tests because the phpunit.xml.dist file specifies the test specific bootstrap file to kick things off with the line bootstrap="tests/bootstrap.php"

- name: Setup MySQL
  run: |
    sudo service mysql start
    mysql -e 'CREATE DATABASE IF NOT EXISTS cms_test;' -uroot -proot
    mysql -e 'SHOW DATABASES;' -uroot -proot

Setup PHP:

The shivammathur/setup-php@v2 action is used to set up the specified PHP versions from the matrix one at a time. It also installs necessary PHP extensions like mbstring, intl, pdo_mysql, etc., and enables code coverage with xdebug.

- name: Setup PHP
  uses: shivammathur/setup-php@v2
  with:
    php-version: ${{ matrix.php-version }}
    extensions: mbstring, intl, pdo_mysql, pcntl, sockets, bcmath, zip
    coverage: xdebug

Install Composer Dependencies:

Composer is configured to use the PHP version from the matrix, and dependencies are installed. The --no-interaction, --prefer-dist, and --ignore-platform-reqs flags ensure a smooth and non-interactive installation process. This is the step that installs the CakePHP framework and other things Willow CMS uses into the /vendors folder.

- name: Install Composer dependencies
  run: |
    composer config platform.php ${{ matrix.php-version }}
    composer update --no-interaction --prefer-dist --ignore-platform-reqs

Copy Configs:

Configuration files are copied from a specified directory to the application’s config directory. This step ensures that the application is using the correct environment-specific configurations during the tests. Given the Docker development environment has multiple containers for different services and GitHub Actions gives the job a single VM, there are important configuration differences such as the MySQL server address. We could do more with environment variables for this, but for now, this is simple and works.

- name: Copy Configs
  run: |
    cp docker/github/cms_app_local_github.php config/app_local.php
    cp docker/github/app_github.php config/app.php

Debug app_local.php:

The contents of app_local.php are displayed. This is useful for debugging and verifying that the correct configuration is being used should something go wrong.

- name: Debug app_local.php
  run: cat "config/app_local.php"

Run PHPUnit:

PHPUnit tests are executed with error display enabled. The XDEBUG_MODE: coverage environment variable is set to collect code coverage data so that you can read the coverage stats in the output. I think in future I might turn this off as in practice they are not very easy to read on GitHub.

- name: Run PHPUnit
  run: php -d display_errors=on -d error_reporting=E_ALL vendor/bin/phpunit
  env:
    XDEBUG_MODE: coverage

Run PHP CodeSniffer:

PHP CodeSniffer is run to ensure that the code adheres to the CakePHP coding standards. This helps maintain code quality and consistency. It will pick up on white-space errors, yoda comparisons, PHP use statements that are out of alphabetical order, unused variables and more. Notice I onyl run this on the src and tests folders as they hold the pure PHP code and folders like templates are a mixture of PHP and HTML.

- name: Run PHP CodeSniffer
  run: vendor/bin/phpcs --standard=vendor/cakephp/cakephp-codesniffer/CakePHP src/ tests/

Run PHPStan:

PHPStan is used for static analysis of the code. It checks for potential errors and code quality issues. The continue-on-error: true option allows the workflow to proceed even if PHPStan finds issues, which can be useful for non-blocking feedback. You’ll see that I’ve configured the job to continue on error with this step as right now, focussing on writing tests and adhering to the CakePHP coding standards is good enough.

- name: Run PHPStan
  run: php -d memory_limit=-1 vendor/bin/phpstan analyse src/
  continue-on-error: true