Chapter 4 - Scaling Tests

Scaling Tests with Docker

Chapter 1 - Intro to Docker

Chapter 2 - Installing Docker

Chapter 3 - Setting up Docker

Chapter 5 - Scaling Containers

Transcript

Before we can start using our containers in our tests, we need to make sure that the tests are able to run concurrently. If you're watching this, I assume you already have your tests written and set up this way, but here are a couple of requirements:

Test Requirements and Tips

Tests are 100% independent. This means that Test B does not require Test A to run first.
The Driver should not shared. Each test needs to have its own instance of a Driver. This means anything dependent on the Driver, like page classes in the Page Object Model, must also follow this rule.
The test results and data will be saved to the machine running the Docker Engine or that is the main Manager node. That is currently my laptop, so my TestResult.xml file, screenshots, logs, etc. will be saved on my laptop. This obviously doesn't apply if you are running in a configuration where these are saved elsewhere like an S3 bucket.
It may be helpful to label files or directories with a unique hash, GUID, or timestamp since your tests will be run concurrently multiple times.

What are we Testing?

With that out of the way, let me introduce you to the example scenario that our tests will be covering.

The website is lolesports.com, but we'll be working on the Standings Page. To get to this page:

Go to lolesports.com
Go to Standings
Filter by League
Assert first place team is correct

Dive into Code

I'll show you a few classes so you have enough context on what this demo framework looks like and how it's going to run the tests, but you do not need to follow this exact structure for your tests to run on the containers.

The most important pieces are the RemoteWebDriver class and the URL we pass into its constructor to connect that driver to the Grid.

Let's start in the DriverFactory class!

RemoteWebDriver

DriverFactory.Build() requires a type of browser, either local to run on my laptop or remote to run on my Grid, and the browser I want to use, either Chrome or Firefox. If type equals "remote", the code will reach this else/if block and call BuildRemoteWebdriver().

This method uses the RemoteWebDriver class and requires that we pass in the URI to the port to our Grid, which is http://localhost:4444/wd/hub, and any other options like --start-maximized that we'd like to use as well. Then we return our newly created driver.

TestBase and Configuration

We're calling our DriverFactory.Build() method whenever we initialize a new Driver, and that's done in our TestBase class. This class has a Setup method, which is run before every test, and a Cleanup method, which is run after every test. Notice how we pass in a type and browser into our Driver.Init() method? Those are the values that determine the driver configuration and are located in an XML settings file called .runsettings that NUnit uses in its TestContext class.

Be default, we are using a remote chrome driver, but we can easily change these values and have them affect all of our test suites.

Test Suite Structure

Lastly, let's take a quick look at our Standings test suite. Each test is decorated with the Parallelizable attribute to they can run in parallel, and each test will have its own instance of a Driver and Page Objects. We then have them categorized for easy filtering.

Running the Tests

I want to run my demo tests locally first so you can see them actually running.

Run category=demo

Now we'll run those tests remotely on the Grid. I'll change my type variable to remote first and then make sure my Grid is down since we tore it down in the previous chapter.

Awesome. Let's spin it up with

docker-compose up -d

and now I'll run my tests via the dotnet test command:

dotnet test --settings .runsettings --filter testcategory=demo

We don't see the browsers open up anymore because they are running remotely on the Grid! While these are running, I want to quickly touch on two things:

That last command was specific to my tests, so make sure you run your tests as you normally would
Adapt this approach to however you or your team is configuring and setting up your tests. For example, you may have a different .txt or .json as your configuration and use pytest to run your tests.

Scale our Containers to Run More Tests

Voila! The tests are done and now we know how to connect to our Grid and run our tests. However, we're currently only running two tests. This is convenient with our single container with 2 chrome browsers, but we have more than 2 tests to run.

Docker handles this easily by using the docker-compose.yml we already created! In the terminal, run the following:

docker-compose up -d --scale chrome=10

Once complete, run

docker ps -a.

We will see a total of 10 chrome containers now! And that was just in seconds! Let's head back over to our more visual view of the grid in the browser. Sure enough, there are now 10 chrome containers with 2 browsers each!

Let's tear down our current structure with

docker-compose down

I have 12 tests in my PlayerStats suite, so I want to spin up my Grid with 6 chrome containers. I'm sure you already know what to run in the terminal:

docker-compose up -d --scale chrome=6

What if I want my docker-compose to just give me 6 chrome containers with 2 browsers each and no Firefox containers from the get go? This is easily done by adding the replicas field to our yaml file. Include replicas dash six, comment out the Firefox service, save the file, and then run:

docker-compose up -d

Sure enough, we now have the hub with 6 chrome containers.

You are now able to run as many tests as your machine can handle by scaling your containers however you need. Hopefully you see how simple Docker and containers have made it to spin up, restart, and teardown entire grids with just a few commands.

Resources

Add AI to your existing test scripts in minutes!