Transcripted Summary

Every time we have run our tests in previous chapters, we have always run all tests using the python -m pytest command without any options or arguments.

In this chapter, we will learn about pytest's powerful command line options.

Pytest has several options available. We won't cover them all in this course, but we will cover many important ones.

# help

For starters, you can always learn about pytest's available command line options by using the --help or -h option:

python -m pytest --help

As is typical for help commands, pytest will list all available options with descriptions. pytest will also list options for its available plugins. We'll cover plugins in a later chapter.

# verbose

When running tests, you can control how much output pytest prints. I'm going to temporarily inject a failure into one of our test cases to demonstrate output differences:

def test_one_plus_one():
  assert 1 + 1 == 0

When we run the tests normally, we expect to see the pytest banner followed by a line for each module with dots and F's for test functions.

Pytest also outputs the failure introspection for each test.

However, when we run the tests with the --verbose option or -v, pytest prints more data.

At the top, it prints "cachedir" and metadata. It also prints a line for each test case, giving its path, name, and explicit PASSED or FAILED status.

Failure introspection is also printed.

The list of test case paths makes it easier to see what test cases are failing at a glance.

# quiet

We can also do the opposite of verbose.

When we run our tests with the --quiet option or -q, pytest doesn't print the top banner or even the test modules.

It prints only the dots and the F's. Even though the output is quiet, pytest still prints the failure introspection.

The quiet option is great when you want to avoid polluting your console with too many lines. It's particularly helpful when developing new tests.

# exitfirst

Pytest also provides a mechanism for stopping test execution after failures happen.

By default, pytest will run all tests it finds, no matter how many failures happen.

However, you might want to stop execution sooner.

To stop running tests after one failure, use the --exitfirst or -x option.

As soon as the first failure happens, pytest stops.

# maxfail

To stop running tests after a certain number of failures, use the --maxfail argument and set it to however many failures you want to stop after. For example:

python -m pytest --maxfail=1

These options can be helpful when debugging test cases so that you don't become overwhelmed by too many failures at once.

It can also be helpful when running very large test suites.

If, for example, a hundred tests fail in a suite of a thousand, there might be a system-wide problem.

Ending that test suite early would speed up the feedback loop and avoid needlessly burning resources for tests that would likely fail.

# junit-xml

The pytest console output is helpful when manually triggering tests, but sometimes you might need to save test results to a file.

Many tools like Continuous Integration servers need test result files in order to report test results.

One of the most common formats is JUnit's XML report form.

Pytest includes an option that will generate a JUnit XML file for test results.

Simply use --junit-xml and add a report file path like so:

python -m pytest --junit-xml report.xml

Note that the version of pytest I'm using for this course prints a warning message for this option, saying that it will be changed in upcoming releases.

We will make this warning go away in just a moment.

# Configuration Files

Now, let's talk about configuration. Command line options are great for running tests in specific ways on the fly.

However, there might be some options you want to set more permanently, especially when running tests in a Continuous Integration environment.

Some options are not available directly through the command line either.

Thankfully, you can create a configuration file for pytest to specify options in a file rather than on the command line.

Pytest supports a few different file formats for configuration files.

The most common one I've seen is "pytest.ini".

Whichever format you choose, be sure you follow appropriate conventions. Configuration files should be loaded in the project's root directory.

pytest configuration files support many options.

addopts is one of the most useful. It lets you set options directly as they would be entered on the command line.

Several other options exist as well, so take time to review them on the pytest docs online.

# Adding a Configuration File

Let's add a configuration file to our project. Create a new file named "pytest.ini" in the project root directory. Add the following lines and save the file:

junit_family = xunit2

This option should make that JUnit XML warning message go away. Re-run the tests with the JUnit XML option.

This time, the warning message isn't printed. Nice!

That's the end of this chapter.

Don't forget to undo the failure we deliberately injected at the beginning of the chapter.

As I said, there are more pytest options available than we can cover in this course. Please take time to review them on your own.

You never know what might be useful for your project. Resource links will be provided in the transcript.


© 2023 Applitools. All rights reserved. Terms and Conditions Privacy Policy GDPR