There is a difference between testing code and testing features.
So far in this course, all of our tests have been what we call "unit" tests (or "white box" tests). They directly test code. The math tests exercise Python's map operators, and the Accumulator tests exercise the behaviors of a class we created.
Testing code directly is very important for several reasons. Unit tests are typically small and simple to write. Unit tests catch low-level problems at the source where they're easy to identify and fix. And unit tests run very quickly, meaning you can write several unit tests to maximize coverage.
However, pytest can also handle what we call "feature" tests (alternately: "black box" tests, "integration tests", "end-to-end" tests, "system" tests, or a whole bunch of other similar names).
Instead of making direct calls to the product code, they interact with a live instance of the product.
For example, calling a REST API and loading a web page in a browser would both be examples of feature tests.
Feature tests are important because they test the product in ways that it would actually be used. For instance, all of the individual methods in the code may work as intended, but building and running everything together might expose integration problems.
Feature testing has downsides, though:
Feature tests require extra setup because they need a live product instance, whether that's the deployed service, a web app, or something else.
Feature tests are prone to interruptions and intermittent errors due to their dependencies on the systems under test.
Feature tests are also slower than unit tests because they need to wait for the product under test to respond.
Nevertheless, testing both the code and the features is important. Each type of test mitigates unique types of risks.
In this chapter, we will write a simple REST API test using pytest to show that pytest can handle feature tests as well as unit tests.
REST APIs are just web services. Callers send requests, and the service returns data.
To call REST APIs in Python, we will use the
requests is one of the most popular Python packages of all time. It provides a simple API for calling APIs.
Please note that the purpose of this chapter is not to teach you how to automate tests for REST APIs. That topic needs its own course.
Instead, this chapter will walk through a very simple REST API test to show you how it's done using pytest.
If you want to learn more about calling REST APIs in Python, please review the docs for the request package.
DuckDuckGo Feature Test
We will write a test for the DuckDuckGo Instant Answer API. DuckDuckGo is a popular search engine and its Instant Answer API returns information for search phrases. We will use this API because it is simple to call and verify.
Before we can run our test, we need to install the “requests” package. Do this by executing the following command from the terminal:
pip install requests
Now, it's time to automate. Let's create a new module named
test_rest_api.py in the
Add the following import statements:
import pytest import requests
Then create a test function named "test_duckduckgo_instant_answer_api". This test will search a basic phrase and verify parts of the response.
For the "Arrange" steps, set the URL for the request.
The base URL is "api.duckduckgo.com".
The URL needs two query parameters. The first is
q for the search phrase. So, let's use "python+programming". The plus sign is a filler for whitespace.
The second parameter is
format, which we will set to "json" so that the response comes back in JSON format.
# Arrange url = 'https://api.duckduckgo.com/?q=python+programming&format=json'
For the "Act" steps, the line
response = requests.get(url) sends the request as a HTTP GET to the DuckDuckGo Instant Answer API endpoint. It's so concise!
This is exactly why people love using the requests package. Then write a second line,
body = response.json().
This will parse the response data from JSON text into a Python dictionary object which will be useful for validation.
# Act response = requests.get(url) body = response.json()
For the "Assert" steps, write two lines.
assert response.status_code == 200 to verify that the request was successful.
assert 'Python' in body ['AbstractText'] to check the actual content of the response.
If the search for Python programming was successful, then the answer given should include the word "Python" somewhere.
# Assert assert response.status_code == 200 assert 'Python' in body['AbstractText']
Finally, to keep things consistent, let's add markers for "duckduckgo" and "rest_api".
@pytest.mark.duckduckgo @pytest.mark.rest_api def test_duckduckgo_instant_answer_api():
Let's run this new test to make sure it works. I recommend using "-m rest_api" to pinpoint the new test case:
python -m pytest -m rest_api
Bam, it works. Notice how it takes a little longer to complete than the unit tests.
The requests package is arguably the simplest way to test REST APIs in Python.
However, there's also a Python extension called "Tavern" that lets you write REST APIs using YAML. Definitely check it out if you want to learn more about REST API testing in Python. Test cases looks something like this:
REST API tests are not the only kinds of feature tests that pytest can handle. Pytest can handle any type of functional tests, including Web UI tests.
Web UI testing is a bit complicated because it needs extra setup and involves multiple steps.
For that reason, we won't cover Web UI testing in this course. If you want to learn Web UI testing with Python, there's another Test Automation University course out there: Selenium WebDriver with Python.
It teaches how to integrate the pytest framework with Selenium WebDriver to automate Web UI tests in a live browser. That course could be seen as a "Part 2" for this pytest course and I highly recommend taking it!