How to use Snapshot testing in Django?
What is a ‘Snapshot’?
According to Urban Dictionary, a snapshot is what people that aren’t up to date call screenshots.
Duh… totally clear. To a Django developer, a snapshot looks like,
snapshots['SnapshotTests::test_choices_list_zero 1'] = { 'count': 0, 'next': None, 'previous': None, 'results': [] }Let’s read the above python code, snapshots is a dict with key ‘SnapshotTests::test_choices_list_zero 1’ . This key is assigned a JSON object value {…}. This JSON object is the response of the API endpoint GET /api/choices/ . This JSON object is a snapshot of the API response.
If we assume this JSON object is the expected response and we make code changes as one does, we can test the functionality of GET /api/choices/ by calling it at any point as comparing the response with the previously saved snapshot response. If the responses match, then the API works as before. If the responses don’t match, then look for what caused the change. If the change in the response is intentional, save the new response as the updated snapshot. The best part about snapshot testing is that the snapshots are auto-generated. Yes, the above python code was automatically generated and not manually written.
To help explain snapshot testing, I’ve created a simple Django app on GitHub. The snapshot test changes are diffed by comparing commits. You’ll need snapshottest and factoryboy packages to run the tests.
Commands
Run tests using python manage.py test . This command executes tests defined in the tests.py file and compares the API responses with the snapshots saved in the snapshots/snap_tests.py file.
To update the snapshots of the API responses, execute python manage.py test --snapshot-update . This command updates the snapshots/snap_tests.py file with the API responses of the tests defined in the tests.py file.
Do NOT manually edit the contents of the
snapshots/folder.
How to write a test case?
Write tests in the
tests.pyfile in the Django app folder.— Gulfaraz Rahman, 2020
First, create a factory for each model in the app. A model’s factory can be used to create multiple instances of the model. For example, we define a simple model,
class ExampleModel(models.Model):
name = models.TextField(verbose_name=_('name'))
age = models.IntegerField(verbose_name=_('age'))How to create a factory?
To test API endpoints which depend on ExampleModel, we first create a factory using a handy library called factoryboy. ExampleFactory is defined as,
class ExampleFactory(factory.django.DjangoModelFactory):
name = 'Example Instance'
age = 42A good factory covers all possible values accepted by the model. To achieve sufficient randomness, we use Faker that generate random values for each instance of ExampleModel created by ExampleFactory.
class ExampleFactory(factory.django.DjangoModelFactory):
name = factory.Faker("name")
age = factory.Faker("pyint")Now we are ready to write test cases for API endpoints involving ExampleModel. A real-world factory is ChoiceFactory which is used to create instances of the Choice model.
Test case for list API
For list test cases we make a GET request,
class TestExampleAPI(TestCase):
def test_example_list(self):
# submit list request
response = self.client.get("/api/example/") # check response
self.assertEqual(response.status_code, 200)
self.assertMatchSnapshot(json.loads(response.content))
The test_example_list test case calls the GET /api/example/ endpoint. It checks for the response status code using assetEqual and response body using assertMatchSnapshot and json.loads (only if the expected response is in JSON format).
By default, the test run starts with an empty database. So test_example_list will always return an empty list.
To check for a response with a non-empty list response, populate the database with an instance of ExampleModel. Call the create function on ExampleFactory to create an instance of ExampleModel,
def test_example_list_one(self):
# create instance
ExampleFactory.create() # submit list request
response = self.client.get("/api/example/") # check response
self.assertEqual(response.status_code, 200)
self.assertMatchSnapshot(json.loads(response.content))
To create multiple instances, call the create_batch function.
Time for a comic break,
Test case for create API with authentication
We create an instance of ExampleModel to submit to the POST endpoint,
create endpoints require user authentication. Authentication is possible via the force_login function.
def test_example_create(self):
# create instance
new_name = "Mock Example for Create API Test"
new_example = ExampleFactory.stub(name=new_name) # authenticate
new_user = UserFactory.create()
self.client.force_login(new_user) # submit create request
response = self.client.post(
"/api/example/",
new_example,
content_type="application/json"
) # check response
self.assertEqual(response.status_code, 201)
self.assertMatchSnapshot(json.loads(response.content))
self.assertTrue(ExampleModel.objects.get(name=new_name))
The test_example_create test case calls the POST /api/example/ endpoint with a request body. It also uses assertTrue to check if the ExampleModel object is created as a result of the POST request.
Using methods used in the above examples, test cases can be written for update, read and delete endpoints.
Reproducible Randomness
Although our factories generate random attributes, we require the API response to be exactly the same at each run to have comparable snapshots.
Reproducibility is achieved by setting a random seed, using mocks and resetting the database before executing each test.
Examples of real-world snapshot tests are defined in SnapshotTests.
Test Coverage
After installing the coverage package, executecoverage run — source=’.’ manage.py test.
Run coverage report to view a report of the lines of code executed to run the tests. The coverage report includes coverage of all tests including snapshot tests.
For a line-by-line coverage report run coverage html and open htmlcov/index.html in a browser.
