Effective Snapshot Testing

Snapshot tests verify the entire state of a system rather than individual assertions. This significantly reduces the effort required to write tests, as a single snapshot represents the expected outcome.

Keeping in mind that a snapshot test should be as narrow as it can be so that any changes to the snapshot are easily reviewed, let’s focus on a simple java example:

@Test
@Sql(scripts = "classpath:fixture/io2_data.sql")
void computes_kpis_from_db_state() {
    var result = kpiService.calculateKpis(IO2, ALL);
    // Verify snapshot
    JsonApprovals.verifyJson(new Gson().toJson(result));
}

In this example, a fixture provides data preparation for a test and the snapshot represent a single table state after logic execution. Even in complex systems, a snapshot should be very small; otherwise it becomes hard to maintain.


One important thing to consider is to make sure that the final snapshot is deterministic to avoid flaky tests. In the example where we have JSON snapshot, a developer should:

@Test
@Sql(scripts = "classpath:fixture/io2_data.sql")
void computes_kpis_from_db_state() {

    // Enforce time usage to have deterministic dates
    try (WithTimeZone tz = new WithTimeZone("UTC")) {

        var result = kpiService.calculateKpis(IO2, ALL);
        var normalizedJson = new Gson().toJson(normalize(result));

        JsonApprovals.verifyJson(normalizedJson);
    }
}

A risk with tests like this, is that the simplicity of the assertion makes it easy to correct when a test starts failing, for that reason I suggest to use classic assertions to capture broader concepts that should remain true for the particular test.

@Test
@Sql(scripts = "classpath:fixture/io2_data.sql")
void computes_kpis_from_db_state() {

    // Enforce time usage to have deterministic dates
    try (WithTimeZone tz = new WithTimeZone("UTC")) {

        var result = kpiService.calculateKpis(IO2, ALL);
        var normalizedJson = new Gson().toJson(normalize(result));

        assertNull(result.getError());
        assertEquals(100, result.getProgressPercent());

        JsonApprovals.verifyJson(normalizedJson);
    }
}

This does mitigate the risk but does not prevent developers from updating snapshots without fully understanding the change; however, if you are serious about programming, a good approach to this problem is fixing a snapshot instead of replacing it when code changes invalidate the test. For that, one can leverage tools such as jq which is useful to compare two JSON files.

For example, you can normalize and diff snapshots locally:

diff <(jq 'del(.timestamp) | sort_keys' received.json) \
     <(jq 'del(.timestamp) | sort_keys' approved.json)

This approach makes any change intentional and specific so that any code review can treat it just like any other piece of logic.

Other readings


back to all articles