How to tackle randomness in automated game testing

Randomness has a special place in game development. Its usage can span from small visual effects (particles), to entire gameplay mechanics (dice rolls), or even business models (loot boxes).
Since randomness is ubiquitous, it is important to understand how it works, and take it into account in your tests.

True Randomness vs Pseudo Randomness

As computers are deterministic machines, they’re not designed to produce randomness by default. Instead they rely on specific algorithms for generating an approximation of randomness. These algorithms are called Pseudo Random Number Generators (PRNG).

True randomness on the other hand is achieved via hardware, to generate completely unpredictable numbers.

However video games don’t need true randomness, and thus always use pseudo randomness to introduce random elements.

Why does it matter for game testing?

Non-determinism in tests is a big issue because it damages the usefulness of your test suite. A non-deterministic test might fail, and you won’t know if that’s because of a newly introduced bug, or the non-deterministic part of your test.

Trying to get rid of non-determinism is at odds with randomness, which is, by definition, non-deterministic.

Or is it? As I said earlier, PRNGs are not truly random. Most PRNGs will accept a seed as an input which determines the entire sequence of numbers the PRNG will return.

For example in Minecraft every generated world corresponds to an initial seed you can provide yourself. One seed always generates the exact same world.

The ability to seed a PRNG is thus very valuable for your tests, as it allows you to ensure their determinism.

Unity Example

Unity has its own Random class, and if you give the same seed, it will always return the same sequence:

Random.InitState(42);
Debug.Log(Random.value); // => 0.000857711
Debug.Log(Random.value); // => 0.4177545
Debug.Log(Random.value); // => 0.6763434
Debug.Log(Random.value); // => 0.4247012
Debug.Log(Random.value); // => 0.292351
Debug.Log(Random.value); // => 0.6424699

Here is a full example as a MonoBehaviour:

using UnityEngine;

public class RandomnessExample : MonoBehaviour
{
    private float timePassedInSeconds = 0;

    void Start()
    {
        Random.InitState(42);
    }

    void Update()
    {
        if (timePassedInSeconds > 1)
        {
            Debug.Log(Random.value);
            timePassedInSeconds = 0;
        }
        timePassedInSeconds += Time.deltaTime;
    }
}

It displays the sequence of pseudo random numbers corresponding to the seed 42. If you run it yourself, you’ll see the same output as written above.

How to write automated tests with PRNG?

There are two ways to write tests with PRNG in mind. The first is to provide a seeded PRNG to the instances that need it (via dependency injection). The second is to mock the PRNG to enforce specific returned values.

If you want to go beyond determinism, it is also valuable to have tests using different seeds each time, to make sure some values are not breaking the game. It is important to note however, that you will need a way to reproduce the failed tests, which means you will have to keep a record of the seed used for these tests.

Wrapping this up

As we saw, randomness is not something to shy away from, since it is used everywhere in game design. So it’s important to know how to write tests to account for it. I hope this article enlightened you on how to do that with your own tests.