Game Quality Forum 2023, a review

As an QA automation engineer in video games, it is hard to find conferences that fit the intersections of my skills and professional interest. QA automation engineers are not only interested in QA, but also the technical gritty bits of automation engineering. This leads to situations where you either have to choose between conferences focused on automation but non-game related, or conferences about QA in video games.

Part of the issue is because automation is still very much not widespread in the video game industry. I talked about it previously. However this year, a conference caught my interest. The Game Quality Forum is a conference focusing on QA, localisation, and player experience in the video game industry. And this year they had many workshop and talks about automation.

I thus booked a train to Amsterdam and spent 2 days there to learn, get inspired, and meet fellow QA automation engineers.

The automation talks

The conference had a good number of talks dedicated to automation. I went to most of them and here are the ones I found most interesting:

Case-Study: Achieving effective test automation scale through partnership and mindful infrastructure investments

Adrian Maroiu and Jesper Klittmark are respectively Development Director and Software Quality Engineer at Electronic Arts. They presented how test automation can be a challenge for the different teams involved when done at scale. One of the issue is the noise produced by the sheer amount of automated test results. Their solution was to improve the categorization of known failures, so the most relevant team can react to test failures without the need of an intermediary. This speeds up the resolution process.
Another key take away was how shifting left requires a mutation of the team structures. While older models of QA expect quality to be the responsibility of a separate team, shifting left needs QV (Quality Verification) to be more and more embedded into the dev teams. This leads to better quality standards.

9 years, 11 months, and 20 days of automated dev QA

Matt Ditton and Ben Britten are respectively Studio Head and CTO of Mighty Games. They gave quite an energized presentation about their automation journey. The presentation was illustrated through multiple stories on how they built and improved the automation they have today.
Their automation is based on a bot mapping the entire game as a baseline for other tools. Their main "selling" point to game developers inside their studio was to build a wall showing the live automation happening on the build machines. This wall of screen was allowing devs to be more proactive about defect when the bot was stumbling on an issue.
Since the bot was able to map the whole game they used it to provide other tools like the ability to provide screenshots for every narrative path in a choice based game.
The main key take away from their presentation is how the accumulated value of automation allowed game designers and developers to focus on making their game instead of other things.

Automation 101 – Testing deeper, earlier, all the time

Michel Topuzoglu is a Technical Director at Ubisoft Paris. He presented an introduction on how to integrate test automation in video game development cycles. The talk focused on automated test scenario done in collaboration between QA analysts, designers, and developers. The example given was an integration test, with a script following the gherkin syntax.
The take away of this presentation was how automation can look like at a basic level, and what are the starting steps for anyone who wants to implement it in their own studio.

Standardizing automated testing for gaming and XR

Shane Evans is the CPO of Game Driver. He talked about the history of automated testing and how it got standardized through Selenium in the web industry. He then explained how Game Driver was hoping to provide a standardized solution via their patented technology. This talk echoed a lot of what I wrote in "Automated game testing is still in its infancy".

Simple as possible as a mindset for test automation

Maël Nagot and Bilal El Medkouri are respectively Test Automation Lead and Test Automation Engineer at Fatshark. The talk focused on tools and processes they used to improve the adoption of test automation inside their dev teams.
They key take away was to provide tools reducing the barrier of entry and utilization. For example their tools provide deep links to build crashes opening the relevant app immediately.

Those 5 presentations were the most relevant to me, as they provided some insights that can be applied in my daily job. However other talks were also interesting to attend.

Notable sessions outside the topic of automation

The conference covered a large spectrum of topics and some were notable to me:

Diversity, equity, and inclusion – Pressing play on ambition into action

The panel was held by Marina Ilari (CEO of Terra Localizations), Tamara Tirjak (head of localisation at Frontier Developments), Marina Ilinykh (Supervisor, Localization Operations at Riot Games), and Jonathan Garcia (Manager, Inclusion, Diversity, and Purpose at Discord). Discussions around diversity are inevitable in the recent years, and for good reasons. This panel was quite interesting, and Jonathan's contribution resonated with me a lot as he was framing the topic not from a profit point-of-view, but a larger-than-us topic.

Interactive discussions

The conference had very interesting interactive group discussions, which allowed attending people to talk in time-boxed sessions around a specific topic. This is a very good add to such conference.

Networking with people

A large part of the interest of such conferences is not only to learn from the speakers, but also to discuss with other people in similar positions in other game studios. It was inspiring to talk to like-minded people, who care about quality, localisation, and players. I learned a lot and managed to provide some insights myself.

Conclusion

The Game Quality Forum 2023 was an interesting conference and I'm glad I found a place that was relevant to the intersection of Games, QA, and automation. Meeting with people there was great and I'm sure I'll come back next year.