As EGO’s global domination expands onto the mobile platforms all of EGO’s QA minions have been working timelessly towards the eagerly anticipated and fast approaching global launch on iPhone and iPad. We thought it was about time to give you an insight into the lives of our QA analysts.
This is what our QA minion’s desks look like (but with more empty coffee mugs).
QA Analysts are production’s eyes, they are responsible for verifying the functionality, compatibility, usability as well as technical compliance of each version of a title that gets compiled and deployed. Once as thorough testing as possible (before the impending deadline) has been completed, and the QA team is confident in the build at hand, it may be sent to a first party for a submission process preceding the release to the public.
Let’s take a quick look at some of the key testing approaches:
- Functionality testing: making sure that the game functions as it was designed and intended by verifying all of the game’s features and mechanisms (e.g.: confirming that your minion’s animation is correctly displayed)
- Compatibility testing: making sure that the game functions on all supported devices
- Usability testing: making sure that a user will have the optimal gameplay experience (e.g.: confirming that the UI is clean and accessible)
- Technical requirements: most gaming platforms, whether mobile or console, require publishers to pass a submission process for each code iteration that is to be released; the title must comply with the first party’s guidelines (e.g.: correct terminology and use of branding)
Advanced game features like the 'Leaderboard of Infamy' need to be tested thoroughly by the QA team to ensure everything works as it should on release.
The majority of issues found can be attributed to one of the four areas listed above, in addition to these approaches there are various techniques/styles a QA Analyst can use. Let’s have a quick glance at some of the standard methods we use:
- Scripted testing: test-case driven testing, the analyst goes through a test plan with conditions that can be set to Pass/Fail based on the results achieved from following the test-case’s instructions
- Ad-hoc: unstructured testing that is typically performed once all the test plans have been completed and the Analyst simulates end-user behaviour, triple checking things to catch anything missed by the scripted test round
- Destructive: unstructured testing that has the Analyst try to forcefully break the game, whether it is by attempting to bypass the correct design of a feature, continuously change settings or say spam the same action over and over again
- Network testing: making sure that the title behaves correctly and gracefully handles a loss of connectivity to the network
- Regression testing: verifying that a previously entered bug that is now claimed by a programmer as resolved is indeed resolved and can be closed off
Now that you’re aware of the basics of testing, let’s take a brief look at the day-to-day and some of the challenges faced by an Analyst on EGO.
The day starts off with a morning stand-up meeting – the development team of EGO runs a tight and agile ship. Typically, a stand-up meeting covers the basics of: “What was done yesterday? What are you going to be doing today? Is there anything impeding your progress?” for each member of the team. Any necessary shifts in tasks or priorities are communicated, and the team spreads back out to their desks to begin the day.
Due to the vast size and functionality, our team on EGO heavily relies on the use of testplans to make sure that each Analyst has verified every feature on every platform – most of our time is spent in-game with a testplan in hand.
As bugs are found, the Analysts are responsible for correctly identifying the issue and confirming accurate reproduction steps. Once that is done, they then enter it into a bug-tracking database at which point the cycle of life of a bug begins (entered=>assigned to a dev=>claimed fixed=> regressed and closed, or regressed and re-assigned for further fixing).
As soon as a bug is found it's reported into the bug-tracking database to go on to the Developers.
As the day progresses and testplans are nearing completion, the team gets a better idea of the state of the build on each platform. With every additional supported platform, QA must obviously make sure that the game runs smoothly without any bugs; it must also have the same functionality across all platforms supported - this is where it gets tricky.
The concept of a cross platform title is simple: provide you the ability to play the game on both iOS and Facebook so that you, the player, can enjoy spreading evilness from anywhere in the world on your mobile device. This sounds fairly straightforward, but bear in mind that it requires the build to work properly on both Facebook and iOS, or any other platform in development (e.g.: Android devices have back button – iOS ones do not; the content must be in-sync and functional.
So anytime a change is required on one platform, it also affects the others. Very often this means creating a new automated build that keeps each platform in sync and a fresh restart of the testing cycle. To complicate things further, EGO depends on its robust progress saving and syncing structure, it is essential that the saving works flawlessly across all platforms. And if that wasn’t tricky enough, we must also time our first-party submissions correctly and accurately to make sure that the build goes live across the platform simultaneously.
As the testplans are filled up and approach completion, the team gathers and correlates their results, cross-platform issues (bugs occurring on all platforms) are reported as such and tracking of the devices tested on is updated. As the Analysts get used to the game, the QA team normally has a gut feel on the status of the build in test within the first half-hour to an hour. This is timely communicated to the Producer and any required adjustments are made accordingly within the development team. At the end of the day the team sends out their report to the QA Lead, who compiles his own report and sends it along to the producer and the dev team.
Once all the bugs have been compiled by QA the Dev team can then work on updating Evil Genius Online.
There are many more facets to the skills of a good QA, but I hope that this brief post was interesting to you and that you now have a better idea of what it is exactly that QA does.