Addressing the challenges involved in game testing
Testing software in ‘Gaming’ needs serious skills and discipline such as winning strategies, competition goals, rules, team work elements and usage of external devices, sensors and applications of various laws of physics. While traditional application software testing is more structured and focused on features, functionality, usability and nonfunctional testing, the same in ‘Game testing’ is more on strategies, decisions, unpredictable situations and scoring mechanism. Game testing does become a challenge as most game scenarios are dynamic and respond differently at different situations. These scenarios are often undocumented when the game arrives for testing, so there is an element of intuitiveness that the tester must build in order to be good at testing ‘software games’.
The ‘Gaming industry’ can be segmented based on players (gender, age group), type of interfaces or platforms (PC based, console based, smart phone, handheld independent), genre of the games (sports, fun, competition, strategy to add a few), and the complexity and size of application (which involves the levels, # of players, duration).
The games are designed for the players or players decide the game. Both are true to a large extent. Games such as Pokémon, Candy crush, Scrabble has users across all the age groups, and on the other hand games related to learning (education) are designed for children. Also, there are many sport games designed for the working class including those involving virtual reality and physical movements. Games are also developed for the clinical and rehabilitation programs for geriatrics. The data collected from USA in 2017 reveals that people under 18 years are the largest audience at 29%, followed by 27% in the age group 18-35 years. Interestingly, these numbers are the lowest among the age group 35- 50 years, and then increases steadily in 50+ age group, as shown in Figure 1.
 |
 |
|
| Figure 1: Gamers by age group in the US region | Figure 2: Global gender variation by age |
Another research report in the global front indicates that the age group in 21-35 years are the dominant users of the games, and the least in the geriatrics group as shown in Figure 2. The male gamers are dominant in all age groups under 50 years old. The highest gap between genders is observed in 21-35 yr group. Interestingly, this study reveals that the females are marginally higher than males in the age group over 50. Research on the gender classification show that females to males are in the ratio of 4:6 in the US region.
The proposed test design elements for ‘software games’ are as follows:
|
Element |
Description |
|
User centric |
The success of a game is to have maximum acceptance and ‘stickiness’. The game needs to have elements of surprises and be highly unpredictable. Even known games like ‘scrabbler’ or ‘chess’, by bringing in scenarios or interactive elements at the right time and moment, have a longing effect. These are mostly undocumented and very difficult to test or even explore. The tester has to be imaginative and intuitive to test these design elements. |
|
Learnability |
Most of the new concepts have the issue on the learnability. While the ‘game designers’ are usually familiar with the rules and objective, the end user has a threshold range. It should neither be too simple nor be too complex for learning. The former is bad to kill the enthusiasm, and the latter is bad for ‘gluing the gamer’. The complexity of the game is not to be misunderstood for the complexity of learning. The learnability of the game may be a more important factor among the games designed for younger age group and the senior citizens. |
|
User Interfaces |
The interfaces for playing is varying rapidly from external consoles, sticks, keyboards, touchscreen, and in some cases augmented interfaces depicting real ones. In virtual reality based games, it is possible to have a sensor based tennis racket in a playing area. Testing these interfaces needs a lot more engineering knowledge than the regular QA skills. |
|
Content |
The challenge in the content is not the text part but the contextual relevance, and timing. Each has its own challenge driven by the age group. A game designed for the teens has to be verified for adult content. A learning game for children has to be verified for simplicity of the language. Then there are challenges in the cultural side as well. There are certain English words used in the America regions as standard not acceptable in the Asian regions. The games designed for senior citizen must have more emphasis on clarity, size of fonts and contrast aspect. The timing of content published in the game has to be appropriate. In general, the content level testing is an extreme challenge for the QA team. About 70-80% of these design elements come to scoping during the final phases of the projects due to poor documentation process. |
|
Audio Video |
The importance of audio-video is the icing on the cake. The presentation layers are the most critical elements to draw the audience. Many visual and auditory elements such as the sound levels, frames, color contrast, audio sync and characters, motion of the screen elements and graphics contribute to the success of the game. |
Some of the common challenges in the gaming software are:
- To expect a higher number of undocumented requirements and implicit requirements
- The clarity of ‘What to test’, ‘how to test’ and ‘where to test’ has to be defined in early stages of the testing to avoid delays and improve the effectiveness of testing
- Multiple paths and alternatives in the game strategy, AI/ML algorithms, increase the challenges for the QA team
- Dynamic motion of multiple elements increases the complexity and coverage of testing from an engineering perspective
- Usually online games and mobile app based games have additional challenges of interoperability, compatibility and support of multi browser or platform, apart from the nonfunctional testing
- There is an increased focus and demand of omni channel experience and hence this leads to testing effort on multiple devices and system
- Though many nonfunctional requirements are critical, performance testing is one of the critical element for online games
- Many times, the lack of understanding of the end users (of the game) in terms of their age group, target region, purpose, genre can lead to significant impact on the quality of testing and type of defects raised by the QA team
Conclusion: In conclusion, testing ‘software games’ demands intuitive mindset from the QA professional in the form of planning, defining strategy, structured thinking and application of engineering tools in the analysis. While there are many challenges in the form of undocumented requirements, undefined scenarios and unknown outcomes, it is also an opportunity to bring in creativity and thinking skills and developing QA solutions, frameworks and engineering tools as the field is wide open.
Reference:
- statistica.com
- https://www.bigfishgames.com/blog/2017-video-game-trends-and-statistics-whos-playing-what-and-why/
- Explore: Testing the Game, Kiran Marri, Testing Experience
