Apakah teknologi AI yang mengautomasikan ujian fungsi sistem termasuk permainan? Laporan tentang sesi memperkenalkan usaha pasukan QA perisian sistem PS5 [CEDEC 2024]

Pada 21 Ogos 2024, di persidangan pembangun permainan "CEDEC 2024", sesi "Teknologi AI yang merealisasikan automasi permainan permainan di bawah keadaan yang sama seperti pemain manusia di PlayStation 5" telah diadakan.

Dalam sesi ini, teknologi AI yang merealisasikan automasi permainan telah diperkenalkan, yang digunakan dalam beberapa ujian berfungsi semasa QA (jaminan kualiti) perisian sistem PS5. Penceramah adalah tiga orang berikut.

Perisian sistem usaha QA

PS5 mempunyai fungsi sistem seperti rumah dan pusat kawalan, tetapi untuk memastikan kualiti kandungan paparan perisian sistem dan peralihan skrin, pasukan QA perisian sistem (selepas ini dirujuk sebagai pasukan QA) mempunyai ujian automatik berkata bahawa ia secara aktif menerima pakai

Nampaknya terdapat beberapa faedah untuk menguji automasi, dan satu contoh ialah keupayaan untuk meneruskan ujian sehingga perisian dikeluarkan. Jika ini adalah ujian manual, ia akan dilakukan berdasarkan kes demi kes, tetapi dalam kebanyakan kes ia hanya akan dilakukan beberapa kali bagi setiap projek.
Dalam kes itu, jika pepijat diperkenalkan pada masa yang ditunjukkan oleh anak panah merah dalam slaid di bawah, ia akan dikesan semasa ujian di penghujung projek, ditunjukkan oleh anak panah biru, dan akan ada risiko ia akan menjejaskan pelepasan.

On the other hand, automatic testing can be performed even every day, and bugs can be detected immediately at the timing indicated by the green arrow, resulting in the advantage of ``early bug detection.'' Additionally, if you can run automated tests before committing development code, you can prevent bugs from being introduced.

From this perspective, the QA team is promoting test automation, but it is necessary to give some consideration to system functions linked to gameplay.
One of these is "activity," which displays the progress of the game currently being played as a card, and QA also includes whether the progress and estimated playing time are displayed correctly on the card. It was also shown that the contents of cards sometimes change depending on gameplay, and that QA is also required to check whether the parts related to these changes are working correctly.

Furthermore, functional testing of system software sometimes requires a certain amount of gameplay, such as when checking activity updates. Therefore, it is desirable to automate this gameplay as well from the standpoint of efficiency and early detection of defects.

There are other system software functions linked to game play other than activities, and some are used depending on the title and others are not. Therefore, when automating the testing of functions linked to gameplay, a general-purpose form of automation that is title-independent is required.
In addition, the automated test must be conducted under the same conditions as a human player, that is, using only screen and audio information. Because of these constraints, it seems that automatic testing of functions linked to gameplay was extremely difficult.

Autoplay System Overview

As mentioned above, functional testing of the PS5's system software requires that the only information that can be used is the game's screen and audio information, and that it is a general-purpose technology that does not depend on specific game titles. Additionally, in order to bring this to the QA field, it is necessary to be able to achieve automation at a realistic cost.
Given these constraints, the QA team developed an automatic play system by combining multiple technologies based on ``imitation learning,'' a technology that learns and reproduces human play.

In this session, an overview of this automatic play system was introduced using the example of "ASTRO's PLAYROOM", which is pre-installed on PS5. This system runs on a PC and only obtains screen information from the PS5 as game information. The system then determines the content of the controller operation and sends it to the PS5 to automatically operate the game.

Additionally, this system uses two types of agents to automatically operate the game. One is called a ``replay agent,'' which replays pre-recorded manual gameplay to reproduce the same operations as the manual gameplay.
The other type of "imitation agent" is an AI model that has learned human game play using imitation learning, a type of machine learning, and reproduces the human game.

Because the replay agent reproduces the exact same controller operations as in previous manual play, there are no random elements, and it is only used in scenes that can be progressed with the same operations at all times.
Specifically, this includes some UI operations such as starting the game on the title screen, and moving along a fixed route where the starting and finishing points always remain the same. In addition, the Replay Agent is also used to perform operations on the PS5 itself that are necessary for functional testing during QA.

Imitation learning, which allows an imitation agent to learn human game play, is a type of machine learning that creates a model that reproduces the behavior pattern from model behavior data. The specific procedure for creating an imitation agent is to manually play the game multiple times in advance, perform imitation learning using that data, and build a model that can reproduce the operations of that manual play.

This imitation agent model requires only game screen information as input. When you input game screen information, it is set to output the controller status in the next frame, and by operating at 10 frames per second, you can decide on operations in real time. This imitation agent targets all scenes to which replay agents cannot be applied, that is, all scenes that have even the slightest random element.

In this case of "ASTRO's PLAYROOM", models for each stage were prepared for the imitation agent. The reason is that the fewer scenes a single model is responsible for, the more stable the performance of each model will be.

Additionally, "scene recognition" is also available as a function to determine when to switch between two agents. This function uses screen information to determine when the game has reached a specific scene, and uses two main technologies. One of these is ``template matching,'' which determines whether the same object as a pre-prepared template image exists on the game screen.
For example, it seems to be used to recognize icons and pop-ups that appear when completing a quest.

The other method is "feature point matching," which constantly checks the similarity between a target image prepared in advance and the game screen. When the similarity exceeds a threshold, the game determines that the same scene as the target image has been reached and switches agents.

It was also mentioned that ``robustness against visual blur'' is important when using feature point matching for scene recognition. For example, even if you reach the same area, it is not uncommon for the screen to look different depending on the direction of the camera. Similarly, the lighting may change depending on the time in the game. It was also revealed that the QA team used LoFTR, a machine learning-based image matching method, to address these changes in appearance.

Gameplay Automation Steps

When automating gameplay, it is first necessary to determine the content of the test that you want to automate, including gameplay. Then, the content of the test is divided into agent units.
Next, the game is actually played manually, and play data is obtained once for replay agents and 10 or more times for imitation agents. In the case of imitation agents, imitation learning is performed using the acquired data.

Once imitation learning is complete, we will actually run the two agents and check whether automatic play is possible. If automatic play fails, the process returns to the data acquisition phase, and the flow of data acquisition → imitation learning → operation confirmation → ... is repeated until automatic play is successful.
Once automatic play is successful, use a periodic execution pipeline such as Jenkins to organize the process. This is the sequence of gameplay automation.

The QA team uses this automatic play system to actually automate and regularly execute functional tests, including gameplay. As a result, it was found that three bugs in the system were automatically detected, and that these bugs were detected during a functional test immediately after they were introduced. The reason why the number of bugs detected is so low is because the automation of functional tests is currently being positioned as a trial within a limited scope.

Technical explanation of imitation agents

In the technical explanation of imitative agents, the imitative learning algorithm was first introduced. In addition to the aforementioned ``versatility that can be applied to any title,'' this project aims to achieve ``simplicity that anyone can use,'' allowing them to create imitation agents without the need for engineering skills.

Based on these two goals, the QA team adopted "Behaviour Cloning" as an algorithm. This algorithm is the simplest type of imitation learning, and is supervised learning that connects model input and output. In this case, learning will proceed with the input being the game screen and the output being controller operations. The specific process for creating an imitation agent, as described above, is to record the game screen and controller information as data through manual play in advance, and then have the model learn the play data.

The structure of the model is extremely simple; the single input image is passed through the image feature extraction layer and the controller state output layer (fully connected layer), and the controller state is finally output. It was also explained that the operation and recording frequency was set at 10 frames per second to make learning easier.

The controller's analog stick output is treated as a total of 9 classes, including up, down, left, right, diagonal, and neutral, rather than analog values. This is said to be a result of the ease of learning.
Also, regarding the output of the model, in order to output the status of each button and stick on the controller, we used a network with a branched final layer for each. The final layer, the fully connected layer, outputs on/off guidance for a button, and guidance in each direction for a stick. This is said to have the meaning of preventing unnecessary learning from occurring and improving model performance by not creating a network of redundant operations tailored to the dataset.


A pre-trained EfficientNet encoder is used for the image feature extraction layer. During imitation learning, the EfficientNet encoder did not perform any further learning, and only the fully connected layer learned. This is because it has been confirmed in various cases that model performance improves by using an EfficientNet encoder that has been pre-trained on a large number of live-action images, rather than having the encoder learn from a small number of game screens. . In addition, training using only fully connected layers has been shown to have the advantage of reducing the amount of time required for training, as the network becomes smaller.

It was also revealed that the reason for using a single image as input is based on previous experimental results showing that model performance deteriorates as the number of inputs increases. This tendency is seen not only in ``ASTRO's PLAYROOM,'' but also in tasks in other games.

They also introduced how they responded to the challenges they faced during this initiative. For example, operations such as ``pressing a button to pick up an item that has fallen at a random location and is essential to progress'' are not uncommon in games. However, deep learning is slow to learn such operations that appear infrequently but have a large impact on the completion rate, making it extremely difficult to create a model that presses the button as expected.

Therefore, in this effort, they used "Class Balance" to adjust the degree of influence that learning has on the model, and the lower the occurrence rate of operations, the more weight was given to them, so that the learning was more strongly reflected in the model. .


Another weakness of imitation learning is that it becomes difficult to recover if the operation fails and deviates from the model data. This problem was solved by learning additional data that allowed the robot to return from an unknown state to a model state.

At the end of the session, the opinion was expressed that through the verification of the automated play system introduced this time, it was confirmed that the quality of future QA tests can be expected to improve.

Atas ialah kandungan terperinci Apakah teknologi AI yang mengautomasikan ujian fungsi sistem termasuk permainan? Laporan tentang sesi memperkenalkan usaha pasukan QA perisian sistem PS5 [CEDEC 2024]. Untuk maklumat lanjut, sila ikut artikel berkaitan lain di laman web China PHP!