Will we soon see the day when AI will analyze your skill and automatically adjust the difficulty level? Report on the session where the method and its application to automatic level generation were discussed [CEDEC 2024]

The day may soon come when AI will finely adjust the difficulty based on the player's skill, and even create stages. Held on August 22, 2024 on the second day of the developer conference "CEDEC 2024" ``Methods for analyzing player's 'skill' and game's 'difficulty' and for automatic level generation. In "Applications" , the research results of Square Enix's AI & Engine Development Division were revealed.

● “Methods for analyzing player skill and game difficulty and application to automatic level generation” Speaker

• A-seong Song (Square Enix AI & Engine Development Division Programmer)
• Ken Shirodokoro (Square Enix AI & Engine Development Division AI Programmer)

AI analyzes the player's skill and creates a pleasant difficulty

To play the game, players are required to have various abilities such as being able to land attacks and avoid attacks. Mr. Song points out that while each player has areas of strength and weakness, current game difficulty settings are uniform and are not tailored to each individual player.

In such a situation, if you change the difficulty level, all the elements will go up or down uniformly, which could lead to discrepancies. For example, if a person who is bad at evasion but good at attack lowers the difficulty and evades becomes easier, it will become too easy because they already have high skill at attacking.

Therefore, Mr. Song thought, ``If we could dynamically adjust the difficulty level for each field, we could provide an experience that fits each player's strengths and weaknesses.'' To do this, it is necessary to have a system that classifies players' abilities in advance and then allows AI to make judgments and analyses.


This is how the ``Ability Analysis'' mechanism using the ``Ability Graph'' was created. The "Game Master AI" not only monitors play and judges abilities, but also analyzes this and interferes with the development of the game, checking the player's skill and providing assistance or increasing the number of enemies. . An example was given where this was applied to a vertical scrolling shooting technology demo (players walk on land and receive interference from the terrain when moving. Terrain can be destroyed and items appear from within it).

The ability graph has a graph structure in which "ability nodes" are linked by "edges (arrows)." Ability nodes indicate the abilities you want to analyze, such as ``Basic (ability to perform basic tasks such as movement and attacks)'' and ``Action (ability to accomplish goals by making full use of multiple basic abilities, such as defeating enemies and picking up items). )" and "Tactics (abilities that represent gameplay strategies, such as reducing threats, preventing damage, and increasing status)."

Each ability node has a ``Mastery Rate (hereinafter referred to as MR)'' which indicates the player's proficiency level, and a ``Challenge Rate (hereinafter referred to as CR)'' which is the ability requested by the game. A link has a weight (relevance) value set, and the higher the value, the more important it is.
When you actually play the game, the game master AI checks the player's skill in each ability node item based on the ability graph. Ability characteristics are judged using a graph that has two axes: good←→weak and difficult←→easy.High MR means good (low means bad), and high the difference between CR and MR means difficult (low means bad). That's easy).

Seeing this, the game master AI intervenes in the development of the game. If you are good at something but find it difficult, ask a friendly NPC to help you, and if you are weak at something but find the task itself easy, you can display tips to help you improve, giving detailed follow-up. It can also be applied to production, and if you're having a hard time, you can create a disturbing atmosphere by making the background tattered and raining.

If the player is having a hard time, you can help them, or if the difficulty level is not enough, you can increase the number of enemies and obstacles. Express a ``comfortable difficulty'' that the player will find rewarding. It is said that it will be possible to do so.

In this ability node method, the calculation of MR and CR used for evaluation is important, so various ideas have been devised. Some ability nodes can be directly evaluated and others cannot. For example, in the case of "defeating an enemy," MR can be calculated from the number of enemies defeated and the number not defeated, and CR can be calculated from the type and number of enemies arranged in a level.

However, since tactical abilities are abstract, such as ``increase status'' or ``clear a stage successfully,'' they are estimated using weights from the MR and CR of the parent ability on the graph. This idea itself can be used regardless of the genre or map format, such as FPS, turn-based RPG, FPS, etc.

Looking to the future, Mr. Song said that he would like to be able to estimate the reasons why a play went well or not, and present content that helps people practice the abilities that caused the problem.

Using AI to automatically generate stages that match the player's skill

In the second half of the lecture, Mr. Shirodokoro talked about an example of using ability nodes and CR to automatically generate levels, an initiative in which AI creates stages according to the player's skill .

When using Procedural Content Generation (hereinafter referred to as PCG) for level generation, in this example, a PCG algorithm corresponding to each CR is required, but if the CR specifications change, the algorithm cannot be used, and multiple An algorithm that takes CR into consideration has the drawback of becoming too complex.

In order to solve this problem, Procedural Content Generation via Reinforcement Learning (level generation using reinforcement learning, hereinafter referred to as PCGRL) was used. It is possible to generate levels based on rewards set by humans, and in this case, if you use CR as a reward, the AI ​​will learn how to create levels.

Reinforcement learning is an agent that learns the best strategy based on rewards, searching for ways to obtain more rewards. In PCGRL, agents receive rewards when they place obstacles in the level they are editing, so they learn to aim for higher rewards (placements that match the intent of the stage design).

Even when it comes to PCGRL, there are various methods, each with its own limitations. For example, when using PCGRL to automatically generate a puzzle game where you push a crate into a goal (probably the so-called "Sokoban"), you can create a level with a certain degree of difficulty by adjusting the rewards, It is not possible to create levels according to parameters such as controlling the number.

In other words, it is impossible to generate levels according to the target CR. Controllable PCGRL solves this problem. In the example I gave earlier, PCGRL can control aspects such as the number of crates, goals, and the shortest number of steps, but it cannot generate the levels of a complex game like the shooting game that I want to create this time.

Multi-layer PCGRL automatically generates complex game levels by combining "level layers" such as enemies and terrain, "information layers" with information such as CR, terrain generation modules, enemy placement modules, etc. can. The terrain generation module creates the terrain, the enemy placement module places enemies based on this, and the item placement module places items to complete the level.

The lecture also explained how Multi-layer PCGRL automatically generates levels regarding the ability to destroy terrain. Multi-layer PCGRL automatically generates one screen at a time. The goal is to obtain an average CR of 20 rows, which is the average CR of 20 rows x 16 columns for one screen. Multi-layer PCGRL repeats trial and error so that the average CR of the generated level is an average of 20 rows CR. In other words, the policy of what level you want to generate is an average of 20 rows CR, and the more it matches, the higher the reward.

In this example, we are conducting a test in which we learn 20 million steps in about 4 days, specify random CRs, and create 100 levels (in this case, 100 screens worth of maps). It takes an average of 0.62 seconds to generate normally, 0.46 seconds to generate roads and other terrain to guide the player, and 0.74 seconds to add obstacles, all of which are fast, and all generated levels take an average of 20 lines. It is said that it met the criteria of CR.

When placing enemies and items on flat terrain with an average CR of 20 lines, enemies could be placed in an average of 0.32 seconds, items in 0.48 seconds, and 100% had an average CR of 20 lines. By having PCGRL edit existing levels, it is also possible to create a level that is easy at first but difficult in the second half. Additionally, when a test was conducted in which PCGRL generated levels in real time during play, good results were obtained.

Lastly, Mr. Song talked about ability analysis methods, such as dynamically generating content and quests, having the AI ​​complete maps that the player has begun to create, and creating new games that are single-player but reminiscent of multiplayer, such as player vs. game master AI. He concluded his lecture by talking about future prospects and thinking that it could be applied to sex.

Auto-adjusting the difficulty level using AI has been a dream of game developers, something that has been a long-standing endeavor. It can be said that modern AI research and machine power have made that dream more realistic and far-reaching. As a player, you can enjoy a game that suits your skill level, and the automatic generation of levels will make the development more varied, so it's a wish that comes true. There is a recent trend of creating environments that are easy to play for a wide range of people, and there seems to be a strong need for automatic difficulty adjustment and automatic level generation, and this lecture gave a strong sense of their potential.

The above is the detailed content of Will we soon see the day when AI will analyze your skill and automatically adjust the difficulty level? Report on the session where the method and its application to automatic level generation were discussed [CEDEC 2024]. For more information, please follow other related articles on the PHP Chinese website!