The world’s leading publication for data science, AI, and ML professionals.

Project Pendragon Part 3: The Colosseum Showdown

A head to battle of two AI bots I have built for the game Fate Grand Order to see which comes out ontop.

Screenshot from one of my speed runs for this blog post.
Screenshot from one of my speed runs for this blog post.

In my previous two posts I walked through how I built two bots to play the game Fate Grand Order (FGO). The first bot (Pendragon) utilized three Pytorch CNNs, two classic and one siamese, to identify cards and play them based on an algorithm I built. The second bot (Pendragon Alter) was trained in a custom game environment using Keras+Tensorflow based Reinforcement Learning to learn on its own to pick cards to play FGO.

I found that both of the bots are able to clear most quests including some of the end game level 80–90+ quests which shows they are able to play FGO fairly well. However something that I had not explicitly tested was how the bots fared in comparison to each other and to see how they compare to a human player, in this case me.

This third post is going to cover my process for benchmarking the performance of the two bots against each other and my thoughts on how to improve them. By its nature it is not as technical as the previous two posts, but is a quantitative and qualitative comparison between the performances of the two bots.

Since FGO does not keep track of things like efficiency or score, the best way that I could think to benchmark the two bots was to have them speed run a specific quest repeatedly using the same teams. The winner would be the one who can run it faster.

Name a Time and Place:

Based on the title, you can probably guess the showdown location. I selected one of the final events of FGO’s yearly Nero Festival where players battle in the Roman Colosseum against a myriad of different enemy teams. The particular quest’s name is "The Rounds". The Rounds is an end game level 80+. Besides the high recommended level, another aspect of this quest that makes it difficult is that there is a mixture of enemy types.

This mixture of enemies means that it is harder to brute force your way through the quest simply by using advantageous class match ups. So this is one of the harder Quests I could pick for the bots, since they by nature have to brute force their way through. This added difficulty means they will be punished harder by the game if they choose bad card combinations. If they pick bad card combinations at best it will take a few turns longer to win or in the worst case will lose completely and be wiped out.

screenshot from the final battle of "The Rounds" quest. It is basically a battle against different knights of the round table. On several occasions the bots got stuck trying to defeat Tristan the red haired character on the left. He would fire his own ultimate several times and defeat multiple members of the bot teams and even managed to wipe out the Pendragon Bot on one occasion
screenshot from the final battle of "The Rounds" quest. It is basically a battle against different knights of the round table. On several occasions the bots got stuck trying to defeat Tristan the red haired character on the left. He would fire his own ultimate several times and defeat multiple members of the bot teams and even managed to wipe out the Pendragon Bot on one occasion

Choosing the Teams

It took a bit of thought to design the team compositions so they would play to the strength of the bots. A human player might try to use a composition where there was one main damage dealer and two support characters. Since my bots cannot make use of support characters, featuring them in the lineup is fairly useless. So instead I decided to front-load as much power into the team composition as I could to make it a brute force fight by fielding three max leveled damage dealers as the main part of the team. The rest of the team were weaker characters that I was able to fit into the team based on available points (see the total cost 105/105 in the below image).

If anyone watches Critical Role, yes that is a Critical Role team name reference
If anyone watches Critical Role, yes that is a Critical Role team name reference

Rules and Constraints

Since I am also competing here as a human benchmark, it is important to explain the constraints that the bots have at the moment so I can match them.

There are three main constraints that the bots have that are important for me to also comply with.

First, I have not created a way for the bots to use character skills. Skills are generally powerful bonuses which can be stacked on top of one another to amplify attacks. I have not built a way for the bots to use skills since it is a bit harder to have them use skills intelligently rather than just scripting them to use them on turn X.

Second, the bots fight front to back clearing enemy 1, 2, and 3 in that order. Players can technically select which enemy to attack if their hand is not advantageous against a specific enemy or maybe they want to target a specific character because they are dangerous.

Lastly, I added a protocol into the bots where they will not use their ultimate abilities, Noble Phantasms (NPs), until turn ten. I found that this was about how long it took the bots to get to the final round of combat. If I did not include this the bots may use their NPs at very inopportune times and they largely go to waste. This models the common player strategy of charging NPs on the first two waves and unleashing them on the third wave, which contains the final bosses, to end the final wave in one turn.

So as the human player in this contest I also constrained myself to those rules. Quick re-summary below

  1. cannot use character skills
  2. cannot select enemies, must attack them from front to back
  3. cannot use ultimate abilities, Noble Phantasms (NPs) until turn ten.

Adding any of these capabilities onto the bots would be a cool addition to the Project Pendragon or Pendragon Alter codebases!

Due to the enemies in this quest and these constraints I believe that the theoretical best lowest number of turns a team could complete this quest in is 10. If all 3 NPs are not used then the quests can easily drag on for another 4–5 turns.

Who Lives, Who Dies, Who Tells Your Story?

For the competition I had each of the two bots and myself run "The Rounds" quest 10 times using the above team composition and specified rules.

It took several hours to run that quest 30 times, partly because as I mentioned in Part 1 of this blog series my phone heats up after prolonged use of Teamviewer. However the results are quite interesting…

The original Pendragon bot and reinforcement learning trained Pendragon Alter perform quite similarly at first glance with averages of 12.8 and 12.5 turns respectively. Also each of the bots was able to hit he theoretical best score for this test at 10 turns each. The fun is in the differences and you will notice that the standard deviation is larger for the Pendragon bot since scores range from 10–17 while Pendragon Alter has a much smaller spread scoring between 10–14.

What I noticed while running these tests is that both bots did fine when they were able to meet the best case scenario. The starting three characters survived the first two waves of combat, had their NPs charged, and then unleashed them all at once on the third wave that contained the bosses. Both of the bots were able to do this some of the time because they both prioritized playing Arts cards to charge their NPs. The original Pendragon because I hard coded that behavior and Pendragon Alter because it learned that behavior itself in order to get that 80% win rate in the custom environment I built for it.

The differences between the two bots can be seen when they are not able to meet that turn 10 win, this usually occurred when one or more of the starting three characters were picked off and defeated before turn 10. Original Pendragon was forced with its back against the wall several times so out of the trials it has a number of scores at 15 and one at 17. In particular the round where it went to 17 was a round where the entire team was wiped out by the computer AI. This could be considered the worst case outcome. Pendragon Alter had a tighter distribution with a highest score of 14 and was never pushed back against the wall to the degree that the original bot was.

This tighter distribution is likely due to the more optimal gameplay that Pendragon Alter learned during its reinforcement training in the custom environment from Part 2. In particular the gameplay Pendragon Alter learned was to maximize damage and NP gain by prioritizing Arts and Buster cards while deprioritizing **** the third card type, Quick cards.

Similar to what I highlighted in Part 2, some examples of this prioritization of Arts and Buster cards can be seen below combined with Pendragon Alter’s knowledge of first and third card bonuses.

In this case Pendragon Alter chooses Arts cards in slots 1 and 3 and a buster card in slot 2. This is a choice the maximizes NP gain and helps Pendragon Alter get to that final wave with all of its NPs charged
In this case Pendragon Alter chooses Arts cards in slots 1 and 3 and a buster card in slot 2. This is a choice the maximizes NP gain and helps Pendragon Alter get to that final wave with all of its NPs charged

It is not that Quick cards are bad, but they have lower damage and do not charge NPs by that much. They create the random chance for higher damage due to critical hits and need for teams to be tailored to use them. However the teams that I play do not benefit from critical hits as much as certain compositions. So Pendragon Alter learning to deprioritize Quick cards lets it deal more damage than Original Pendragon on average using the characters that I play.

This deprioritization of Quick cards manifests in Pendragon Alter in being able to defeat an additional enemy in the early turns or being able to output more damage in scrappy late game situations. In comparison the original Pendragon bot would waste a crucial late game turn dealing less damage by playing more Quick cards which made its game drag out longer. This difference saves Pendragon Alter’s teams from taking additional damage throughout the battle and helps it clear the quest consistently while the original bot struggles to finish off opponents and takes extra damage because of it.

Below is an early game example where Pendragon Alter picks the two Arts cards in card slots 1 and 3 and plays a quick card to fill in slot 2. Original Pendragon would have played the three quick cards, and I am not sure if it would have been able to defeat the last enemy due to the low damage output of the Quick cards. If the enemy was not defeated then the bot’s team would have taken another round of damage.

Another area where Pendragon Alter was able to deal more damage was in the case where three quick cards were present. The original Pendragon bot would play the quick chain because that is what it was coded to do. However Pendragon Alter would not always play it and instead would play either higher damage hands or hands that would help it build up its NP gauge faster.

This is an example Pendgraon Alter playing a higher damage hand that still generates critical stars, rather than simply generating critical stars, but dealing relatively little damage.
This is an example Pendgraon Alter playing a higher damage hand that still generates critical stars, rather than simply generating critical stars, but dealing relatively little damage.

Closing Thoughts

So in this case the reinforcement learning trained Pendragon Alter beat out my hand crafted Pendragon bot. I think that this is a pretty cool result because Pendragon Alter learned how to play FGO better than I as a fairly experienced player did before I built the reinforcement learning environment. For that environment I had to do a good bit of research into how FGO calculates damage and Pendragon Alter was able to learn from that.

While Pendragon Alter beats out my original Pendragon bot, they both still do not outperform human players. I was able to win consistently in ten turns even with the same constraints as them. The main difference is I am able to apply more knowledge like making use of the class affinity system, the rock paper scissors style class system that FGO uses and making sure that I charge all of the character’s NPs so they all reach 100% rather than just trying to charge NPs without knowing whose they are charging like the bots do.

These differences are areas where the bots could be improved or given greater context of the current status of the game. Feeding in information about which cards are effective against the currently targeted enemy would help the bots prioritize which cards to play. Making it so the bot could check the current status of each character on their team might help them choose to play cards to help charge specific character’s NP even if it a sub optimal move based solely on card choice.

If I could build some of those features in the bots may be able to get to near human performance in FGO. That would be a more lofty but also quite fun update in the future that I will have to work towards.

Thank you for reading this series, I hope you found it interesting as well!


Related Articles