mcts | An implementation of Monte Carlo Tree Search in python | Reinforcement Learning library

I'm using C++ templates and encounter the following issue.

I try to use class ImplG's method func in class ImplM, however the linker reports an error.

I've been working on a MCTS AI for a couple days now. I tried to implement it on Tic-Tac-Toe, the least complex game I could think of, but for some reason, my AI keeps making bad decisions. I've tried change the values of UCB1's exploration constant, the number of iterations per search, and even the points awarded to winning, losing, and getting to tie the game (trying to make a tie more rewarding, as this AI only plays second, and try to get a draw, win otherwise). As of now, the code looks like this:

I'm trying to implement MCTS on Openai's atari gym environments, which requires the ability to plan: acting in the environment and restoring it to a previous state. I read that this can be done with the ram version of the games:

recording the current state in a snapshot: snapshot = env.ale.cloneState()

restoring the environment to a specific state recorded in snapshot: env.ale.restoreState(snapshot)

so I tried using the ram version of breakout:

I've been recently trying to play with MCTS implementation for simple board game. I'd like to make AI play with itself to gather some sample playthroughs. I'd figure out I could make them use the same MCTS tree (for better performance). Or so it looks like.

But would that be valid ? Or I need 2 separate trees for both AI with separate win/plays data to behave correctly ?

How can I use related generic types in Rust?

Here's what I've got (only the first line is giving me trouble):

I am looking for a method of simulating my game till win or lose, to feed into a Monte Carlo Tree Search algorithm.

My game is a turn based , tile based tactical RPG similar to Final Fantasty Tactics, Fire Emblem etc.

The idea is that the AI would perform thousands of playouts (or up until a threshold) until they determine the optimal next move.

Each AI and Player agent would make a random valid move, until the game is over.

Why a MCTS Simulation? Why not minmax?

I need to simulate the game as closely to the real thing for several reasons:

1. Encoding the game state into a lower-dimensional structure is impossible as most actions are tightly coupled to Unity constructs like Colliders and Rays.
2. It is fairly difficult, if not impossible to statically evaluate the game state at X moves ahead - without any knowledge of previous moves. Therefore I would need to carry out each move sequentially on a game state, to produce the next game state before anything can be evaluated.

To expand on point 2: Using a simple minmax approach, and statically evaluating the game state by looking at something like the Current Health of all players, would be useful but not accurate. As not every action will provide an immediate change to health.

Example:

Which produces a higher (max damage) dealt over 2 turns:

1. Move infront of player, attack -> Move behind player, attack

OR

1. Move infront of player, Use attack buff -> Attack for x4 damage

In this example, the minmax approach would never result in the 2nd option, even though it does more damage over 2 turns, due to its static evaluation of the buff move resulting in 0, or perhaps even negatively.

In order for it to select the 2nd option, it would need to retain knowledge of previous actions. Ie. it would need to simulate the game almost perfectly.

When we add in other elements like: Stage Traps, destructible environment and status effects. It becomes pretty much impossible to use a static evaluation

Time.timeScale

This allows me to speed up physics and other interactions, which is exactly what I need. However - this is a global property, so the game will appear to run at super speed for a fraction of a second when the AI is "thinking".

Increasing the speed of NavMesh Agents

All my movements take place on a NavMesh - so the only perceivable way of making these movements "instant" is to increase the speed. This is problematic as the movements are not fast enough - and it causes physics issues due to the increased velocity, sometimes characters spin out of control and fly off the map.

For reference here is a screenshot of my (in active development) game.

What I need is a method for "playing" my game extremely quickly.

I just need to be able to run these simulations quickly and efficiently before every AI move.

I would love to hear from someone with some experience doing something like this - but any input would be greatly appreciated!

Thanks

I am working on macOS Catalina and I want to compile some c++ code using CMake.

This code needs the library boost that I installed via Brew (brew install boost). In my code I added #include to use it.

When I compile I got the following error: