neuroevolution | direct policy search deep reinforcement learning | Reinforcement Learning library

Template might help your case better, but has other implication.

For example, your list cannot be heterogeneous in which genome type it contains. It must be all of the same type. If you need heterogenicity, then you'll have to implement some sort of type erasure.

Here's an example that look like your example but with static polymorphism:

In short they have barely anything in common.

NEAT is an evolutionary method. This is a black box approach to optimization of functions. In this case - performance of the neural net (which can be easily measured) wrt. to its architecture (which you alter during evolution).

Reinforcement learning is about agents, learning policies to behave well in the environment. Thus they solve different, more complex problem. In theory you could learn NEAT using RL, as you might pose the problem of "given a neural network as a state, learn how to modify it over time to get better performance". The crucial difference will be - NEAT output is a network, RL output is a policy, strategy, algorithm. Something that can be used multiple times to work in some environment, take actions and obtain rewards.

Mutating a list that you are iterating over is a recipe for disaster. Consider the following simple case:

This is not the average JS question! Thanks for the links, it's a really interesting paper. I can't claim to be an expert, I have only done toy GA problems, but I did read this paper and related ones. Here is what I understand:

1. I think all you need to worry about is whether a parent, by mutation, produces the same novel gene more than once in a generation. That is, two children, whose gene with the newest innovation number are identical. You can cull those right away. I think they say that it is possible for the same gene to appear in two species at the same time, and they basically say that's fine, that's rare enough not to worry about.

2. I can find at least one reason: "In NEAT, a bias is a node that can connect to any node other than inputs."

3. I believe your question is "must nodes have an innovation number to do crossover?" The answer is no. In the original paper (e.g. Figure 4) they show crossover implemented in a way where only connections have innovation numbers.
4. If you want to change the mutation function to be architecture aware, rather than avoiding recurrent structure, you might want to explicitly add structures you do want. Suppose you want to avoid recurrent connections because you are evolving an image classifier, and you know that convolutions are more suited to the task. In this case, you want your mutation function to be able to add/remove layers (and the needed connections). This was explored in detail last year by Google Brain:

Some of the mutations acting on this DNA are reminiscent of NEAT. However, instead of single nodes, one mutation can insert whole layers—i.e. tens to hundreds of nodes at a time. We also allow for these layers to be removed, so that the evolutionary process can simplify an architecture in addition to complexifying it.

Based on your comment about your motivation for question 4, I think you are mistaken. In the XOR example in the original paper, figure 5, they show a starting phenotype that involves no hidden layer. This starting phenotype is not a solution to the XOR problem, but it provides a good starting point: "NEAT is very consistent in finding a solution. It did not fail once in 100 simulations." That is without any penalization for recurrence.

You get this error because your n_filhos or n_vars type is not an integer. I can run only the first variable separately and it returns the array.