multiagent-particle-envs | agent particle environment used in the paper | Reinforcement Learning library

Answering my own question, let's consider the simple_tag environment.

env.action_space for this environment gives:

[Discrete(5), Discrete(5), Discrete(5), Discrete(5)] (4 agents)

This is what I found misleading. I thought the actions would have to be a list of 4 elements, something like: [0, 3, 4, 1] but what it expects is a one-hot vector (of 5 elements) for all 4 agents. So, the correct way to encode actions is:

[array([1., 0., 0., 0., 0.]), array([0., 0., 1., 0., 0.]), array([0., 0., 0., 0., 1.]), array([0., 0., 0., 1., 0.])]

(depending on the environment)