stable-baselines | OpenAI Baselines , implementations of reinforcement | Reinforcement Learning library

try this instead:

Replaced this:

You can use compatibility layer of TensorFlow.

All v1 functionality is available under tf.compat.v1 namespace.

I managed to load your model in TF 2.1 (nothing special about that version, I just have it locally):

Yes! I got the answer!!

So, when I looked at __init__.py in the cv2 library, I found this line:

You created a custom environment alright, but you didn't register it with the openai gym interface. That's what the env_id refers to. All environments in gym can be set up by calling their registered name.

So basically what you need to do is follow the set up instructions here and create the appropriate __init__.py and setup.py scripts, and follow the same file structure.

At the end locally install your package using pip install -e . from within your environment directory.

While @VonC has provided the fact that it's not possible to have two different fork of the forks, I want to add two things to this answer:

First, this seems to be an unfortunate design decision by GitHub and there is no technical reason for not allowing this. Every repo should be able to point to whatever upstream it wants to enable PR mechanism to work. For a platform that intends to promote collaborative development, this is quite a mind-boggling limitation.

Second, I want to provide a way so you can still do this but with extra work from your part. Let's say there is repo R and their two forks R1 and R2 by some users. You want to be able to work on R, R1 and R2 and send them your pull requests. Here's how you can accomplish this:

1. Clone R locally and then create new github repo from it. This would mean that your repo would not appear as fork and you can't send PRs from it to R.
2. Add remotes for each fork using git remote add r1_fork https://github.com//R1.git.
3. For each fork create its own branch using git checkout -b r1_fork.
4. In each branch, fetch the changes from that fork: git fetch r1_fork.
5. After fetch do hard reset in that branch for that remote: git reset --hard r1_fork/master.

Now you can work in branch for each individual remote and make changes. However, you can't send PRs yet. For sending PRs, you will need to fork that repo, then merge changes from your branch to the fork, send PR and delete the fork. Obviously, this is very non-ideal but this is what we got until GitHub fixes their stuff.

You may want to try rllib package of ray, which is extended in UC at Berkeley.

https://rise.cs.berkeley.edu/projects/ray/

It includes a lot of implemented algorithms:

and it is quote easy to use. You just need to add your environment which is fully explained at: https://ray.readthedocs.io/en/latest/rllib-env.html

I posted the question as an issue in stable-baselines and they answered. I will copy here as a reference to others:

You are trying to save the action placeholder used in PPO training (part of PPO agent), but for inference you only need the trained policy and its placeholders (model.act_model). The code on colab runs without errors by changing call to simple_save to this:

tf.saved_model.simple_save(model.sess, checkpoint_name, inputs={"obs": model.act_model.obs_ph}, outputs={"action": model.act_model._policy_proba})

The value of _policy_proba depends on the environment/algorithm.

The definition of the class MlpLnLstmPolicy is given here. This class inherits methods from LstmPolicy which is defined here.

The default activations used in tanh. You could go through the other default parameters given in the definition of LstmPolicy for more information.