ppo | Proximal Policy Optimization implementation with TensorFlow | Reinforcement Learning library

You can always create your own/custom policy network then you have full control over the layers and also the initialization of the weights.

If you want to use the default model you have the following params to adapt it to your needs:

Define the variables L0, L1, L2 and L3 in the function lag()

If I understand you correctly you just need a row number partitioned by the patient and then a CASE expression to convert that into a multiplier. I've added an id column to the sample data to allow for an order by (which you need for a row number).

Depending on how messy you're ok with it getting, there are a few options.

First, let me point out that it's natural for tables to have columns of varying sizes and you rarely get a better look out of them by equalizing their widths. With that said, here is how to accomplish what you want.

1. Bootstrap/Reactstap has classes for popular width percents (e.g. 25% 50% etc). So if you know that your table will always have 4 columns for example, you can give your ths the appropriate classes:

As you mentioned in your question, PPO, DDPG, TRPO, SAC, etc. are indeed suitable for handling continuous action spaces for reinforcement learning problems. These algorithms will give out a vector of size equal to your action dimension and each element in this vector will be a real number instead of a discrete value. Note that stochastic algorithms like PPO will give a multivariate probability distribution from which you sample the actions.

Most of the robotic environments in Mujoco-py, PyBullet, Robosuite, etc. are environment with multiple continuous action spaces. Here the action spaces can be of the form [torque_for_joint_1, torque_for_join_2, ..., torque_for_joint_n] where torque_for_joint_i can be a real valued number determining by how much would that joint move.

Regarding implementations for solving these environments, robosuite does offer sample solutions for benchmarking the environments with different algorithms. You could also look up stable-baselines or one of the standard RL libraries.

The first problem is that you have a print command inside your loop. Since you won't know which rows have the datetime closest to chosen_datetime until after you have looped over all the items, this is premature and is a significant cause of your erroneous output.

Secondly, since you're looking for the closest datetime per vehicle number you're going to need some logic to group things by vehicle number.

One option would be a solution using itertools.groupby; another solution -- that I've implemented here -- would be store results in a dictionary keyed by the vehicle number.

There are a few comments in the following code, but let me know if you'd like some additional detail.

The A2C code fails due to the configuration you copied from the PPO trial: "sgd_minibatch_size", "kl_coeff" and many others are PPO-specific configs, which cause the problem when running using A2C.

The error is explained in the "error.txt" in the logdir.

Are you sure, this is your actual code? In the code snippet above, the name PPOPolicy is not even defined. We would need to see the code of PPOPolicy. Obviously its constructor (its __init__ method) expects something as its first argument which has a shape arttribute - so I guess, it expects a pandas dataframe. Your envF does not have a shape attribute, so this leads to the error.

Just judging from the names in your snippet, I guess you should write

The function in multiprocessing.Pool.map expects one argument. One way to adapt your code is to write a small wrapper function that takes env, alg, and seed as one argument, separates them, and passes them to run.

Another option is to use multiprocessing.Pool.starmap, which allows multiple arguments to be passed to the function.