tf-agent | tensorflow reinforcement learning agents for OpenAI gym | Reinforcement Learning library

I have a bunch of Java code that constitutes an environment and an agent. I want to use one of the Python reinforcement learning libraries (stable-baselines, tf-agents, rllib, etc.) to train a policy for the Java agent/environment. And then deploy the policy on the Java side for production. Is there standard practice for incorporating other languages into Python RL libraries? I was thinking of one of the following solutions:

1. Wrap Java env/agent code into REST API, and implement custom environment in Python that calls that API to step through the environment.
2. Use Py4j to invoke Java from Python and implement custom environment.

Which one would be better? Are there any other ways?

Edit: I ended up going the former - deploying a web server that encapsulates the environments. works quite well for me. Leaving the question open in case there is a better practice to handle this kind of situations!

I have a policy that I read from disk using the function SavedModelPyTFEagerPolicy. For troubleshooting the environment definitions, I would like to examine the predicted value of different states.

I have had success using these instructions to extract the actions from the policy for test cases. Is there a function that will allow me to extract the predicted values associated with those actions?

I am current working on using Python.NET to build C# environments for interaction TensorFlow Agents and am receiving a TensorFlow error attempting to load Cuda DLLs.

When I run pure python examples Tensor flow loads the CUDA DLLs without issue:

I apologize in advance for the question in the title not being very clear. I'm trying to train a reinforcement learning policy using tf-agents in which there exists some unobservable stochastic variable that affects the state.

For example, consider the standard CartPole problem, but we add wind where the velocity changes over time. I don't want to train an agent that relies on having observed the wind velocity at each step; I instead want the wind to affect the position and angular velocity of the pole, and the agent to learn to adapt just as it would in the wind-free environment. In this example however, we would need the wind velocity at the current time to be correlated with the wind velocity at the previous time e.g. we wouldn't want the wind velocity to change from 10m/s at time t to -10m/s at time t+1.

The problem I'm trying to solve is how to track the state of the exogenous variable without making it part of the observation spec that gets fed into the neural network when training the agent. Any guidance would be appreciated.

I'm using TF-Agents library for reinforcement learning, and I would like to take into account that, for a given state, some actions are invalid.

How can this be implemented?

Should I define a "observation_and_action_constraint_splitter" function when creating the DqnAgent?

If yes: do you know any tutorial on this?

Suppose that two integer arrays min and max are given and they have equal shape. How to generate all Numpy arrays such that min[indices] <= ar[indices] <= max[indices] for all indices in np.ndindex(shape)? I have looked at the Numpy array creation routines but none of them seem to do what I want. I considered also starting with the min array and looping over its indices, adding 1 until the corresponding entry in max was reached, but I want to know if Numpy provides methods to do this more cleanly. As an example, if