replay | React video player facilitating adaptive stream playback | Video Utils library

Use Subject. It multicasts emissions to all subscribers but doesn't emit any value to a new subscriber upon subscribe -- new subscribers simply listen in for new emissions.

To use the rand crate without std you need to manually use one of the generators that work without it. Those generators are the OsRng and the SmallRng structs. As the name suggests the first one uses the operating system's generator, which requires the getrandom crate, which probably isn't supported on SEGA Megadrive.

SmallRng should work without problems though. We cannot use the random() function, we need to manually create the generator and then call its methods.

To do this we first have to create a generator, like this:

The easy way to achieve what you want is by using RxJava 3 refCount operator. You can use an RxJava 2 <=> 3 interoperability library RxJavaBridge.

self.model.predict(state) will return a tensor of shape of (1, 2) containing the estimated Q values for each action (in cartpole the action space is {0,1}). As you know the Q value is a measure of the expected reward.

By setting self.model.predict(state)[0][action] = target (where target is the expected sum of rewards) it is creating a target Q value on which to train the model. By then calling model.fit(state, train_target) it is using the target Q value to train said model to approximate better Q values for each state.

I don't understand why you are saying that the loss becomes 0: the target is set to the discounted sum of rewards plus the current reward

I have managed to implement a workaround. However, I am not sure this is the best approach. But, it works:

You're adding a type to your list, not an instance of the type. What you're doing is essentially the same as this:

At the moment some information available in live debug and dump files are not available in TTD. The two main features are: a) QueryVirtual does not work; b) Process and thread security token information are not saved. I used QueryVirtual a lot in my Debug extension and I made changes to deal with it. Sorry to be the bearer of bad news.

Turns out I can do this;

The difference is much more conventional rather than functional - the difference being side-effects vs a final consumer.

The doOnXXX series of methods are meant for user-designed side-effects as the reactive chain executes - logging being the most normal of these, but you may also have metrics, analytics, etc. that require a view into each element as it passes through. The key with all of these is that it doesn't make much sense to have any of these as a final consumer (such as the println() in your above example.)

On the contrary, the subscribe() consumers are meant to be a "final consumer", and usually called by your framework (such as Webflux) rather than by user code - so this case is a bit of an exception to that rule. In this case they're actively passing the messages in this reactive chain to another sink for further processing - so it doesn't make much sense to have this as a "side-effect" style method, as you wouldn't want the Flux to continue beyond this point.

(Addendum: As said above, the normal approach with reactor / Webflux is to let Webflux handle the subscription, which isn't what's happening here. I haven't looked in detail to see if there's a more sensible way to achieve this without a user subscription, but in my experience there usually is, and calling subscribe manually is usually a bit of a code smell as a result. You should certainly avoid it in your own code wherever you can.)