dqn | DQN implementation in PyTorch | Machine Learning library

I want to compile my DQN Agent but I get error: AttributeError: 'Adam' object has no attribute '_name',

I built a simulation model where trucks collect garbage containers based on their fill level. I used OpenAi Gym and Tensorflow/keras to create my Deep Reinforcement Learning model... But my training has a very high loss... Where did I go wrong? Thanks in advance

this is the Env

I am trying to create a DQN model for mario environment. But when I try to create the model it gives me this error:

MemoryError: Unable to allocate 229. GiB for an array with shape (1000000, 1, 4, 240, 256) and data type uint8

This is the code for creating the model:

I am trying to set a Deep-Q-Learning agent with a custom environment in OpenAI Gym. I have 4 continuous state variables with individual limits and 3 integer action variables with individual limits.

Here is the code:

Hello I am struggling with a Problem in my Python code.

My problem is the following: I want to copy the value of an attribute of an object. I create the object agent, then in the function train_agent() I copy the value of the attribute state_vec to the variable last_state_vec. The Problem now is, when I change last_state_vec I automatically also change the attribute state_vec of agent state_vec = np.array(0,0,10,0,0). How can I only copy the value of state_vec, so that it doesn't get changed in this case. I want state_vec to stay the zero vector. Here's some of my code:

I have two different problems occurs at the same time.

I am having dimensionality problems with MaxPooling2d and having same dimensionality problem with DQNAgent.

The thing is, I can fix them seperately but cannot at the same time.

First Problem

I am trying to build a CNN network with several layers. After I build my model, when I try to run it, it gives me an error.

I would like to train a DQN Agent with Keras-rl. My environment has both multi-discrete action and observation spaces. I am adapting the code of this video: https://www.youtube.com/watch?v=bD6V3rcr_54&t=5s

Then, I am sharing my code

I am new to pytorch and I am working on DQN for a timeseries using Reinforcement Learning and I needed to have a complex observation of timeseries and some sensor readings, so I merged two neural networks and I am not sure if that's what is ruining my loss.backward or something else. I know there is multiple questions with the same title but none worked for me, maybe I am missing something.
First of all, this is my network:

I'm trying to implement a DQN. As a warm up I want to solve CartPole-v0 with a MLP consisting of two hidden layers along with input and output layers. The input is a 4 element array [cart position, cart velocity, pole angle, pole angular velocity] and output is an action value for each action (left or right). I am not exactly implementing a DQN from the "Playing Atari with DRL" paper (no frame stacking for inputs etc). I also made a few non standard choices like putting done and the target network prediction of action value in the experience replay, but those choices shouldn't affect learning.

In any case I'm having a lot of trouble getting the thing to work. No matter how long I train the agent it keeps predicting a higher value for one action over another, for example Q(s, Right)> Q(s, Left) for all states s. Below is my learning code, my network definition, and some results I get from training