backprop | Backpropagation in Python , C , and Cuda | GPU library

My network is not trained to recognize inputs separately, it either outputs the averaged result or becomes biased to one particular output. What am I doing wrong?

I have a pytorch model I'm trying to use to do facial recognition. I am using the same model structure, loss, and optimizer as a working code, but it seems like the backprop won't do anything, any output of the NN is just 0.5. Here is the code, any help/suggestions is/are appreciated.

I have two tensors:

I recently coded a neural network based on this online book and Sebastian Lague's brief series on neural networks on youtube. I coded it as faithfully to the original as possible but it didn't end up working. I am trying to solve a simple XOR problem with it but it always seems to give me random but similar values. I even tried copying and pasting the author's code, without changing anything, but it still didn't work.

I am trying to implement the code from a Pytorch beginner's tutorial. But I have written the code for loading the saved model in another Python file. The FashionClassify file contains the code exactly as its in the tutorial.

Below is the code:

I am working on 3D point clouds. I have the SPARSE MATRIX representation of the graph structure of the point cloud (like csr_matrix in scipy.sparse). I want to club together the points that are within certain threshold of the Geodesic distance (approximated by the path length in the graph) and process them together. TO FIND such points, I need to run some shortest path finding algorithm like Dijkstra's. In a nutshell, my idea is like this

1. Sample K points out of N points (that I could do using Furthest Point Sampling)
2. Find the nearest Geodesic neighbours (using BackProp supported algorithm) for each of K points
3. Process the neighbours for each point using some Neural Network

This will go in my forward function. Is there a way to implement Dijkstra’s in my functionality?

Or any other idea that I can implement?

Thank you very much!

I have attempted to solve this error but it has been to no avail. My CNN model is below:

The shape of X_train and X_test are: X_train shape: torch.Size([12271, 3, 100, 100]) | X_test shape: torch.Size([3068, 3, 100, 100])

I have two tensors of shape [B, 3 , 240, 320] where B represents the batch size 3 represents the channels, 240 the height(H), 320 the width(W).

I need to find the dot product along the channels dimension(3 channels) thus the resulting tensor would be of shape [B, 1 , 240, 320]. My tensors have float32 elements in gpu(cuda to backprop).

Can you all please suggest how I can do that?

Thanks!

More clarification:

Let's say we have B=10, H=100, W=200. So from the above would be common for both the first and seconds tensors. If we keep B, H, W constant we get a 1D vector as the resultant tensor(with 3 elements). I need to take the dot product of these two vectors. Thus the resultant tensor is of dimension [B, 1, 240, 320]

After a full week of print statements, dimensional analysis, refactoring, and talking through the code out loud, I can say I'm completely stuck.

The gradients my cost function produces are too far from those produced by finite differences.

I have confirmed my cost function produces correct costs for regularized inputs and not. Here's the cost function: