transfer_learning_tutorial | A guide to transfer | Machine Learning library

It's probably more difficult for the network to find the matching class between 20 classes than between two classes.

For example if you give it a dog image and it need to classify it between cat, dog and horse it could send 60% cat, 30% dog 10% horse and then be wrong while if it needs to classify it only between dog and horse it would give may be 75% dog, 25% horse and then be wright.

The finetunnig will also be longer so you could have better result if you train it longer with the 20 classes if you haven't stop it after convergence but after a fix number of epochs.

The particular way the tutorial on dataloading uses the custom dataset is with self defined transforms. The transforms must be designed to fit the dataset. As such, the dataset must output a sample compatible with the library transform functions, or transforms must be defined for the particular sample case. Choosing the latter, among other things has resulted in completely functional code.

Answer given by ptrblck of PyTorch community. Thanks a lot!

That is something that can happen when performing transfer learning called catastrophic forgetting. Basically, you update your pretrained weights too much and you 'forget' what was previously learned. This can happen notably if your learning rate is too high. I would suggest trying at first a lower learning rate, or using diffentiable learning rate (different learning rate for the head of the network and the pretrained part, so that you can have a higher learning rate on the fc layers than for the rest of the network).

This is probably not the best idea, but you can do something like this:

transforms.Compose just clubs all the transforms provided to it. So, all the transforms in the transforms.Compose are applied to the input one by one.

1. transforms.RandomResizedCrop(224): This will extract a patch of size (224, 224) from your input image randomly. So, it might pick this path from topleft, bottomright or anywhere in between. So, you are doing data augmentation in this part. Also, changing this value won't play nice with the fully-connected layers in your model, so not advised to change this.
2. transforms.RandomHorizontalFlip(): Once we have our image of size (224, 224), we can choose to flip it. This is another part of data augmentation.
3. transforms.ToTensor(): This just converts your input image to PyTorch tensor.
4. transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]): This is just input data scaling and these values (mean and std) must have been precomputed for your dataset. Changing these values is also not advised.

1. transforms.Resize(256): First your input image is resized to be of size (256, 256)
2. transforms.CentreCrop(224): Crops the center part of the image of shape (224, 224)

Rest are the same as train

P.S.: You can read more about these transformations in the official docs

1. Why ...ResizedCrop? - This answer is straightforward. Resizing crops to the same dimensions allows you to batch your input data. Since the training images in your toy dataset have different dimensions, this is the best way to make your training more efficient.

2. Why Random...? - Generating different random crops per image every iteration (i.e. random center and random cropping dimensions/ratio before resizing) is a nice way to artificially augment your dataset, i.e. feeding your network different-looking inputs (extracted from the same original images) every iteration. This helps to partially avoid over-fitting for small datasets, and makes your network overall more robust.

   You are however right that, since some of your training images are up to 500px wide and the semantic targets (ant/bee) sometimes cover only a small portion of the images, there is a chance that some of these random crops won't contain an insect... But as long as the chances this happens stay relatively low, it won't really impact your training. The advantage of feeding different training crops every iteration (instead of always the same non-augmented images) vastly counterbalances the side-effect of sometimes giving "empty" crops. You could verify this assertion by replacing RandomResizedCrop(224) by Resize(224) (fixed resizing) in your code and compare the final accuracies on the test set.

   Furthermore, I would add that neural networks are smart cookies, and sometimes learn to recognize images through features you wouldn't expect (i.e. they tend to learn recognition shortcuts if your dataset or losses are biased, c.f. over-fitting). I wouldn't be surprised if this toy network is performing so well despite being trained sometimes on "empty" crops just because it learns e.g. to distinguish between usual "ant backgrounds" (ground floor, leaves, etc.) and "bee backgrounds" (flowers).

Its purpose is also to artificially augment your dataset. For the network, an image and its flipped version are two different inputs, so you are basically artificially doubling the size of your training dataset for "free".

There are plenty more operations one can use to augment training datasets (e.g. RandomAffine, ColorJitter, etc). One has however to be careful to choose transformations which are meaningful for the target use-case / which are not impacting the target semantic information (e.g. for ant/bee classification, RandomHorizontalFlip is fine as you will probably get as many images of insects facing right than facing left; however RandomVerticalFlip doesn't make much sense as you won't get pictures of insects upside-down most certainly).

You are subclassing nn.Module to define a function, in your case Loss function. So, when you compute loss.backward(), it tries to store the gradients in the loss itself, instead of the model and there is no variable in the loss for which to store the gradients. Your loss needs to be a function and not a module. See Extending autograd.

You have two options here -

1. The easiest one is to directly pass cust_loss function as criterion parameter to train_model.
2. You can extend torch.autograd.Function to define the custom loss (and if you wish, the backward function as well).

P.S. - It is mentioned that you need to implement the backward of the custom loss functions. This is not always the case. It is required only when your loss function is non-differentiable at some point. But, I do not think so that you’ll need to do that.

Finally, I solved this problem. First, I compared two models' parameters and found out they were the same. So I confirmed that the model is the same. And then, I checked out two inputs and surprisedly found out they were different.

So I reviewed two models' inputs carefully and the answer was that the arguments passed to the second model did not update.

Code: