transfer_learning | Transfer Learning , Multi-Modal Learning | Machine Learning library

use model.predict(img) when inference is done on one image instead of model(img)

the model will output a numerical value which is then compared with the binary value, either 1 or 0. The difference will create a loss which will then be reduced via back propagation which will drive the model output to the desired value. I think however it is better practice to use a sigmoid activation for the layer. On another level they always warn you to initially freeze the weights of the base model for initial training because of exploding gradients. I have run at least 500 models using transfer learning by leaving the base model as trainable. As a matter of fact in general I get better results training the entire model for N epochs than I do if I first train the model with the base model non trainable for M epochs, then fine tune the model for K epochs where K + M = N. In other words for the same number of epochs I get better results training the entire model vs the two step process with the same number of epochs. The difference though is in general fairly small in terms of validation and test accuracy. Further they say that if you change the base model from not trainable to trainable you have to recompile the model. I do not think that is true based on the code below

training is a boolean argument that determines whether this call function runs in training mode or inference mode. For example, the Dropout layer is primarily used to as regularize in model training, randomly dropping weights but in inference time or prediction time we don't want it to happen.

Here a simple way to operate transfer learning with your model

After checking in more detail it seems that the number of parameters depends on the kernel sizes and the number of filters of each convolutional layer, as well as the number of neurons on the final fully connected layer and some due to Batch Normalization layers in between.

Since none of these aspects depend on the size of the input images, that is, the spatial resolution may change in the output of each Convolution layer, but the size of the convolutional kernel will still be the same (e.g. 3x3x3), consequently, the number of parameters will also be fixed.

The number of parameters of this kind of network (i.e. Convolutional Neural Networks) is independent of the spatial size of the input. Nevertheless, the number of channels (e.g. 3 in an RGB colored image) must be exactly 3.

When model is trained from initialization, batchnorm should be enabled to tune their mean and variance as you mentioned. Finetuning or transfer learning is a bit different thing: you already has a model that can do more than you need and you want to perform particular specialization of pre-trained model to do your task/work on your data set. In this case part of weights are frozen and only some layers closest to output are changed. Since BN layers are used all around model you should froze them as well. Check again this explanation:

Important note about BatchNormalization layers Many models contain tf.keras.layers.BatchNormalization layers. This layer is a special case and precautions should be taken in the context of fine-tuning, as shown later in this tutorial.

When you set layer.trainable = False, the BatchNormalization layer will run in inference mode, and will not update its mean and variance statistics.

When you unfreeze a model that contains BatchNormalization layers in order to do fine-tuning, you should keep the BatchNormalization layers in inference mode by passing training = False when calling the base model. Otherwise, the updates applied to the non-trainable weights will destroy what the model has learned.

Source: transfer learning, details regarding freeze

Replace this code:

This is a good observation.

TLDR, different Input Shapes can be passed for Models of tf.keras.applications with the argument, include_top = False but that is not possible when we use tf.keras.applications with the argument, include_top = True and when we use Models of Tensorflow Hub.

Detailed Explanation:

This Tensorflow Hub Documentation states

Both are correct. One is using binary classification and another one is using categorical classification. Let's try to find the differences.

Binary Classification: In this case, the output layer has only one neuron. From this single neuron output, you have to decide either it's a cat or a dog. You can set any threshold level to classify the output. Let's say cats are labeled as 0 and dogs are labeled as 1 and your threshold value is 0.5. So, if the output is greater than 0.5, then it's a dog because it's closer to 1 otherwise it's a cat. In this case, binary_crossentropy is being used for most of the cases.

Categorical Classification: The number of output layers are exactly the same as the number of classes. This time you're not allowed to label your data as 0 or 1. Label shape should be same as the output layer. In your case, your output layer has two neurons(for classes). You will have to label your data in the same way. To achieve this, you will have to encode your label data. We call this "one-hot-encode". the cats will be encoded as (1,0) and the dogs will be encoded as (0,1) for example. Now your prediction will have two floating-point numbers. If the first number is greater than the second, it's a cat otherwise it's a dog. We call this numbers - confidence score. Let's say, for a test image, your model predicted (0.70, 0.30). which means your model is 70% for confident that it's a cat and 30% confident that it's a dog. Please note that the value of the output layer completely depends on the activation of your layer. To know deeper, please read about activation functions.