triplet-loss

This loss follow the ordinary TripletLoss form, but using the maximum positive distance and minimum negative distance plus the margin constant within the batch when computing the loss, as we can see in the formula:

Look into source code of tfa.losses.TripletHardLoss we can see above formula been implement exactly:

Here's a way you can do:

Ok I solved these issues with a lot of research. Now it didn't fix my problem as the code still does not work, but the issue of the loss function is fixed. Following this blog https://medium.com/@Bloomore/how-to-write-a-custom-loss-function-with-additional-arguments-in-keras-5f193929f7a0

I changes the loss function to this:

The loss should not be a Lambda layer. Remove the Lambda layer and update your code such that:

Ok so the solution was

imgplot = plt.imshow(out/255)

I finally found a way to compute the value of the triplet_loss function keeping the original architecture of my code by adding a lambda layer.

Does this work?

There are a lot of reasons why your network is performing poorly. From what I understand, your triplet generation method is fine. Here are some tips that may help improve your performance.

In deep metric learning, people usually use some pre-trained models on ImageNet classification task as these models are pretty expressive and can generate good representation for image. You can fine-tuning your model on the basis of these pre-trained models, e.g., VGG16, GoogleNet, ResNet.

Even if you have a good pre-trained model, it is often difficult to directly optimize the triplet loss using these model on your own dataset. Since these pre-trained models are trained on ImageNet, if your dataset is vastly different from ImageNet, you can first fine-tuning the model using classification task on your dataset. Once your model performs reasonably well on the classification task on your custom dataset, you can use the classification model as base network (maybe a little tweak) for triplet network. It will often lead to much better performance.

Hyper parameters such as learning rate, momentum, weight_decay etc. are also extremely important for good performance (learning rate maybe the most important factor). Since your are fine-tuning and not training the network from scratch. You should use a small learning rate, for example, lr=0.001 or lr=0.0001. For momentum, 0.9 is a good choice. For weight_decay, people usually use 0.0005 or 0.00005.

If you add some fully connected layers, then for these layers, the learning rate may be higher than other layers (0.01 for example).

As your network has several layers, you need to decide which layer to fine-tune. Researcher have found that the lower layers in network just produce some generic features such as line or edges. Typically, people will freeze the updating of lower layers and only update the weight of upper layers which tend to produce task-oriented features. You should try to optimize starting from different lower layers and see which setting performs best.

1. Fast rcnn(Section 4.5, which layers to fine-tune)
2. Deep image retrieval(section 5.2, Influence of fine-tuning the representation)

What do you get after taking tf.sqrt(d_pos) and tf.sqrt(d_neg) ?