I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.

The downstream task is a classification problem.

Transfer Learning Due to limited data, I tried to use Transfer learning model trained on "Imagenet"(ResNet50 V2 "Deep Residual Learning for Image Recognition Kaiming He, et.al").

I used the embedding from the pretrained model and trained Linear SVM and achieved a F1 score of 0.84.

Contrastive Learning I also trained a model for contrastive learning using Facenet technique("FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff, et.al") and further used the embedding for training a Linear SVM for classification problem. The achieved F1 score is 0.83.

Problem Though the scores of both the concept are closeby, I tried to evaluate the quality of both the embeddings using Silhouette Coefficient.

Overall Silhouette Coefficient:

• Transfer Learning Embedding = 0.05
• Contrastive Learning Embedding = 0.49

I do not understand this behaviour of the system that even with lower Silhouette Coefficient, the transfer learning model is able to perform well.

Kindly provide me with your views on it