proof-of-concepts | little collection of fun and creative proof | Security Testing library

There are two problems with your training dataset:

1. It is too small and not representative enough
2. You gave twice more data when training your Japan label comparing with other labels

So, Japan label's model is trained on two sentences whose words are completely non-related and do not repeat. Other labels are trained on just one short sentence.

This leads to underfitted Japan label model that has "not learned enough" from the training data, and is not able to model the training data properly nor generalize to new data. In other words, it is too general and triggers on almost any sentence. Rest labels' models are overfitted - they model the training data too well and trigger only on those sentences that are very close to training set data.

So Japan label catches almost any sentence. And going in the begin of your labels list, it catches all sentences before any label that goes after it in list has a change to evaluate a sentence. Of course you can move Japan labels at the end of the list, but the better solution is - to enlarge your training data set for all labels.

You can also evaluate overfitted label model effect - try for example add to your test set "London bridge down" and "London down" sentences - the first gives you London, the second - Japan, because the first sentence is close enough to the sentence training set for London label and the second - isn't.

So keep adding the training set data exactly in this manner, just make your training set big and representative enough.