Twitter-Sentiment | Sentiment analyzer for your tweets | Natural Language Processing library

Found the solution, and it was in fact simple! Posting the answer here, to help others facing a similar issue, because I was totally stuck on this for 2 days!

My issue was caused by a dependency update.

According to the release notes:

The new (V 2.0.0) major release brings several performance improvements, but also snuck into the bottom of the release notes was this:

This release is not backward-compatible, due to:

• A breaking change related to utf-8 encoding in engine.io-parser (socketio/engine.io-parser#81)
• an update to make the socket id on the client match the id on the server-side (socketio/socket.io-client#1058)

Please note that if you are using a self-signed certificate, rejectUnauthorized now defaults to true (socketio/engine.io-client#558).

Here's the full V2 release notes: https://github.com/socketio/socket.io/releases/tag/2.0.0

Make your app https, or sign your own requests, as per documentation. As a quick fix though, I just downgraded back to socket.io V 1.7.0 and it works perfectly

You absolutely do want the x_validation to be related to the y_validation, i.e. correspond to the same rows as you had in your original dataset. e.g. if Validation takes rows 1,3,7 from the input x, you would want rows 1, 3, 7 in both the x_validation and y_validation.

The idea of the train_test_split function to divide your dataset up into a two sets of features (the xs) and the corresponding labels (the ys). So you want and require

if you have strings in column then you would have to split every sentence into list of words and then put all list in one list - you can use it sum() for this - it should give you all words. To get unique words you can convert it to set() - and later you can convert back to list()

But at start you would have to clean sentences to remove chars like ., ?, etc. I uses regex to keep only some chars and space. Eventually you would have to convert all words into lower or upper case.

When doing machine learning, we want to learn an algorithms that performs well on new (unseen) data. This is called generalization.

The purpose of the test set is, amongst others, to verify the generalization behavior of your classifier. If your model predicts the same labels for each test instance, than we cannot confirm that hypothesis. The test set should be representative of the conditions in which you apply it later.

As a rule of thumb, I like to think that you keep 50-25% of their data as a test set. This of course depends on the situation. 30/4000 is less than one percent.

A second point that comes to mind is that when your classifier is biased towards one class, make sure each class is represented nearly equally in the training and validation set. This prevents the classifier from 'just' learning the distribution of the whole set, instead of learning which features are relevant.

As a final note, normally we report metrics such as precision, recall and F_β=1 to evaluate our classifier. The code in your sample seems to report something based on the global sentiment in all tweets, are you sure that is what you want? Are the tweets a representative collection?

Your dataset is highly imbalanced. You yourself mentioned it in one of the comment, you have 550 positive and 550 negative labelled tweets but 4000 neutral that's why it always favours the majority class. You should have equal number of utterances for all classes if possible. You also need to learn about evaluation metrics, then you'll see most probably your recall is not good. An ideal model should stand good on all evaluation metrics. To avoid overfitting some people also add a fourth 'others' class as well but for now you can skip that.

Here's something you can do to improve performance of your model, either (add more data) oversample the minority classes by adding possible similar utterances or undersample the majority class or use a combination of both. You can read about oversampling, undersampling online.

In this new datset try to have utterances of all classes in this ratio 1:1:1 if possible. Finally try other algos as well with hyperparameters tuned through grid search,random search or tpot.

edit: in your case irrelevant is the 'others' class so you now have 4 classes try to have dataset in this ratio 1:1:1:1 for each class.

One should preprocess the text before feeding into the model, following is the minimal working script(adapted from https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis/):

The example you are trying to run is desinged for running in spark 1.5. You should either download spark 1.5 or run the spark-submit from spark 2.1.0 but with kafka package related to 2.1.0, for example: ./bin/spark-submit --packages org.apache.spark:spark-streaming-kafka-0-8_2.11:2.1.0.

You're creating a virtual environment, activating it, and then installing twine outside of it.

Remove --user from pip install --user --upgrade twine