svm | C wrappers to libsvm for easy-to-use Support Vector | Machine Learning library

I saw anna_phog on other portal.

Problem is because this function needs numpy array but you read image with pillow Image.open() and you have to convert img to numpy array

Here is how I would do it:

I'm not sure what is the cause of the problem, but since the output of you SVM model and DecisionTreeClassfier always output 1, I suggest you try a more complex model like RandomForestClassifier and see what it comes out.

I've similar experience before, no matter how I tuned the training hyperparameters, the model always give the same performance metric -- this may cause by 2 probabilities:

1. Our data is not suitable for the model, for example all values in the vector is zero: [0, 0, 0, 0, 0, 0, 0]
2. Our model is too simple, which could only perform linear modeling, so that it could not learn too complex mapping function.

Since your SVM is built with linear kernel, could you try an more complex model and see what it comes out? And could you examine that if your X_train_vectors is all zero's in the matrix?

The actual problem is really silly, but since it took quite some time to track down, I'll share an outline of my debugging.

First, rather than plotting, print the actual values of the decision_function: you'll find that the first one comes out unique, but that after that everything is constant. Running the same on various slices of the dataset, this pattern persists. So I thought perhaps some values were being overwritten, and I dug into the SVC code a bit. That lead to some useful internal functions/attributes, like ._BaseLibSVM__Xfit containing the training data, _decision_function and _dense_decision_function, and _compute_kernel. But none of the code indicated a problem, and running them just showed the same problem. Running _compute_kernel gave results that were all-zeros past the first row, and then coming back to your code, running linear_kernel does that already. So, finally, it comes back to your linear_kernel function.

You return inside the outer for loop, so you only ever use the first row of X, never computing the rest of the matrix. (This brings up a surprise: why did the predictions look good? That seems to have been a fluke. Changing the definition for f_lin, to change the classes, the model still learns the slope-1 line.)

I recommend to use Functional API in order to define multiple outputs of your model because of a more clear code. However, you can do this with Sequential model by getting the output of any layer you want and add to your model's output.

Print your model.summary() and check your layers to find which layer you want to branch. You can access each layer's output by it's index with model.layers[index].output .

Then you can create a multi-output model of the layers you want, like this:

Use more_itertools.powerset which will result in list of tuples so convert it into list of list using map()

This is happening because you have a text transformer object in your pipeline. The problem with this approach is that the pipeline will pass the whole dataframe to the TfidfVectorizer. However, the text transformers of scikit-learn expect a 1d input.

Passing a 2d dataframe to TfidfVectorizer causes some weird processing where it mistakes the column names as documents. You can check with this simple example:

.scatter() funciton takes float or array-like of x and y coordinates. As these are coordinates they are supposed to be in pairs and therefore x and y should be of equal length. In your code x is clf.support_vectors_[:0] and y is clf.support_vectors_[:1]. The syntax iterator[:k] says that we should pick every element in the iterator until the k-th element which we should not pick. So when we combine these things together we can spot that x and y have different lengths and therefore throw an error.

There is nothing wrong with your code by itself. But your step-by-step approach is using bad practice in Machine Learning theory:

Do not resample your testing data

In your step-by-step approach, you resample all of the data first and then split them into train and test sets. This will lead to an overestimation of model performance because you have altered the original distribution of classes in your test set and it is not representative of the original problem anymore.

What you should do instead is to leave the testing data in its original distribution in order to get a valid approximation of how your model will perform on the original data, which is representing the situation in production. Therefore, your approach with the pipeline is the way to go.

As a side note: you could think about shifting the whole data preparation (vectorization and resampling) out of your fitting and testing loop as you probably want to compare the model performance against the same data anyway. Then you would only have to run these steps once and your code executes faster.