feature_extraction | 文本特征提取算法，卡方校验（chi-square）和信息增益算法提取文本特征算法实现 | Computer Vision library

A ROC curve is based on soft predictions, i.e. it uses the predicted probability of an instance to belong to the positive class rather than the predicted class. For example with sklearn one can obtain the probabilities with predict_proba instead of predict (for the classifiers which provide it, example).

Note: OP used the tag multiclass-classification, but it's important to note that ROC curves can only be applied to binary classification problems.

One can find a short explanation of ROC curves here.

Imo there are a couple of points to be highlighted on this example:

• CountVectorizer requires its input to be 1D. In such cases, documentation for ColumnTransformer states that

columns: str, array-like of str, int, array-like of int, array-like of bool, slice or callable

Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where transformer expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer.

Therefore, the columns parameter should be passed as an int rather than as a list of int. I would also suggest Sklearn custom transformers with pipeline: all the input array dimensions for the concatenation axis must match exactly for another reference.

• Given that you're using a column transformer, I would pass the whole dataframe to method .fit() called on the ColumnTransformer instance, rather than X only.

• The dataframe seems to have missing values; it might be convenient to process them somehow. For instance, by dropping them and applying what is described above I was able to make it work, but you can also decide to proceed differently.

Your question is a combination of text mining tasks, which I try to briefly address here. The first step is, as always in NLP and text mining projects, the cleaning one, including removing stop words, stop characters, etc.:

Instead of using accuracy for model evaluation, you should use a measure that works well with class imbalance.

Have a look at https://scikit-learn.org/stable/modules/generated/sklearn.metrics.balanced_accuracy_score.html

If you average the accuracy per class, then also classes with a small amount of samples will be optimized.

Using only accuracy the best thing your classifier can learn is to always say: No spam. (Because after all, most of mails are not spam.)

The issue is that both CountVectorizer and TfidfVectorizer require their input to be 1D (and not 2D). In such cases the doc of ColumnTransformer states that parameter columns of the transformers tuple should be passed as a string rather than as a list.

columns: str, array-like of str, int, array-like of int, array-like of bool, slice or callable

Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where transformer expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer. A callable is passed the input data X and can return any of the above. To select multiple columns by name or dtype, you can use make_column_selector.

Therefore, the following will work in your case (i.e. changing ['Text'] into 'Text').

The reason you are getting the error is because TfidfVectorizer only accepts lists as the input. You can check this from the documentation itself.

Here you are passing a Dataframe as the input. Hence the weird output. First convert your dataframe to lists using:

You can use remainder='passthrough' to avoid transforming already processed columns (therefore in your case you can just consider the binary columns as residual columns that your ColumnTransformer object won't process, but on which it will pass through). Then you should be aware that CountVectorizer expects a 1D array as input and therefore you should specify the columns to be passed to make_column_transformer as a string ('Text'), rather than as an array (['Text']) (see reference from make_column_transformer() doc).

columns : str, array-like of str, int, array-like of int, slice, array-like of bool or callable

Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where transformer expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer. A callable is passed the input data X and can return any of the above. To select multiple columns by name or dtype, you can use make_column_selector.

You can use options in CountVectorizer to change behaviour - ie. token_pattern or tokenizer.

If you use token_pattern='.+'

You could try using a preprocessing function in your ImageDataGenerator combined with tf.image.extract_patches: