Xtest | Unit-testing domain-specific language for Java

One approach could be to base the threshold on the proportion of positive classes seen in your training data. For example, if 20% of cases in your training data have a positive class, then you could pick the same proportion of cases in your test data, classifying the 20% with the highest probabilities as positive.

Whether this approach is appropriate or not really depends on your classification problem. The model can tell you the probability of a case being positive or negative - how you decide to use those probabilities to make a classification is a question to be considered outside of the modelling, and really depends on the cost of mis-classifying in your particular case.

The encoder and decoder functions expect an input_shape sequence. But with

IIUC, You need Recurrent Neural Network(RNN) or Long short-term memory(LSTM), you can solve your problem with tensorflow like below:

Linear regression is an affine model, in the sense that the prediction is of form

You just have to make sure that x and y have the same number of samples, meaning their first dimensions are the same. Here is a working example:

Did you look at the probabilities versus the fitted values? You can read about how probability works with SVM here.

If you want to look at the performance you can use the library DescTools and the function Conf or with the library caret and the function confusionMatrix. (They provide the same output.)

Look at the last layer in your model summary. Since you set the parameter return_sequences to True in the RNN layer, you are getting a sequence with the same number of time steps as your input and an output space of 200 for each timestep, hence the shape (2400, 1403, 200), where 2400 is the batch size. If you set this parameter to False, everything should work, since your labels have the shape (2400, 1).

Working example:

TL;DR: PowerTransformer, which you seem to apply only in the GridSearchCV case, produces negative data, which makes MultinomialNB to expectedly fail, es explained in detail below; if your initial xtrain and ytrain are indeed TF-IDF features, and you do not transform them similarly with PowerTransformer (you don't show something like that), the fact that they work OK is also unsurprising and expected.

Although not terribly clear from the documentation:

The multinomial Naive Bayes classifier is suitable for classification with discrete features (e.g., word counts for text classification). The multinomial distribution normally requires integer feature counts. However, in practice, fractional counts such as tf-idf may also work.

reading closely you realize that it implies that all the features should be positive.

This has a statistical basis indeed; from the Cross Validated thread Naive Bayes questions: continus data, negative data, and MultinomialNB in scikit-learn:

MultinomialNB assumes that features have multinomial distribution which is a generalization of the binomial distribution. Neither binomial nor multinomial distributions can contain negative values.

See also the (open) Github issue MultinomialNB fails when features have negative values (it is for a different library, not scikit-learn, but the underlying mathematical rationale is the same).

It is not actually difficult to demonstrate this; using the example available in the documentation:

A lot of this is the same, but not all.

This is because of your dependent variable. You chose make. Did you inspect this field? You have training and testing; where do you put an outcome with only one observation, like make = "mercury"? How can you train with that? How could you test for it if you didn't train for it?