extraTrees | ExtraTrees method for Java and R | Machine Learning library

you can plugin the following code in your for loop, where you are evaluating each model.

You set name of legend using scale_color_discrete and scale_shape_discrete.

The issue is that when you draw a plot in base graphics draw directly on a device. The line of your code grob= plot(r_fit$unique.death.times, pred[1,], type = "l", col = "red") creates a NULL object (unlike ggplot which would return a plot object).

You can make the plot directly in ggplot (there are a few ways of doing this but I've done a simple example bolow) and convert it with ggplotly:

Without deeply exploring all the data you are using it is hard to say for certain what is causing the drop in accuracy (or AUC) when moving from your training set to the testing set. It is unlikely to be caused by the mixed discrete/continuous data.

The drop just suggests that your models are over-fitting to your training data (and therefore not transferring well). This could be caused by too many learned parameters (given the amount of data you have)--more often a problem with neural networks than with some of the other methods you mentioned. Or, the problem could be with the way the data was split into training/testing. If the distribution of the data has a significant difference (that's maybe not obvious) then you wouldn't expect the testing performance to be as good. If it were me, I'd look carefully at how the data was split into training/testing (assuming you have a reasonably large set of data). You may try repeating your experiments with a number of random training/testing splits (search k-fold cross validation if you're not familiar with it).

Without testing on your data, I think you're using the wrong map function. purrr::map works on one argument (one list, one vector, whatever) and returns a list. You are passing it two values (data3 and data2), so we need to use:

For the models that require scaling, you can build a pipeline, which then goes into the voting classifier. Example with scaled and unscaled support vector classifier:

StackingClassifier does not support multi label classification as of now. You could get to understand these functionalities by looking at the shape value for the fit parameters such as here.

Solution would be to put the OneVsRestClassifier wrapper on top of StackingClassifier rather on the individual models.

Example:

Any tree ensemble (i.e forest), that relies on various ways of injecting randomness to grow diverse and uncorrelated trees, can be called random forest. All variants of random forests is based on the same principle that the more diverse we can make the individual trees, the lower will be the resulting generalization error.

One such way of injecting randomness is called Bootstrap Aggregating (Bagging), which injects randomness in datasets sent to each tree**. Another is Random Subspace method, that basically randomly samples a subset of features at each tree node, to find the best (feature, value) split (instead of considering all features). Here the randomness lies in tree building process. ExtraTree is another example that introduces randomness in tree building phase, first by randomly selecting cut-point for each feature, then choosing the best (feature, value) split. An interesting variant intentionally introduces label noise independently in each base tree's dataset- I think you get the point.

However, for many, the term Random Forest actually means the most famous member of random forest family, the variant detailed in Breiman's famous paper. This basically uses both the Bagging and Random subspace method discussed above, and that's just it!

**Dataset randomization techniques, like bagging or that label noise one, can be used with any algorithm beside decision tree. So Bagging isn't exactly an example of Random Forest- it's more like a component of Random Forest.

It looks like your random forest has almost no predictive power on the second class "left". The best scores all have extremely high sensitivity and low specificity, which basically means that you classifier just classifies everything to class "stayed", which I imagine is the majority class. Unfortunately this is pretty bad, as it does not go too far from a naive classifier saying everything is from the first class.
Also, I can't quite understand if you only tried values for mtry 2,14 and 27, but in that case I would strongly suggest trying the whole 3-25 range (best values will most likely be somewhere in the middle).

Apart from that, since the performance looks to be rather bad (judging by the ROC) I suggest you work more on the feature engineering to extract some more information. Otherwise if you're OK with what you have or you think nothing more can be extracted, just tweak the probability threshold for the classification so that you have a sensitivity and specificity that mirror your requirement on the classes (you might care more about miscassifying "stayed" than "left" or vice versa, I dont know your problem).

Hope it helps!