estimators | Estimators to perform off-policy evaluation | Reinforcement Learning library

TL;DR

No wonder you have not actually seen anyone doing it - it is an absurd and bad idea.

You are trying to build an ensemble (Adaboost) which in itself consists of ensemble base classifiers (RFs) - essentially an "ensemble-squared"; so, no wonder about the high computation time.

But even if it was practical, there are good theoretical reasons not to do it; quoting from my own answer in Execution time of AdaBoost with SVM base classifier:

Adaboost (and similar ensemble methods) were conceived using decision trees as base classifiers (more specifically, decision stumps, i.e. DTs with a depth of only 1); there is good reason why still today, if you don't specify explicitly the base_classifier argument, it assumes a value of DecisionTreeClassifier(max_depth=1). DTs are suitable for such ensembling because they are essentially unstable classifiers, which is not the case with SVMs, hence the latter are not expected to offer much when used as base classifiers.

On top of this, SVMs are computationally much more expensive than decision trees (let alone decision stumps), which is the reason for the long processing times you have observed.

The argument holds for RFs, too - they are not unstable classifiers, hence there is not any reason to actually expect performance improvements when using them as base classifiers for boosting algorithms, like Adaboost.

I do not know of scikit-learn estimators or transformers that let you flip positive and negative class identifiers as a parameter. But I can think of two ways to work around this:

Method 1: You transform the array labels yourself before fitting the estimator

That can be easily achieved for numpy arrays:

The error itself is due to a typo in your method declaration. You implemented a function called tranform (note the missing 's') in your custom transformer class. That is why the interpreter is complaining that your custom transformer has not implemented transform.

While this will be a simple fix, you should also be aware that you have not adjusted your custom function to be used in the class you defined. For example:

• the variable df should be renamed to X
• weight and minimum are now object attributes and need to be referenced to as self.weight and self.minimum
• the variable column is undeclared

You will need to fix these issues as well. In regard to this, be aware that ColumnTransformer will only pass the subset of columns to the transformer that is meant to be transformed by this particular transformer. That means if you only pass the columns VehAge and DrivAge to dynamic_bin it cannot access the column Exposure.

I must agree with the commenters that just using the normal subclassing syntax would be best, but I still want to provide an example using decorators. to avoid the issues you raised, why not just do what a normal decorator does and replace the function with something new (normally a new function wrapping the original, but we can make that a class!)

The code posted is the following:

Generally to debug, you should check random_model_search.cv_results_ to find out which hyperparameter combinations lead to nan scores, and whether they occur in all the folds for a given hyperparameter combination.

In this case, I strongly suspect the issue is that max_features=0 is a possibility, and the model will fail to train in that case.

I tried something similar to your code with a few different sklearn versions. As it turns out, version 0.24.1 does not print out the scores when verbose=3.

Here's my code and output with sklearn version 0.22.2.post1:

The default values of the baseline model is different from the ones given in the grid search. for example The default value of n_estimators is 100. Take a look here