gini | Calculate the Gini coefficient of a numpy array | Data Manipulation library

I have some data that I am trying to apply a function over. It goes to a URL, collects the JSON data and then stores it into a folder on my computer.

I apply the following code:

I have some data that I would like to plot, visualizing in a normalized chart. Dataset:

I'm trying to adapt some code for positive unlabeled learning from this example, which runs with my data but I want to also calculate the ROC AUC score which I'm getting stuck on.

My data is divided into positive samples (data_P) and unlabeled samples (data_U), each with only 2 features/columns of data such as:

I am trying to increase the performance of a RandomForestClassifier that categorises negative and positive reviews using GridSearchCV but it seems that the accuracy is always around 10% lower than the base algorithm. Why is this? Please find my code below:

Base algorithm with 90% accuracy:

I'm trying to build an ensemble of some models using VotingClassifier() from Sklearn to see if it works better than the individual models. I'm trying it in 2 different ways.

1. I'm trying to do it with individual Random Forest, Gradient Boosting, and XGBoost models.
2. I'm trying to build it using an ensemble of many Random Forest models (using different parameters for n_estimators and max_depth.

In the first condition, I'm doing this

I have been constructing my own Extra Trees (XT) classifier in Rust for binary classification. To verify correctness of my classifier, I have been comparing it against Sklearns implementation of XT, but I constantly get different results. I thought that there must be a bug in my code at first, but now I realize it's not a bug, but instead a different method of calculating votes amongst the different trees in the ensemble. In my code, each tree votes based on the most frequent classification in a leafs' subset of data. So for example, if we are traversing a tree, and find ourselves at a leaf node that has 40 classifications of 0, and 60 classifications of 1, the tree classifies the data as 1.

Looking at Sklearn's documentation for XT (As seen here), I read the following line in regards to the predict method

The predicted class of an input sample is a vote by the trees in the forest, weighted by their probability estimates. That is, the predicted class is the one with highest mean probability estimate across the trees.

While this gives me some idea about how individual trees vote, I still have more questions. Perhaps an exact mathematical expression of how these weights are calculated would help, but I have yet to find one in the documentation.

I will provide more details in the upcoming paragraphs, but I wish to ask my question concisely here. How are these weights calculated at a high level, what are the mathematics behind it? Is there a way to change how individual XT trees calculate their votes?

---------------------------------------- Additional Details -----------------------------------------------

For my current tests, this is how I build my classifier

I'm trying to get the optimized parameters using GridSearchCV but I get the erorr:

I was left with a little question by the end of a video I've watched about regression tree algorithm: when some feature of the dataset has the threshold with the lower value for the sum of squared residuals, than it is used to split the node (if the number of observations in the node is greater than some predefined value). But can this same feature be used once again to split a node of this branch of the tree? Or the following splits of this branch have to be splitted by thresholds defined by other features (even if the feature that has already splitted other node has a threshold with the lower value for the sum of squared residuals of the observations of this node)?

Furthermore, I've got the same doubt when studying decision tree classifier: if a feature that has already been used in this branch can split the observations of some node with the lower value for gini's impurity compared to the splits that other features could make, than is this "already used" feature allowed to perform the split or not?

Thanks in advance for the attention!

So I have ran the following random forest grid search using balanced_accuracy as my scoring: