category_encoders | A library of sklearn compatible categorical variable | Machine Learning library

You just do the transform() on the test data (and do not fit the encoder again). The values that don't occur in "training" datasets would be encoded as 0 in all of the categories (as long as you won't change the handle_unknown parameter). For example:

The error comes from this line in the inverse_transform source code:

As stated in the documentation: you can get return the encoded column/feature names by using get_feature_names() and then drop the original feature names.

Also, do you need to encode your target features (y)?

In the below example, I assumed that you only need to encode features that corresponded to the 'object' datatype in your X_train dataset.

Finally, it is good practice to first split your dataset into train and test and then do fit_transform on the training set and only fit on the test set. In this way you prevent leakage.

After a lot of Google search, I found out that there is already an in-built method. Try this:

From the source, you can see that an OrdinalEncoder (the category_encoder version, not sklearn) is used to convert from categories to integers before doing the WoE-encoding. That object is available through the attribute ordinal_encoder. And those themselves have an attribute mapping (or category_mapping) that is a dictionary with the appropriate mapping.

The format of those mapping attributes isn't particularly pleasant, but here's a stab at "composing" the two for a given feature:

xgboost library is currently under updating to fix this bug, so the current solution is to downgrade the library to older versions, for me I have solved this problem by downgrading to xgboost v0.90

Try to check your xgboost version by cmd:

By default OneHotEncoder returns numpy array of dtype=float64. So one could simply cast the input-data X when being fed into forward() of the model:

The way to use multiple columns as input in scikit-learn is by using the ColumnTransformer.

Here is an example on how to use it with heterogeneous data.

Is that the way to use the library in order to encode high categorical values?

Yes. There is nothing wrong with your implementation.

You can think about the hashing trick as a "reduced size one-hot encoding with a small risk of collision, that you won't need to use if you can tolerate the original feature dimension".

This idea was first introduced by Kilian Weinberger. You can find in their paper the whole analysis of the algorithm theoretically and practically/empirically.

Why are some values negative?

To avoid collision, a signed hash function is used. That is, the strings are hashed by using the usual hash function first (e.g. a string is converted to its corresponding numerical value by summing ASCII value of each char, then modulo n_feature to get an index in (0, n_features]). Then another single-bit output hash function is used. The latter produces +1 or -1 by definition, where it's added to the index resulted from the first hashing function.

Pseudo code (it looks like Python, though):