Lasso | implementation Lasso with Coordinate Descent | Machine Learning library

The first issue is actually independent from the ColumnTransformer usage and it is due to a bug in method transform's implementation in your HealthyAttributeAdder class.

In order to get a consistent result you should modify line

When you set Lasso(..normalize=True) the normalization is different from that in StandardScaler(). It divides by the l2-norm instead of the standard deviation. If you read the help page:

normalize bool, default=False This parameter is ignored when fit_intercept is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use StandardScaler before calling fit on an estimator with normalize=False.

Deprecated since version 1.0: normalize was deprecated in version 1.0 and will be removed in 1.2.

It is also touched upon in this post. Since it will be deprecated, I think it's better to just use the StandardScaler normalization. You can see it's reproducible as long as you scale it in the same way:

You end up with the error with precision because some of your penalization is too strong for this model, if you check the results, you get 0 for f1 score when C = 0.001 and C = 0.01

Though very late, I'll try to give my answer. According to the doc, here's the considered primal optimization problem for LinearSVC: ,phi being the Identity matrix, given that LinearSVC only solves linear problems.

Effectively, this is just one of the possible problems that LinearSVC admits (it is the L2-regularized, L1-loss in the terms of the LIBLINEAR paper) and not the default one (which is the L2-regularized, L2-loss). The LIBLINEAR paper gives a more general formulation for what concerns what's referred to as loss in Chapter 2, then it further elaborates also on what's referred to as penalty within the Appendix (A2+A4).

Basically, it states that LIBLINEAR is meant to solve the following unconstrained optimization pb with different loss functions xi(w;x,y) (which are hinge and squared_hinge); the default setting of the model in LIBLINEAR does not consider the bias term, that's why you won't see any reference to b from now on (there are many posts on SO on this).

• , hinge or L1-loss
• , squared_hinge or L2-loss.

For what concerns the penalty, basically this represents the norm of the vector w used. The appendix elaborates on the different problems:

• L2-regularized, L1-loss (penalty='l2', loss='hinge'):
• L2-regularized, L2-loss (penalty='l2', loss='squared_hinge'), default in LinearSVC:
• L1-regularized, L2-loss (penalty='l1', loss='squared_hinge'):

Instead, as stated within the documentation, LinearSVC does not support the combination of penalty='l1' and loss='hinge'. As far as I see the paper does not specify why, but I found a possible answer here (within the answer by Arun Iyer).

Eventually, effectively the combination of penalty='l2', loss='hinge', dual=False is not supported as specified in here (it is just not implemented in LIBLINEAR) or here; not sure whether that's the case, but within the LIBLINEAR paper from Appendix B onwards it is specified the optimization pb that's solved (which in the case of L2-regularized, L1-loss seems to be the dual).

For a theoretical discussion on SVC pbs in general, I found that chapter really useful; it shows how the minimization of the norm of w relates to the idea of the maximum-margin.

I've been messing with this, and looking at some other NODE implementations (this one in particular) and have adjusted my cost function so that it is:

This happens just because RMSE and R-squared are meaningless for factor outcomes. You have to use caret::confusionMatrix or convert factor to integer (not a so good option in my opinion):

Use DataFrame.stack with convert first level to column by reset_index, so possible join values in GroupBy.agg, for unique values in order is used dict.fromkeys trick:

You can try the following:

The output of make_scorer (and the expected form of a scoring method for a grid search) is a callable with signature estimator, X, y; you should skip make_scorer and define such a callable directly. Then you can use the estimator's fitted attribute coefs_ directly. (The greater_is_better=False option of make_scorer just negates the score, so you should probably define this alternate custom scorer as negative BIC.)

Note however that in a GridSearchCV, you'll always be computing the score on the test folds, which deviates from the intention behind BIC.