parsnip | A tidy unified interface to models

Here's a simple modelling workflow using the palmerpenguins dataset:

Requesting your help or expert opinion on a parallelization issue I am facing.

I regularly run an Xgboost classifier model on a rather large dataset (dim(train_data) = 357,401 x 281, dims after recipe prep() are 147,304 x 1159 ) for a multiclass prediction. In base R the model runs in just over 4 hours using registerDoParallel(using all 24 cores of my server). I am now trying to run it in the Tidymodels environment, however, I am yet to find a robust parallelization option to tune the grid.

I attempted the following parallelization options within tidymodels. All of them seem to work on a smaller subsample (eg 20% data), but options 1-4 fail when I run the entire dataset, mostly due to memory allocation issues.

1. makePSOCKcluster(), library(doParallel)
2. registerDoFuture(), library(doFuture)
3. doMC::registerDoMC()
4. plan(cluster, workers), doFuture, parallel
5. registerDoParallel(), library(doParallel)
6. future::plan(multisession), library(furrr)

Option 5 (doParallel) has worked with 100% data in the tidymodel environment, however, it takes 4-6 hours to tune the grid. I would request your attention to option 6 (future/ furrr), this appeared to be the most efficient of all methods I tried. This method however worked only once (successful code included below, please note I have incorporated a racing method and stopping grid into the tuning).

I am doing feature selection using LASSO regression with tidymodels and glmnet.

It is possible to force variables in glmnet by using the penalty.factors argument (see here and here, for example).

Is it possible to do the same using tidymodels ?

I'm unable to deploy a tidymodel with vetiver and get a prediction when the model includes a variable with role as ID in the recipe. See the following error in the image:

{ "error": "500 - Internal server error", "message": "Error: The following required columns are missing: 'Fake_ID'.\n" }

The code for the dummy example is below. Do I need to remove the ID-variable from both the model and recipe to make the Plumber API work?

I want to use purrr::map_* functions to extract info from multiple models involving linear regression method. I am first creating some random dataset. The dataset has three dependent variables, and one independent variable.

WHAT I WANT: I'm trying to fit a GAM model for classification using tidymodels on a given data.

SO FAR: I'm able to fit a logit model.

I'm new to tidymodels but apparently the step_pca() arguments such as nom_comp or threshold are not being implemented when being trained. as in example below, I'm still getting 4 component despite setting nom_comp = 2.

I have a monthly (Jan - Dec) data set for weather and crop yield. This data is collected for multiple years (2002 - 2019). My aim is to obtain bootstrapped slope coefficient of the affect of temperature in each month on yield gap. In bootstrapping, I want to block the year information in a way that the function should randomly sample data from a specific year in each bootstrap rather than choosing rows from mixed years.

I read some blogs and tried different methods but I am not confident about those. I tried to disect the bootstrapped splits to ensure if I am doing it correctly but I was not.

Here is the starting code:

I want to use xgboost for a classification problem, and two predictors (out of several) are binary columns that also happen to have some missing values. Before fitting a model with xgboost, I want to replace those missing values by imputing the mode in each binary column.

My problem is that I want to do this imputation as part of a tidymodels "recipe". That is, not using typical data wrangling procedures such as dplyr/tidyr/data.table, etc. Doing the imputation within a recipe should guard against "information leakage".

Although the recipes package provides many step_*() functions that are designed for data preprocessing, I could not find a way to do the desired imputation by mode on numeric binary columns. While there is a function called step_impute_mode(), it accepts only nominal variables (i.e., of class factor or character). But I need my binary columns to remain numeric so they could be passed to the xgboost engine.

Consider the following toy example. I took it from this reference page and changed the data a bit to reflect the problem.

create toy data