vowpal_wabbit | Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techni | Machine Learning library

if initial_t is equal to zero (which is the setting by default), it seems that the learning rate will always be zero, for all timesteps and epochs. Is this right?

initial_t is set to zero by default. By default the initial learning rate will not use initial_t to calculate its value but will start off at its default value, which is 0.5.

Per the documentation, the flags adaptive, normalized, and invariant are on by default. If any of them is specified, the other flags are turned off. In the case that you turn on the invariant flag (so in the case that we are not using normalized or adaptive) the initial learning rate will be calculated using the initial_t and power_t values, and the default initial_t is set to one instead of zero.

If initial_t is explicitly set to zero combined with the invariant flag being set, then yes, the learning rate will also be zero.

Also, what would happen if both initial_t and power_t are set to zero? I tried initializing a VW with those settings and it didn't complain.

If the initial learning rate is calculated using initial_t and power_t and both are explicitly set to zero, c++ should evaluate powf(0,0) to 1 resulting in the learning rate set to its default value, which can be specified by --learning_rate

If you are running vowpalwabbit via the command line, you should be able to see what these values are set to:

Vowpal Wabbit is based on online learning (SGD-like updates, but there is also --bfgs if you really need batch optimization) and (machine learning) reductions. See some of the tutorials or papers to understand the idea of reductions. Many VW papers are also about Contextual Bandit, which is implemented as a reduction to a cost-sensitive one-against-all (OAA) classification (which is further reduced to regression). See a simple intro into reductions or a simple example how binary classification is reduced into regression.

As far as I know, VowpalWabbit does not support Decision trees nor ensembles, but see --boosting and --bootstrap. It does not support SVM, but see --loss_function hinge (hinge loss is one of the two key concepts of SVM) and --ksvm. It does not support NN, but --nn (and related options) provides a very limited support simulating a single hidden layer (feed-forward with tanh activation function), which can be added into the reduction stack.

I am interested know out how they are calculated to see how I can interpret the results obtained.

Unfortunately knowing the first does not lead to knowing the second.

Your question is concerned with contextual bandits, but it is important to note that interpreting model parameters is an issue that also occurs in supervised learning. Machine learning has made progress recently (i.e., my lifetime) largely by focusing concern on quality of predictions rather than meaningfulness of model parameters. In a blog post, Phoebe Wong outlines the issue while being entertaining.

The bottom line is that our models are not causal, so you simply cannot conclude because "the weight of feature X is for arm A is large means that if I were to intervene in the system and increase this feature value that I will get more reward for playing arm A".

We are currently working on tools for model inspection that leverage techniques such as permutation importance that will help you answer questions like "if I were to stop using a particular feature how would the frequency of playing each arm change for the trained policy". We're hoping that is helpful information.

Having said all that, let me try to answer your original question ...

In vowpawabbit there is an option --audit that prints the weights of the features.

If we have a vw contextual bandit model with four arms, how is this feature weight created?

The format is documented here. Assuming you are using --cb (not --cb_adf) then there are a fixed number of arms and so the offset field will increment over the arms. So for an example like

A Docker container always requires a process to be running in the foreground, otherwise the container will exit. No manner of options will change that.

The options -dit control Docker and how it sets up the process but it's purely down to the ENTRYPOINT and CMD in the Dockerfile (or overrides on the command line) that control if the image stays running.

• -d Detaches your screen from the container and allows it to run in the background. This doesn't keep a process in the container running though.
• -i Keeps standard input open, sometimes required to run processes that expect someone to run it and use a keyboard and the like.
• -t Assigns a pseudo tty to the process, like your terminal.

docker run -dit ubuntu is a trick often used on images that run interactive shells like bash to keep them running in the background so you can attach or exec things in them.

From the image description it looks like you need to supply the options --daemon --foreground when running the image to keep the process running in the foreground.

The problem is the use of autogen.sh.

autogen.sh is provided only as a last-resort for possibly unfamiliar/unsupported environments where the provided Makefile (and child directory */Makefiles) may not work.

Among other things,autogen.sh calls GNU automake which overwrites the Makefiles (based on probing the environment).

To generate the JNI, you need to run make java with the original Makefiles that come with the source.

If you already overwrote the Makefiles, no worries! You can easily restore the originals back by typing:

To get a prediction resembling "probabilities" you could use --loss_function logistic --link logistic. Be aware that in this single-label setting your probabilities risk tending to 1.0 quickly (i.e. become meaningless).

• Working with a single label is problematic in the sense that there's no separation of the goal. Eventually the learner will peg all predictions to 1.0. To counter that - it is recommended to use --noconstant, use strong regularization, decrease the learning rate, avoid multiple passes, etc. (IOW: anything that avoids over-fitting to the single label)
• Even better: add examples where the user hasn't bought/clicked, they should be plentiful, this will make your model much more robust and meaningful.
• There's a better implementation of matrix factorization in vw (much faster and lighter on IO for big models). Check the --lrq option and the full demo under demo/movielens in the source tree.
• You should pass the training-set directly to vw to avoid Useless use of cat

all feature labels must be followed by a float

Yes, but if no colon and float is provided, the default feature value is 1.0.

But when I enter -1 3 |context day:Monday in this validator, it accepts The validator is just approximate and not kept update for several years. I am not aware of any VW base learner that would allow non-float feature values.

A solution to your problem is to escape spaces in your categorical feature values with spaces and convert a categorical feature with N values into N binary features (in the end it's the same). For example: -1 3 |context day_Monday name_A_B