text2vec | Easily generate document/paragraph/sentence vectors | Natural Language Processing library

Here's the situation, one whose solution seemed to be simple at first, but that has turned out to be more complicated than I expected.

I have an R data frame with three columns: an ID, a column with texts (reviews), and one with numeric values which I want to predict based on the text.

I have already done some preprocessing on the text column, so it is free of punctuation, in lower case, and ready to be tokenized and turned into a matrix so I can train a model on it. The problem is I can't figure out how to remove the stop words from that text.

Here's what I am trying to do with the text2vec package. I was planning on doing the stop-word removal before this chunk at first. But anywhere will do.

I am using {text2vec} word embeddings to build a dictionary of similar terms pertaining to a certain semantic category.

Is it OK to compound some tokens in the corpus, but not all? For example, I want to calculate terms similar to “future generation” or “rising generation”, but these collocations occur as separate terms in the original corpus of course. I am wondering if it is bad practice to gsub "rising generation" --> "rising_generation", without compounding all other terms that occur frequently together such as “climate change.”

Thanks!

I am trying to run through text2vec's example on this page. However, whenever I try to see what the vocab_vectorizer function returned, it's just an output of the function itself. In all my years of R coding, I've never seen this before, but it also feels funky enough to extend beyond just this function. Any pointers?

I am trying to implement one of the solutions to the question about How to align two GloVe models in text2vec?. I don't understand what are the proper values for input at GlobalVectors$new(..., init = list(w_i, w_j). How do I ensure the values for w_i and w_j are correct?

Here's a minimal reproducible example. First, prepare some corpora to compare, taken from the quanteda tutorial. I am using dfm_match(all_words) to try and ensure all words are present in each set, but this doesn't seem to have the desired effect.

Here's what I am doing:

1. Loading sparse matrix from a file.
2. Extracting indices(col, row) which have the values in this sparse matrix.
3. Use these indices and the values for further computation.

This works fine when I am executing the steps on R command prompt. But when its done inside a function of a package, step 2 throws the following error:

I'm building docker containers with R, with lines like:

So here is my code

The code below is the one I am using for plotting an LDA plot using text2vec inside topic_model function in a shiny app. input$date is a checkboxGroupInput selection, input$data works perfectly fine for a DT::renderDataTable output & topic_model runs well outside the app. Here I found how to get an LDA plot in a shiny app, but I didn't really get it so copied as it was. input$go is a simple actionButton.

Is there any alternative for rtexttools or another package for this kind of classification methodology, because these package were erased, also maxent and glmnet and they depended on rtexttools and vice verse; here is the script that im trying to apply and classify