ReMorse | Computer Graphics and Visualization Mini-Project | Data Visualization library

I have a large dataframe consisting of tweets, and keyword dictionaries loaded as values that have words associated with morality (kw_Moral) and emotion (kw_Emo). In the past I have used the keyword dictionaries to subset a dataframe to get only the tweets that have one or more of the keywords present.

For example, to create a subset with only those tweets that have emotional keywords, I loaded in my keyword dictionary...

I am trying to setup a script to send email, and I want to add the Senders Name (depending on who runs the script) to the Email Body.

Emailing works, and I'm setup on Juypter, runing Python 3, emailing from Outlook.

My code is below. The mail.SenderName is the part I want to pull the name from the Outlook account of the user. In VBA this would be equivalent to using Application.UserName

I have a large dataframe consisting of tweets, and a keyword dictionary loaded as a list that has words and word stems associated with emotion (kw_Emo). I need to find a way to count how many times any given word/word stem from kw_Emo is present each tweet. In kw_Emo, word stems are marked with an asterisk ( * ). For example, one word stem is ador*, meaning that I need to account for the presence of adorable, adore, adoring, or any pattern of letters that starts with ador….

From a previous Stack Overflow discussion (see previous question on my profile), I was greatly helped with the following solution, but it only counts exact character matches (Ex. only ador, not adorable):

1. Load relevant package.

   library(stringr)

2. Identify and remove the * from word stems in kw_Emo.

   for (x in 1:length(kw_Emo)) { if (grepl("[*]", kw_Emo[x]) == TRUE) { kw_Emo[x] <- substr(kw_Emo[x],1,nchar(kw_Emo[x])-1) } }

3. Create new columns, one for each word/word stem from kw_Emo, with default value 0.

   for (x in 1:length(keywords)) { dataframe[, keywords[x]] <- 0}

4. Split each Tweet to a vector of words, see if the keyword is equal to any, add +1 to the appropriate word/word stems' column.

   for (x in 1:nrow(dataframe)) { partials <- data.frame(str_split(dataframe[x,2], " "), stringsAsFactors=FALSE) partials <- partials[partials[] != ""] for(y in 1:length(partials)) { for (z in 1:length(keywords)) { if (keywords[z] == partials[y]) { dataframe[x, keywords[z]] <- dataframe[x, keywords[z]] + 1 } } } }

Is there a way to alter this solution to account for word stems? I'm wondering if it's possible to first use a stringr pattern to replace occurrences of a word stem with the exact characters, and then use this exact match solution. For instance, something like stringr::str_replace_all(x, "ador[a-z]+", "ador"). But I'm unsure how to do this with my large dictionary and numerous word stems. Maybe the loop removing [*], which essentially identifies all word stems, can be adapted somehow?

Here is a reproducible sample of my dataframe, called TestTweets with the text to be analysed in a column called clean_text:

dput(droplevels(head(TestTweets, 20)))

I'm trying to build a naive bayes based classifier for 1000 positive+negative labled IMDB reviews (txt_sentoken) and weka API for Java.

As I wasn't aware of StringToWordVector, which basically provides a BagOfWords model that reaches an 80% accuracy, so I did the vocabulary building and vector creation myself, with an accuracy of only 75% :(

Now I'm wondering why my solution is performing so much worse.

1) From my 2000 reviews, I build the BagOfWords:

I'm sure this has to be documented somewhere, but I'm so new at html that I can't figure out where to look, so feel free to just direct me to the correct reading if it's something obvious. As a note, I code in several languages, but mostly things like python or LaTeX, nothing like html. I have included my attempt at the bottom, but tried to include what I could in terms of my design setup and what problems I am running into. Any solutions to any of the listed problems would be appreciated :)

The setup;

I need to form gigantic table of information that will be populated based on a JSON string given on another webpage. I am currently using javascript to parse the JSON string and form a single global object of my own that has all the relevant information (the JSON string that the user will plug in is huge and I only need a relatively small portion of it). The created object has the following form;