Cynical | Native Client wrapper for ZenGarden | Reinforcement Learning library

option is too small, the line:

This approach may be a little functional.

You need to first replace the ], in every row to an empty string I have used this Regex to split it.

After that, I split the strings by whitespaces which are more than 2 characters.

Then finally I used .map to project over the splitted items to parse it back and make an array of array.

UPDATE: I didn't solve the problem in eclipse, so I downloaded Visual-Studio 2017 ("Express"/"Community" its called) and it works there. In the beginning it also didn't because it didn't find some DLLs, but allowing an option called "Microsoft Symbol Server" in its' debugger settings solved it and now it works.

So problem "bypassed" rather than "solved" I would say (So if anyone experiences the same problems as mine, doing what I did will allow you to work on c++).

Reading through Weka's StringToWordVector documentation, there seem to be a couple of implementation details different than yours. Here are the top two, based on how likely they are to be the reason for the performance difference you see, in my opinion:

• It seems that by default, the resulting vector is boolean (i.e. noting the existence of a word, rather than number of occurrences)
• If the class attribute is set before vectorizing the text, a separate dictionary is built for each class, then all dictionaries are merged.

While any of them (or other, more minor differences) could be the culprit, my bet is on the second point.

The built-in class allows setting and unsetting each of these options; you could try re-running the 80% version using StringToWordVector with the -C option to use number of occurences rather then a boolean value, and with -O, to use a single dictionary across both classes.

This should allow you to verify whether any of these is indeed the culprit.

EDIT: Regarding the first point, i.e. counting occurences vs. noting word existence (also called Bernoulli and multinomial models), there were several academic papers at the 90s which looked into the differences, e.g. here and here. While usually the multinomial model works better, there are also opposite cases, depending on corpus and classification problem.