word2vec-tutorial | 中文詞向量訓練教學 | Topic Modeling library

If you supply a corpus of sentences to the class-instantiation, as your code has done, you don't need to call train(). It will already have done that automatically, and your second train() is redundant. (I recommend doing all such operations with logging enabled at the INFO level, and review the lgos after each run to understand what is happening – things like two full start-to-finish trainings should stick out in the logs.)

The case where you would call train() explicitly is if you want more control over the interim steps. You leave the sentences out of the class-instantiation, but then it is required for you to perform two explicit steps: both one call to build_vocab() (for initial vocabulary scan) and then one call to train() (for actual multi-epoch training).

In that case, you can use gensim's native .save() to save the model after the vocabulary-discovery, to have a model that's ready for re-training and doesn't need to report that step.

So, you could re-load that vocabulary-built model multiple times, to different variables, to train in different ways. For some of the model's meta-parameters – like window or even dm mode – you can even tamper directly with their values on a model after vocabulary-building to try different variants.

However, if there are any changes to the corpus's words/word-frequencies, or to other parameters that affect the initialization that happens during build_vocab() (like vector size), then the initialization will be out of sync with the configuration you're trying, and you could get strange errors.

In such a case, the best course is to repeat the build_vocab() step entirely. (You could also look into the source code to see the individual steps performed by build_vocab(), and just patch/repeat the initialization steps that are needed, but that requires strong familiarity with the code.)

The doc-word to word-word transform turns out to be more complex (for me at least) than I'd originally supposed. np.dot() is a key to its solution, but I need to apply a mask first. I've created a more complex example for testing...

Imagine a doc-word matrix

FYI that demo code was baed on gensim 0.12.3 (from 2015, as listed in its requirements.txt), and would need updating to work with the latest gensim.

It might be sufficient to add a line to w2v_server.py at line 70 (just after the load_word2vec_format()), to force the creation of the needed syn0norm property (which in older gensims was auto-created on load), before deleting the raw syn0 values. Specifically: