topic-modeling | Topic modeling utilities
kandi X-RAY | topic-modeling Summary
kandi X-RAY | topic-modeling Summary
This project wraps [MALLET] (a machine learning toolkit written in [Java] www.java.com/)) with some simplified interfaces and utilities written in [Scala] a programming language that runs on the Java Virtual Machine. It also uses the [Apache POI] library to export MALLET topic model data to Excel spreadsheets.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of topic-modeling
topic-modeling Key Features
topic-modeling Examples and Code Snippets
Community Discussions
Trending Discussions on topic-modeling
QUESTION
I'm currently trying to develop a code for a paper I have to write. I want to conduct a LDA-based topic modeling. I found some code deposits on GitHub and was able to combine them and slightly adapted them where necessary. Now I would like to add something that would name each identified topic after the word with the highest beta-value assigned to the respective topic. Any ideas? It's the first time I'm coding anything and my expertise is therefore quite limited.
Here's the section of the code where I wanted to insert the "naming part":
...ANSWER
Answered 2021-May-05 at 19:26You can make an additional column in your data that, after grouping by topic, takes the name of the term with the highest beta.
QUESTION
ANSWER
Answered 2020-May-30 at 21:38GSDMM (Gibbs Sampling Dirichlet Multinomial Mixture) is a short text clustering model. It is essentially a modified LDA (Latent Drichlet Allocation) which suppose that a document such as a tweet or any other text encompasses one topic.
Address: github.com/da03/GSDMM
QUESTION
I have 30 text files so far which all have multiple lines. I want to apply a LDA Model based on this tutorial . So, for me it should look this:
...ANSWER
Answered 2020-Jun-03 at 15:05Loop over the files, 1 to 31 (last is skipped by the range() function:
QUESTION
I created a Gensim LDA Model as shown in this tutorial: https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/
...ANSWER
Answered 2020-Feb-16 at 08:45Solved! Coherence Model requires the original text, instead of the training corpus fed to LDA_Model - so when i ran this:
QUESTION
I have a follow-up question to the one asked here: Mallet topic modeling - topic keys output parameter
I hope I can still get a more detailed explanation of this subject because I have trouble understanding these numbers in the output files.
What can the summation of the output numbers tell us? For example, with 20 topics and an optimization value 20 on 2000 iterations, the summation of the output is approximately 2. With the same corpus, but with 15 topics/1000 iterations/optimization 10 the result is 0,77 and with 10 topics/1000 iterations/optimization 10 it's 0,72. What does this mean? Does it even mean anything?
Also, these people are referring to these results as parameters, but for my understanding, the parameter is the optimization interval and not the result in the output. So what is the correct way to refer to the result in the output? Frequency of the topic? Is it a procentage of something? What part did I get wrong?
...ANSWER
Answered 2019-Dec-24 at 16:11You're correct that parameter is being used to mean two different things here.
Parameters of the statistical model are values that determine the properties of that model. In this case they determine which topics we expect to occur more often, and how confident we are of that. In some cases these are set by the user, in other cases they are set by the inference algorithm.
Parameters of the inference algorithm are settings that determine the procedure by which we set the parameters of the statistical model.
An additional confusion is that when model parameters are explicitly set by the user, Mallet uses the same interface as for algorithm settings.
The numbers you see are the parameters of a Dirichlet distribution that describes our prior expectation of the mix of topics in a document. You can think of it as having two parts: proportions and magnitude. If you rescale the numbers to add up to 1.0, the resulting proportions would tell you the model's guess at which topics occur most frequently. The actual sum of the numbers (the magnitude) tells you how confident the model is that this is the actual proportion you will see in a document. Smaller values indicate more variability.
A possible explanation for the numbers you're seeing (and please treat this as raw speculation) is that the 20 topic model has more flexibility to fit consistent topics, and so it is about three times more confident that there are topics that consistently occur more often in documents. As the number of topics decreases, the specificity of topics drops, so it is more likely that any particular topic could be large in any given document.
QUESTION
I would like to build an application to foreign languages learning, something based on creating decks with flashcards, words ordering, add voice (pronounciation) samples, add images etc. (something similar to ANKI app).
I want to use C# language and .NET platform to make it. I wish make both desktop and mobile app (maybe also website, but this is to be considered). And my question is what exactly technologies (type of projects) should I use to do it? It means I should make desktop app using .net framework with WPF and Xamarin to mobile app and then somehow synchronize it? I would like to synchronize desktop and mobile apps together - if someone uses desktop version and then install mobile app then he/she could have access to the whole his/her previous decks/settings and so on, maybe should I consider creating accounts? But I also wonder about the .net core and thanks to that I would make my app to be cross-platfrom.
I don't know how to correctly plan it ... Another thing is I would like to use some python libaries in that, exactly I mean gensim and generally topic-modeling, maybe also some other tools to neural networks. In general I would like to embed python things in my C# code.
Sumarizing, the aim is to create both desktop and mobile app using .net (.net framework or .net core), somehow makes the synchronization/connection between them and then (like a additional feature) use some python topic-modelling and neural network tools to enrich this whole app with some things. Would you recommend me something, some way to do it properly (some good approach to the topic), where should I start? Till now I was using mainly wpf/win forms, also entity framework, ado.net and so on. I have never used .net core but I think it's time to get to know that - it's future I suppose :)
...ANSWER
Answered 2019-Sep-14 at 21:50I think using Xamarin Forms is a good approach to target Android/iOS/Windows.
As for maintaining state (flash cards, etc) between multiple apps/platforms, you will need your users to create account. Dotnet core is an excellent choice for your api. You can decide to go with a monolith or micro-service arch with it.
As for using python, dotnet core allows you load and use python scripts from C# code. You should take a look at IronPython and how to host the interpreter in the dotnet core code.
All in all, the stack looks good, but you need to understand that more important than the stack is your overall architecture, how everything ties together.
Cheers.
QUESTION
I'm using the mallet topic-modeling tool and have some difficulties to make it stable (the topics that I get are not seemed very logic).
I worked with your tutorial and that one: https://programminghistorian.org/en/lessons/topic-modeling-and-mallet#getting-your-own-texts-into-mallet and I got some questions on that:
- Is there some best practices for get that model to work? Except the optimize command (what is a good number for that)? What is good number for iterations command?
- I import my data with the import dir command. In that dir there are my files. Is it matter if those files contain a text with new lines or just a very long line?
- I read about the hLDA model. When I tried to run it I saw that the only output is the state.txt output that is not very clear. I expect for an output like the topic-modeling model (topic_keys.txt, doc_topics.txt) how can I get them?
- When should I use the hLDA rather then the topic-modeling?
Thanks a lot for your help!
...ANSWER
Answered 2019-Apr-12 at 13:27Some references for good practices in topic modeling are The Care and Feeding of Topic Models with Jordan Boyd-Graber and Dave Newman and Applied Topic Modeling with Jordan Boyd-Graber and Yuening Hu.
For hyperparameter optimization --optimize-interval 20 --optimize-burn-in 50
should be fine, it doesn't seem to be very sensitive to specific values. Convergence for Gibbs sampling is hard to assess, the default 1000 iterations should be interpreted as "a number large enough that it's probably ok" rather than a specific value.
If you are reading individual documents from files in a directory, lines don't matter. If documents are longer than about 1000 tokens before stopword removal, consider breaking them into smaller segments.
hLDA is only included because people seem to want it, I don't recommend it for any purpose.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install topic-modeling
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page