wikientities | Linking Entities in CommonCrawl Dataset onto Wikipedia Concepts | Natural Language Processing library

wikientities is a Java library typically used in Artificial Intelligence, Natural Language Processing applications. wikientities has no vulnerabilities and it has low support. However wikientities has 1 bugs and it build file is not available. You can download it from GitHub.

This project is our entry to the [CommonCrawl contest](http://commoncrawl.org/first-ever-code-contest/). The idea is inspired by [Google’s release](http://googleresearch.blogspot.sg/2012/05/from-words-to-concepts-and-back.html) of the [entity linking dataset](http://www-nlp.stanford.edu/pubs/crosswikis-data.tar.bz2/), which provides baseline for research on entity linking and other information retrieval and natural language processing tasks. Human language is ambiguous, and synonymy and polysemy are fundamental problems in natural language processing (NLP) and information retrieval (IR). One of the approaches for Word Sense Disambiguation (WSD) is utilizing external ontologies, e.g. Wikipedia to determine the meaning of a word based on the probabilities that it can be mapped each of the possible Wikipedia concepts. Our entry aims to build such a corpus of anchortext-WikipediaConcept-Count triples from the CommonCrawl dataset, so as to benifit research on WSD, NLP and IR. More specifically, we extract all anchortexts (the text you click on in a webpage link) which point to a Wikipedia page, together with the corresponding Wikipedia page. Based on the corpus, we developed this web application to demonstrate the anchortext-WikipediaConcept-Count structure.