bahdit | A scalable search engine based on Accumulo

bahdit is a JavaScript library. bahdit has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. You can download it from GitHub.

Bahdit is a search engine prototype developed at Booz Allen Hamilton during the summer of 2012 by a team of interns. The goal of Bahdit is to create a modular system of components to support online indexing of data in a scalable way that is not currently realized in other open source search solutions. Bahdit uses Accumulo, a distributed key-value database, to store the document indices. The initial version includes a map/reduce based web crawling system that uses map/reduce primarily for parallel job control and works partially outside of usual map/reduce paradigms. This may eventually be retooled to better conform to map/reduce paradigms and use something like Cascading for job control and coordination. Eventually a storm/message queue based restful indexing api is planned as well, which will mimic the Solr restful indexing api. The actual index design is a hybrid term-document partitioned index, which has become popular for many indexing systems based on Accumulo. Essentially, a dictionary with term frequencies is used to find the lowest frequency term in a user query. This is used to access a term partitioned index, which itself contains a document-partitioned index.