microblob | Serve millions of JSON documents via HTTP

by miku Go Version: v0.2.13 License: GPL-3.0

X-Ray Key Features Code Snippets(3)Community Discussions Vulnerabilities Install Support

kandi X-RAY | microblob Summary

microblob is a Go library. microblob has no bugs, it has no vulnerabilities, it has a Strong Copyleft License and it has low support. You can download it from GitHub.

microblob is a simplistic key-value store, that serves JSON documents from a file over HTTP. It is implemented in a few hundred lines of code and does not contain many features. Warning: This server SHOULD NEVER BE EXPOSED PUBLICLY as it contains no security, rate-limiting or other safety measures whatsoever. microblob was written in 2017 as an ad-hoc solution to replace a previous setup using memcachedb (which was getting slow). The main goal has been to serve about 200M JSON documents from a "persistent key-value store" over HTTP and to support frequent, fast rebuilds; with limited disk space and potentially limited memory. Code lacks tests and I would write it differently today. However, it ran without issues and happily served up to 400 requests/s with limited resources and with average response times of around 1ms. This project has been developed for Project finc at Leipzig University Library. It supports fast rebuilds from scratch, as the preferred way to deploy this is for a build-once update-never use case. It scales up and down with memory and can serve hundred million documents and more. Inspiration: So what's wrong with 1975 programming? Idea: Instead of implementing complicated caching mechanisms, we hand over caching completely to the operating system and try to stay out of its way. Inserts are fast, since no data is actually moved. 150 million (1kB) documents can be serveable within an hour.

Support

Quality

Security

License

Reuse

Support

microblob has a low active ecosystem.

It has 48 star(s) with 4 fork(s). There are 3 watchers for this library.

It had no major release in the last 12 months.

There are 0 open issues and 1 have been closed. On average issues are closed in 469 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of microblob is v0.2.13

Quality

microblob has 0 bugs and 0 code smells.

Security

microblob has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

microblob code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

microblob is licensed under the GPL-3.0 License. This license is Strong Copyleft.

Strong Copyleft licenses enforce sharing, and you can use them when creating open source projects.

Reuse

microblob releases are available to install and integrate.

Installation instructions, examples and code snippets are available.

Top functions reviewed by kandi - BETA

kandi has reviewed microblob and discovered the below as its top functions. This is intended to give you an instant insight into microblob implemented functionality, and help decide if they suit your requirements.

Main entry point .
NewHandler returns an http . Handler that serves metrics .
AppendBatchSize appends a batch of data to a file fn
Get implements the DBBackend interface .
renderString returns the string representation of v .
WithLastResponseTime sets the last response time
IsAllZero returns true if p contains all zero bytes .
Initialize global variables
NewLineProcessorBatchSize returns a new LineProcessor .
NewLineProcessor returns a new LineProcessor .

Get all kandi verified functions for this library.

microblob Key Features

No Key Features are available at this moment for microblob.

microblob Examples and Code Snippets

Usage

Lines of Code : 26

License : Strong Copyleft (GPL-3.0)

Copy

Usage of microblob:
  -addr string
        address to serve (default "127.0.0.1:8820")
  -backend string
        backend to use: leveldb, debug (default "leveldb")
  -batch int
        number of lines in a batch (default 50000)
  -c string
        lo

microblob

Lines of Code : 7

License : Strong Copyleft (GPL-3.0)

Copy

$ cat file.ldj
{"id": "some-id-1", "name": "alice"}
{"id": "some-id-2", "name": "bob"}

$ microblob -key id file.ldj
INFO[0000] creating db fixtures/file.ldj.832a9151.db ...
INFO[0000] listening at http://127.0.0.1:8820 (fixtures/file.ldj.832a9151.db

Update via curl

Lines of Code : 2

License : Strong Copyleft (GPL-3.0)

Copy

$ curl -v --data-binary @- localhost:8820/update?key=id < <(gunzip -c fixtures/fake.ldj.gz)
...

Community Discussions

No Community Discussions are available at this moment for microblob.Refer to stack overflow page for discussions.

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install microblob

Debian and RPM packages: see releases.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: