boltdb | boltdb is redis like bolt operations | Caching library

The config looks good, similar as mine. I would troubleshoot it with following steps:

1. Exec to docker container and check if the rules file is not empty cat /etc/loki/rules/rules.yaml

2. Check the logs of loki. When rules are loaded properly logs like this will pop up:

If Loki is running in single tenant mode, the required ID is fake (yes we know this might seem alarming but it’s totally fine, no it can’t be changed).

To give a bit of background to anyone new to Loki, as stated in the documentation: Loki does not come with any included authentication layer. Operators are expected to run an authenticating reverse proxy in front of your services, such as NGINX using basic auth or an OAuth2 proxy.

This basically means that you'll have to place something in between the client(s) and Loki to enforce e.g. basic authentication. In this case there's a Ingress (Nginx) acting as a reverse proxy with basic authentication.

To troubleshoot problems with authentication using Nginx there's a number of things to check:

• Logs of the Nginx Ingress Pod, check for authentication errors.
• That the added Kubernetes Secret contains what you expect.
• That you have configured the Ingress object with the needed annotations.

When it comes to using Nginx as Ingress and adding basic authentication this resource from the official docs is really helpful.

In general when creating Kubernetes secrets, especially from the command line using kubectl you'll have to single quote the password if it contains special characters. This is so that the special characters wont be interpreted by your shell. More info here.

From bolt doc:

Get retrieves the value for a key in the bucket. Returns a nil value if the key does not exist or if the key is a nested bucket. The returned value is only valid for the life of the transaction.

I finally did work this out. It requires a compactor but gives no warning about it. Best practice is to create an AWS s3 bucket without any public access. Next create an IAM user with programmatic access only. Create an access policy which gives full access only to the bucket you created. Attach the policy to the user's permissions. You do not need to attach a policy to the bucket itself. Check if you have "/" in your URL that you escape it with %2F otherwise you will get an auth error. Note that this config is for loki v2.0.0 which was released yesterday.

Here are my complete working docker-compose and loki config files. I put them on an external network to enable prometheus monitoring.

here is my docker-compose.yaml

Before running. run by root and chmod -R 755 namefile

According to the test, I found that the problem must caused by the property mountOptions of the storage class. I got the same error as you when I use the storage class example that AKS provided here:

But Badger is not about writing "an" entry:

My writes are really slow. Why?

Are you creating a new transaction for every single key update? This will lead to very low throughput.

To get best write performance, batch up multiple writes inside a transaction using single DB.Update() call.
You could also have multiple such DB.Update() calls being made concurrently from multiple goroutines.

That leads to issue 396:

I was looking for fast storage in Go and so my first try was BoltDB. I need a lot of single-write transactions. Bolt was able to do about 240 rq/s.

I just tested Badger and I got a crazy 10k rq/s. I am just baffled

That is because:

LSM tree has an advantage compared to B+ tree when it comes to writes.
Also, values are stored separately in value log files so writes are much faster.

You can read more about the design here.

One of the main point (hard to replicate with simple read/write of files) is:

Key-Value separation

The major performance cost of LSM-trees is the compaction process. During compactions, multiple files are read into memory, sorted, and written back. Sorting is essential for efficient retrieval, for both key lookups and range iterations. With sorting, the key lookups would only require accessing at most one file per level (excluding level zero, where we’d need to check all the files). Iterations would result in sequential access to multiple files.

Each file is of fixed size, to enhance caching. Values tend to be larger than keys. When you store values along with the keys, the amount of data that needs to be compacted grows significantly.

In Badger, only a pointer to the value in the value log is stored alongside the key. Badger employs delta encoding for keys to reduce the effective size even further. Assuming 16 bytes per key and 16 bytes per value pointer, a single 64MB file can store two million key-value pairs.

The reason why the setup shown in my own question was not working, was the 'command' entry in my docker-compose.yml file: