sidekiq | Simple , efficient background processing for Ruby | Job Scheduling library

Let's get started by clarifying the difference between a Headless Service and a Service.

A Service allows one to connect to one Pod, while a headless Service returns the list of available IP addresses from all the available pods, allowing to auto-discover.

A better detailed explanation by Marco Luksa has been published on SO here:

Each connection to the service is forwarded to one randomly selected backing pod. But what if the client needs to connect to all of those pods? What if the backing pods themselves need to each connect to all the other backing pods. Connecting through the service clearly isn’t the way to do this. What is?

For a client to connect to all pods, it needs to figure out the the IP of each individual pod. One option is to have the client call the Kubernetes API server and get the list of pods and their IP addresses through an API call, but because you should always strive to keep your apps Kubernetes-agnostic, using the API server isn’t ideal

Luckily, Kubernetes allows clients to discover pod IPs through DNS lookups. Usually, when you perform a DNS lookup for a service, the DNS server returns a single IP — the service’s cluster IP. But if you tell Kubernetes you don’t need a cluster IP for your service (you do this by setting the clusterIP field to None in the service specification ), the DNS server will return the pod IPs instead of the single service IP. Instead of returning a single DNS A record, the DNS server will return multiple A records for the service, each pointing to the IP of an individual pod backing the service at that moment. Clients can therefore do a simple DNS A record lookup and get the IPs of all the pods that are part of the service. The client can then use that information to connect to one, many, or all of them.

Setting the clusterIP field in a service spec to None makes the service headless, as Kubernetes won’t assign it a cluster IP through which clients could connect to the pods backing it.

"Kubernetes in Action" by Marco Luksa

As the Redis documentation say:

When using the Sentinel support you need to specify a list of sentinels to connect to. The list does not need to enumerate all your Sentinel instances, but a few so that if one is down the client will try the next one. The client is able to remember the last Sentinel that was able to reply correctly and will use it for the next requests.

So the idea is to give what you have, and if you scale up the redis pods, then you don't need to re-configure Sidekiq (or Rails if you're using Redis for caching).

Now you just need a way to fetch the IP addresses from the headless service in Ruby, and configure Redis client sentinels.

Fortunately, since Ruby 2.5.0, the Resolv class is available and can do that for you.

Built-in character classes are more table-driven.
Given that, Negative built-in ones like \W, \S etc...
are difficult for engines to merge into a positive character class.

In this case, there are some obvious bugs because as you've said, it doesn't time out on
some target strings.

In fact, [a-xzA-XZ\W] works given the sample string. It times out when Y is included anywhere
but just for that particular string.

Let's see if we can determine if this is a bug or not.

First, some tests:

Test - Fail [a-zA-Z\W]

https://rextester.com/FHUQG84843

Here is the reason for that:

That’s from the Redis gem. 6.4.1 will be released Monday and fix it. Downgrade that gem or use Sidekiq’s main branch if you want an immediate fix.

https://github.com/mperham/sidekiq/issues/5178#issuecomment-1029545859

You need to tell sidekiq which queues to process. By default, some frameworks have their own queue names.

Create a sidekiq configuration file (e. g. config/sidekiq.yml) and define the queues that it should use

There could be two approach to this:

1. Call Sidekiq worker inside Mailer

You want to save the current shard using a client middleware and restore it in a server middleware. See the notes about ActiveSupport::CurrentAttributes and the Middleware wiki page.

There is no out-of-the-box solution, but there is a simpler way to determine if a job's been already queued (true/false):

You are probably not exposing redis port to the sidekiq service try exposing this in docker compose. This probably may work.

After rebooting my computer, it now works...