graceful-shutdown | Rust library for graceful shutdown | Reactive Programming library

I have an app running in kubernetes, on a couple of pods. I'm trying to improve our deployment experience (we're using rolling deployment), which is currently causing pains.

• each pod first goes not ready, so it gets no more traffic
• then it will finish the requests it's processing currently
• then it can be removed

This should all be possible and just work - you create a deployment that contains readiness and liveness probes. The load balancer will pick these up and route traffic accordingly. However, when I test my deployment, I see pods getting requests even when switching to not ready. Specifically, it looks like the load balancer won't update when a lot of traffic comes in. I can see pods going "not ready" when I signal them - and if they don't get traffic when they switch state, they will not receive traffic afterwards. But if they're getting traffic while switching, the load balancer just ignores the state change.

I'm starting to wonder how to handle this, because I can't see what I'm missing - it must be possible to host a high traffic app on kubernetes with pods going "not ready" without losing tons of requests.

Deployment

I'm trying to use the new options to do graceful shutdown with spring introduced in version 2.3, but I'm struggling to make my scheduled task to behave the same way.

As I need a valid user in the context during scheduled tasks execution, I am using DelegatingSecurityContextScheduledExecutorService to achieve this goal.

Here is a sample of my implementation of SchedulingConfigurer:

I'm completely new with both .NET Core and developing linux daemons. I've been through a couple of similar questions like Killing gracefully a .NET Core daemon running on Linux or Graceful shutdown with Generic Host in .NET Core 2.1 but they didn't solve my problem.

I've built a very simple console application as a test using a hosted service. I want it to run as a daemon but I'm having problems to correctly shut it down. When it runs from the console both in Windows and Linux, everything works fine.

I run npm outdated command and the output I get does not show the current version. This only occurs for this specific project, other projects return the output just fine.

Output example:

I am trying to outhouse central beans of my OSGi bundles into a central bundle, which provides them as a service. This works fine with the ErrorHanlders and Processors, but not with the ShutdownStrategy and RedeliveryPolicy. The Error Message I receive is

Trying to understand how throttling work is two consumers lead to a direct consumer that does some work and then sends a transformed message onward.

I can specify throttle on each consumer, but if the intent is not to overwhelm the destination, can I apply throttle to the direct route?

More importantly, would it act just as if it is throttling the 2 consumers, or would it consume and potentially create a "build up" of messages between the initial routes and the direct route?

Maybe, instead of direct it has to be seda?

Follow-up question: Can the throttled messages be flushed out if a graceful-shutdown begins?

What I found out so far:

• A "docker stop" sends a SIGTERM to process ID 1 in the container.
• The process ID 1 in the container is the java process running tomcat.*)
• Yes, tomcat itself shuts down gracefully, but not do so the servlets.
• Servlets get killed after 2 seconds, even if they are in the middle of processing a reguest(!!)

*) Side note: Although our container entrypoint is [ "/opt/tomcat/bin/catalina.sh", "run" ], but in catalina.sh the java process is started via the bash buildin "exec" command, and therefore the java process replaces the shell process and hereby becomes the new process id 1. (I can verify this by exec into the running container and do a "ps aux" in there.) Btw, I am using tomcat 7.0.88.

I found statements about tomcat doing gracefull shutdown by default (http://tomcat.10.x6.nabble.com/Graceful-Shutdown-td5020523.html - "any in-progress connections will complete"), but all I can see is that the SIGTERM which is sent from docker to the java process results in hardly stopping the ongoing execution of a request.

I wrote a little rest servlet to test this behaviour:

I'm trying to write a Flask app that behaves correctly when running in Kubernetes, especially when it comes to graceful shutdown.

As such, I need to have the code:

• recieve the shutdown signal
• start the "shutdown timer"
• continue serving requests as normal until the time is "up"
• then shut itself down

So far, what I've got is this: