queued | Simple HTTP-based queue server | Runtime Evironment library

I'm working with Symfony 4.4 and Symfony Messenger

Messenger configuration includes a transport and routing:

I have a widget that performs a series of expensive computations (which are carried out on a compute core). The computations are queued in a list in the widget and the computation of each is waited for. The computations are queued in the build function so since before being sent to the compute core require some computations to be performed on the main isolate, I thought not to handle them directly in the build function but to just add them to the queue at build time.

This is the function I currently have to exhaust the queue:

We have micro service which consumes(subscribes)messages from 50+ RabbitMQ queues.

Producing message for this queue happens in two places

1. The application process when encounter short delayed execution business logic ( like send emails OR notify another service), the application directly sends the message to exchange ( which in turn it is sent to the queue ).

2. When we encounter long/delayed execution business logic We have messages table which has entries of messages which has to be executed after some time.

Now we have cron worker which runs every 10 mins which scans the messages table and pushes the messages to RabbitMQ.

Let's say the messages table has 10,000 messages which will be queued in next cron run,

1. 9.00 AM - Cron worker runs and it queues 10,000 messages to RabbitMQ queue.
2. We do have subscribers which are listening to the queue and start consuming the messages, but due to some issue in the system or 3rd party response time delay it takes each message to complete 1 Min.
3. 9.10 AM - Now cron worker once again runs next 10 Mins and see there are yet 9000+ messages yet to get completed and time is also crossed so once again it pushes 9000+ duplicates messages to Queue.

Note: The subscribers which consumes the messages are idempotent, so there is no issue in duplicate processing

I can have 4 status ( RequiresQueuing, Queued, Completed, Failed )

1. Whenever a message is inserted i can set the status to RequiresQueuing
2. Next when cron worker picks and pushes the messages successfully to Queue i can set it to Queued
3. When subscribers completes it mark the queue status as Completed / Failed.

There is an issue with above logic, let's say RabbitMQ somehow goes down OR in some use we have purge the queue for maintenance.

Now the messages which are marked as Queued is in wrong state, because they have to be once again identified and status needs to be changed manually.

Let say I have RabbitMQ Queue named ( events )

This events queue has 5 subscribers, each subscribers gets 1 message from the queue and post this event using REST API to another micro service ( event-aggregator ). Each API Call usually takes 50ms.

Use Case:

1. Due to high load the numbers events produced becomes 3x.
2. Also the micro service ( event-aggregator ) which accepts the event also became slow in processing, the response time increased from 50ms to 1 Min.
3. Cron workers follows your design mentioned above and queues the message for each min. Now the queue is becoming too large, but i cannot also increase the number of subscribers because the dependent micro service ( event-aggregator ) is also lagging.

Now the question is, If keep sending the messages to events queue, it is just bloating the queue.

https://www.rabbitmq.com/memory.html - While reading this page, i found out that rabbitmq won't even accept the connection if it reaches high watermark fraction (default is 40%). Of course this can be changed, but this requires manual intervention.

So if the queue length increases it affects the rabbitmq memory, that is reason i thought of throttling at producer level.

1. How can i throttle my cron worker to skip that particular run or somehow inspect the queue and identify it already being heavily loaded so don't push the messages ?
2. How can i handle the use cases i said above ? Is there design which solves my problem ? Is anyone faced the same issue ?

Thanks in advance.

Check the accepted answer Comments for the throttling using queueCount

In the Programming Language Examples Alike Cookbook's chapter on Sockets, the "Pre-Forking Servers" section uses a SIGCHLD handler like this:

Updated:

2~3days After, my job is failed automatically with below message

I have some trouble all github action jobs are queued and never executed.

I have checked Github action status on statusgithub.com

but canont find something down or trouble sign.

With many searching, I found this thread

It looks so old trouble. so stranger. on other repository, github action is working well.

yaml

I have a list of intensive updates so I am grouping them together and executing them as a batch job in a single thread. Other threads can send their updates at any time.

I have a job server running Bull and express.

Server requirements

Receive a request containing an object and use that object as an input to a local programme (no other choice) which takes several minutes to a couple hours to run. The jobs MUST be run one after the other in the order they are received (no way round this).

TL;DR Server:

Ok, I'm totally lost on deadlock issue. I just don't know how to solve this.

I have these three tables (I have removed not important columns):

I have a busy ASP.NET 5 Core app (thousands of requests per second) that uses SQL Server. Recently we decided to try to switch some hot code paths to async database access and... the app didn't even start. I get this error:

The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

And I see the number of threads in the thread pool growing to 40... 50... 100...

The code pattern we use is fairly simple: