throttler | Throttler fills the gap between sync | Genomics library

This is because you are creating the lock before the asyncio.run runs a loop, try creating the throttler in main() and sending it to f().

As I told you on Discord, with @nestjs/throttler, this functionality currently doesn't exist. You can have one or the other, or you can override the global config to be more specific for one endpoint, but there's not currently a way to have two limits set up.

All requests are triggered simultaneously.

Let's take a look here

The type (IAsyncEnumerator, bool) is a shorthand of the ValueTuple, bool> type, which is a value type. This means that on assignement it's not passed by reference, and instead it's copied. So this lambda does not work as intended:

This is a classic case of deferred execution. Every time you invoke an evaluating method on a non-materialized IEnumerable<>, it does the work to materialize the IEnumerable. In this case that's re-invoking your selector and creating new instances of the tasks that await the GetAsyncEnumerator calls.

With the call to .ToList() you materialize the IEnumerable. Without it, materialization occurs with with every call to .Any(), the call to ForAllAsync(), and at your foreach loop.

The same behavior can be reproduced minimally like this:

Take a look on this other post:

nest Command not found

It seems you need to install and use npm as:

@nestjs/cli instead of just nest

As a workaround, you can make TimePoint a subclass (with a noexcept default constructor) instead of a typedef.

You can do that easily with an ActionBlock Class from Dataflow (Task Parallel Library):

If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?

What you are looking for is to limit the MaximumConcurrencyLevel of what's called the Task Scheduler. You can create your own task scheduler that regulates the MaximumCongruencyLevel of the tasks it manages. I would recommend implementing a queue-like object that tracks incoming requests and currently working requests and waits for the current requests to finish before consuming more. The below information may still be relevant.

The task scheduler is in charge of how Tasks are prioritized, and in charge of tracking the tasks and ensuring that their work is completed, at least eventually.

The way it does this is actually very similar to what you mentioned, in general the way the Task Scheduler handles tasks is in a FIFO (First in first out) model very similar to how a ConcurrentQueue works (at least starting in .NET 4).

Would the framework automatically prevent too many tasks from being executed?

By default the TaskScheduler that is created with most applications appears to default to a MaximumConcurrencyLevel of int.MaxValue. So theoretically yes.

The fact that there practically is no limit to the amount of tasks(at least with the default TaskScheduler) might not be that big of a deal for your case scenario.

Tasks are separated into two types, at least when it comes to how they are assigned to the available thread pools. They're separated into Local and Global queues.

Without going too far into detail, the way it works is if a task creates other tasks, those new tasks are part of the parent tasks queue (a local queue). Tasks spawned by a parent task are limited to the parent's thread pool.(Unless the task scheduler takes it upon itself to move queues around)

If a task isn't created by another task, it's a top-level task and is placed into the Global Queue. These would normally be assigned their own thread(if available) and if one isn't available it's treated in a FIFO model, as mentioned above, until it's work can be completed.

This is important because although you can limit the amount of concurrency that happens with the TaskScheduler, it may not necessarily be important - if for say you have a top-level task that's marked as long running and is in-charge of processing your incoming requests. This would be helpful since all the tasks spawned by this top-level task will be part of that task's local queue and therefor won't spam all your available threads in your thread pool.