taskscheduleR | Schedule R scripts/processes with the Windows | Job Scheduling library

The definition is deliberately states as -1 means that the number of number of concurrent operations will not be artificially limited. and it doesn't say that all actions will start immediately.

The thread pool manager normally keeps the number of available threads at the number of cores (or logical processor which are 2x number of cores) and this is considered the optimum number of threads (I think this number is [number of cores/logical processor + 1]) . This means that when you start executing your actions the number of available threads to immediately start work is this number.

Thread pool manager runs periodically (twice a second) and a if none of the threads have completed a new one is added (or removed in the reverse situation when there are too many threads).

A good experiment to see this in action is too run your experiment twice in quick succession. In the first instance the number of concurrent jobs at the beginning should be around number of cores/logical processor + 1 and in 2nd run it should be the number of jobs run (because these threads were created to service the first run:

Here's a modified version of your code:

Sorry it took so long. Was under a gun to finish a project..

The root cause of the problem turned out to be ICSharpCode.CodeConverter v.8.4.1.0!
Wow, of all the pieces installed (which aren't that many)..

On a hunch (since the problem was local to Visual Studio) I started looking at Tools and Extensions, and noticed on this component the Date Installed being past the most recent Windows Update! The Automatically update this extension checkbox was checked (by default?).
So it must have silently updated upon VS restart?!

Granted, updates are useful and sometimes necessary. But they also may introduce problems. Performing updates automatically is one thing. But not informing the user about it is bad!

Here's an excerpt from the C:\TEMP\VSIXInstaller_f0335270-1a19-4b71-b74b-e50511bcd107.log:

The problem is correctly identified by Jon Skeet in his comment to the original post.

The best resource I found concerning this topic is by Stephen Toub.

tldr: "To make it easier for developers to write asynchronous code based on Tasks, .NET 4.5 changes the default exception behavior for unobserved exceptions. While unobserved exceptions will still cause the UnobservedTaskException event to be raised (not doing so would be a breaking change), the process will not crash by default. Rather, the exception will end up getting eaten after the event is raised, regardless of whether an event handler observes the exception. This behavior can be configured, though. A new CLR configuration flag may be used to revert back to the crashing behavior of .NET 4, e.g."

I don't know much about the implications of using the debugging features of the Visual Studio, but if you run your program without the debugger attached (with Ctrl+F5) the tasks are properly recycled by the garbage collector. The tasks created internally by the ForgottenTask method are not started, so they are not scheduled, and since you are not holding any explicit reference to the tasks there is nothing preventing the garbage collector from recycling them. Here is a minimal demonstration of this behavior:

In a correctly implemented async implementation: there is no wait. Instead, at the bottom of the chain, some code exists that will create some async source, which could be a TaskCompletionSource, an IValueTaskSource[], or something similar - which allows that code to store that token somewhere (for example, in a queue, a correlation dictionary, or an async state object for IOCP), and return the incomplete task to the caller. The caller then discovers that it is incomplete, and registers a "when you have the answer, do this to reactivate me" callback. That calling code now unwinds completely, with every step saying "when you're done, push here", and the thread goes on to do other things, such as service a different request.

At some point in the future (hopefully), the result will come back - again, via IOCP, or via a separate IO reader pulling a response from somewhere and taking the appropriate item out of the queue/correlation-dictionary, and says "the outcome was {...}" (TrySetResult, TrySetException, etc).

For all of that time no threads were blocked. That is, ultimately, the entire point of async/await: to free up threads, to increase scalability.

In incorrectly implemented async systems: anything and everything is possible, including async-over-sync, sync-over-async, and everything else.

If we're an async Task or async Task method, then there's always work to do after the await - we need to ensure that any exception produced by the awaited task is properly propagated to our own Task, or that we pass the appropriate normal return value through.

If we're any kind of async method using structures such as using which insert compiler-generated code at the end of an otherwise appearing empty epilogue to our method, then we'll be inserting code at the end of the method, even if it doesn't appear in the source.

If we're any normal async method that liberally uses awaits then we'll already have a state machine built and running to execute our method and there's no point in optimizing the "no code after the last await" possibility.

In the narrow case that we're an async void¹ method that contains a single await at the end, then other than some minutiae about where an unhandled exception from a task might be reported, we already had the opportunity to avoid excess code generation by not making the method async at all and just ignoring the awaitable.

So there's no reason to try to optimize this situation.

¹We're already in a state of sin at that point anyway.

After reading this article, it seems that the only way to have a synchronization context in ASP Core is to create your own.

The way I imagine it is that there must be some data structure that supports concurrency, e.g. ConcurrentDictionary. Any process can add some values to this data structure and another process will be responsible for reading and broadcasting updated values within the same thread.

Something like this.

Everything I've read about creating a custom ThreadPool says don't.

An alternative would be to use a custom TaskScheduler on the shared ThreadPool.

You can use this TaskScheduler class like this:

Most probably the problem is that the PicturePlot2 component doesn't like being manipulated by non-UI threads. To ensure that the ActionBlock's delegate will be invoked on the UI thread, you can configure the TaskScheduler option of the block like this: