locks | Mutex locks , Read/Write locks | Architecture library

One can achieve this by:

1. Create a dataproc cluster with an environment (your_sample_env) that contains pyspark 3.2 as a package
2. Modify /usr/lib/spark/conf/spark-env.sh by adding

Found the problem.
I was installing pandas_profiling, and this package updated pyyaml to version 6.0 which is not compatible with the current way Google Colab imports packages.
So just reverting back to pyyaml version 5.4.1 solved the problem.

For more information check versions of pyyaml here.
See this issue and formal answers in GitHub

##################################################################
For reverting back to pyyaml version 5.4.1 in your code, add the next line at the end of your packages installations:

I would propose to take a look in the RateLimiter direction. Maybe it does not work as expected, depending on the number of requests your application does over time. From the Javadoc for Ratelimiter: "It is important to note that the number of permits requested never affects the throttling of the request itself ... but it affects the throttling of the next request. I.e., if an expensive task arrives at an idle RateLimiter, it will be granted immediately, but it is the next request that will experience extra throttling, thus paying for the cost of the expensive task." Also helpful might be this discussion: github or github

I could imaginge there is some throttling adding up or other effect in the RateLimiter, i would try to play around with it and make sure this thing really works the way you want. Alternatively, consider using Spring @Scheduled to read from your queue. You might want to spice it up using embedded JMS for further goodies (message persistence etc).

You are going to run into problems with this approach because the call to start a new flow may not immediately cause the list_executions() to show a new number. There may be some seconds between requesting that a new workflow start, and the workflow actually starting. As far as I'm aware there are no strong consistency guarantees for the list_executions() API call.

You need something that is strongly consistent, and DynamoDB atomic counters is a great solution for this problem. Amazon published a blog post detailing the use of DynamoDB for this exact scenario. The gist is that you would attempt to increment an atomic counter in DynamoDB, with a limit expression that causes the increment to fail if it would cause the counter to go above a certain value. Catching that failure/exception is how your Lambda function knows to send the message back to the queue. Then at the end of the workflow you call another Lambda function to decrement the counter.

I suggest to take a look at this package: is_lock_screen

As the description suggest

Useful for determining whether app entered background due to locking screen or leaving app.

I would try with this:

libstdc++ does not produce the error while libc++ does.

iostream.objects.overview Concurrent access to a synchronized ([ios.members.static]) standard iostream object's formatted and unformatted input ([istream]) and output ([ostream]) functions or a standard C stream by multiple threads does not result in a data race ([intro.multithread]).

So this looks like a libc++ bug to me.

I had this exact same problem too. I think it's because of macOS Monterey (12.0). To fix it, run this command in a terminal:

Here is the benchmarking for the existing to OP's question (borrow test data from @Marek Fiołka but with n <- 10000)

Do I need to lock m_task_mutex before signaling [In Stop]?

When the predicate being tested in condition_variable::wait method depends on something happening in the signaling thread (which is almost always), then you should obtain the mutex before signaling. Consider the following possibility if you are not holding the m_task_mutex:

1. The watcher thread (TaskRunnerImpl::Run) wakes up (via spurious wakeup or being notified from elsewhere) and obtains the mutex.
2. The watcher thread checks its predicate and sees that it is false.
3. The signaler thread (TaskRunnerImpl::Stop) changes the predicate to return true (by setting is_running_ = false;).
4. The signaler thread signals the condition variable.
5. The watcher thread waits to be signaled (bad)
   • the signal has already come and gone
   • the predicate was false, so the watcher begins waiting, possibly indefinitely.

The worst that can happen if you are holding the mutex when you signal is that, the blocked thread (TaskRunnerImpl::Run) wakes up and is immediately blocked when trying to obtain the mutex. This can have some performance implications.

In [TaskRunnerImpl::Run] , we are reading is_running_ without locking is_running_mutex. Is this correct?

In general no. Even if it's of type bool. Because a boolean is typically implemented as a single byte, it's possible that one thread is writing to the byte while you are reading, resulting in a partial read. In practice, however, it's safe. That said, you should obtain the mutex before you read (and then release immediately afterwards).

In fact, it may be preferable to use std::atomic instead of a bool + mutex combination (or std::atomic_flag if you want to get fancy) which will have the same effect, but be easier to work with.