substream | Volatile namespaces on top of Primus streams | Websocket library

This is the closest that I can think of which behaves that way you are wanting . I'm putting it here to help with the discussion. In your example, you have two identifiers; for one used when creating A objects, and the other used when creating B objects. With this code, those identifiers are created up-front using the same logic that you had (although I replaced AtomicInteger with IntStream). flatmap is still used, but not at object creation time.

You do get a state per substream in that setup:

The answer to this is silly, but it may help anyone who has trouble. For the /live link, you can write just as it is. But for the real monitor link you need to add the rstp link inside " ".

For example

ffmpeg -rtsp_transport tcp -v verbose -i "rtsp://admin:admin1234@172.xxx.xx.xxx:82/cam/realmonitor?channel=1&subtype=1 " -f hls -hls_flags delete_segments -hls_time 5 -segment_time 5 -hls_list_size 5 C:\Apache24\htdocs\ipcam\video_1\stream.m3u8

So, do I need the .async after the groupBy to ensure that all substreams are executing in parallel or not? Is this test that I am doing validity the parallelism of an operator in Akka stream?

The short answer is "yes", you need async.

As a rule of thumb in Akka Streams (and other reactive streams spec implementation like RxJava or Project Reactor), you need to explicitly demarcate async boundaries. By default the streams are executed single threaded (or single actor in case of Akka Streams). That includes operators like groupBy. This may seem a bit counter intuitive at first, but when you think about it, parallel execution is not really a must in groupBy semantics, even though often you want parallel execution because it's the very reason you apply groupBy, be it to use all cores available for some computation task or perhaps to call some external service in parallel and get better throughput. In those cases you need to explicitly code for that parallelism to occur. One way is using async as you did in your example, where the stream execution implementation logic would introduce that parallelism or you could also user mapAsync where the parallelism is introduced by some means external to the stream execution logic.

A nice example is shown in the stream-cookbook:

Two observations:

• mergeCountReduce will emit the first key it saw with the sum of the values seen (and will fail the stream if it didn't see any elements)
• mergeCountFold will emit the last key it saw and the sum of the values seen (and will emit a key and value of zero if it didn't see any elements)

(in both cases, though the key is always the same)

Neither of those observations are affected by the async boundary.

In the context of the preceding Balance operator, though, async introduces an implicit buffer, which prevents the graph it wraps from backpressuring until that buffer is full. Balance sends stream values to the first output which isn't backpressuring, so if the stage after Balance is not dramatically slower than the upstream, Balance may send values only to one output (B1 in this case).

In that scenario, with reduce, B1 would emit the key and count, while B2 fails, causing the whole stream to fail.

For fold, in that scenario, B1 would emit the key and count, while B2, not having seen any values would emit (0,0). The merge would emit them in the order they emitted (reasonable to assume a 50/50 chance), so the final fold would then either have the key and the count or zero and the count.

Using groupingBy can help here along with a downstream of minBy/maxBy. This would provide a map to look up if the value after filtering is present per type.

statefulMapConcat is the multitool in the Akka Streams library.

According to the implementation of the Source.unfoldAsync the passed in function is only called when the source is pulled: