httpflow | A command line utility helps to capture and dump HTTP stream | Security library

It looks like ET.parse expects a filename, not the contents of the file.

Accidentally (more or less) I found a basic concept for an OAuth addon on github, which does exactly what I was looking for: oauth-mitmproxy

So the code would look like this:

If you want to use Python packages that are not included in mitmproxy's own installation, you need to install mitmproxy via pip or pipx. The normal binaries include their own Python environment.

Source: https://docs.mitmproxy.org/stable/overview-installation/#installation-from-the-python-package-index-pypi.

MergePreferred (in the absence of eagerComplete being true) will complete when all the inputs have completed, which tends to generally be true of stages in Akka Streams (completion flows down from the start).

So that implies that the merge can't propagate completion until both the input and extractSubtree signal completion. extractSubtree won't signal completion (most likely, without knowing the stages in that flow) until bcast signals completion which (again most likely) won't happen until processResponse signals completion which* won't happen until httpFlow signals completion which* won't happen until createRequest signals completion, which* won't happen until merge signals completion. Because detecting this cycle in general is impossible (consider that there are stages for which completion is entirely dynamic), Akka Streams effectively takes the position that if you want to create a cycle like this, it's on you to determine how to break the cycle.

As you've noticed, eagerComplete being true changes this behavior, but since it will complete as soon as any input completes (which in this case will always be the input, thanks to the cycle) merge completes and cancels demand on extractSubtree (which by itself could (depending on whether the Broadcast has eagerCancel set) cause the downstream to cancel), which will likely result in at least some elements emitted by extractSubtree not getting processed.

If you're absolutely sure that the input completing means that the cycle will eventually dry up, you can use eagerComplete = false if you have some means to complete extractSubtree once the cycle is dry and the input has completed. A broad outline (without knowing what, specifically, is in extractSubtree) for going about this:

• map everything coming into extractSubtree from bcast into a Some of the input
• prematerialize a Source.actorRef to which you can send a None, save the ActorRef (which will be the materialized value of this source)
• merge the input with that prematerialized source
• when extracting the subtree, use a statefulMapConcat stage to track whether a) a None has been seen and b) how many subtrees are pending (initial value 1, add the number of (first generation) children of this node minus 1, i.e. no children subtracts 1); if a None has been seen and no subtrees are pending emit a List(None), otherwise emit a List of each subtree wrapped in a Some
• have a takeWhile(_.isDefined), which will complete once it sees a None
• if you have more complex things (e.g. side effects) in extractSubtrees, you'll have to figure out where to put them
• before merging the outside input, pass it through a watchTermination stage, and in the future callback (on success) send a None to the ActorRef you got when prematerializing the Source.actorRef for extractSubtrees. Thus, when the input completes, watchTermination will fire successfully and effectively send a message to extractSubtrees to watch for when it's completed the inflight tree.

It looks like you want to use response.text or response.content, not response.raw_content. raw_content contains the raw compressed HTTP message body, whereas .content contains the uncompressed version.

Although my prior script should work but somehow it didn't I eventually get it working by changing the script as follows:

This functionality has been temporarily removed on master when we shifted to the new sans-io proxy core. I have coincidentally opened a pull request yesterday that brings it back (#4502). The new API is different, but accomplishes the same:

You probably misunderstood what is returnChannel on the Amqp.outboundGateway and try to rely your logic on it. Please, make yourself familiar with that Publisher Confirms and Returns feature: https://docs.spring.io/spring-amqp/docs/current/reference/html/#cf-pub-conf-ret.

It is also not clear what is a replyBackToHttp flow purpose, but it confuses at the moment with mixed references to other beans.

You probably need to investigate what is a request-reply configuration from Spring AMQP respective and you would probably don't try to use another queue for replies. Although it is still possible: see replyAddress property or RabbitTemplate: https://docs.spring.io/spring-amqp/docs/current/reference/html/#request-reply

You are trying to access flow.response in the request hook. The request hook is triggered before mitmproxy sends the request to the target server, so you clearly don't have a response yet. The easy fix here is to use def response(...): ....