EventHub | An open source event analytics platform | Analytics library

I currently have a ASA Job that is streaming to an eventhub. From what I understand it may combine events from my query into batches for throughput reasons. However, when I check my output eventhub using service bus explorer, my events are not kept in a list like this:

I've been trying to fix this since yesterday but have done more damage than good. I have a function app (written in JS) in the Azure Portal. All was working well until two days ago when I received the below error. I've seen a few bits online saying the fix is to update the reference however I'm not really sure where I should update the reference to NuGet package. In my function app's code I have both a js file and the json file but I don't know where the NuGet package comes in. Apologies if this is trivial I'm still learning but would really like to understand what's going on here. For reference I'm on a Mac and have been working off VS code.

Microsoft.Azure.WebJobs.Script: One or more loaded extensions do not meet the minimum requirements. For more information see https://aka.ms/func-min-extension-versions.

ExtensionStartupType EventHubsWebJobsStartup from assembly 'Microsoft.Azure.WebJobs.EventHubs, Version=4.2.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not meet the required minimum version of 4.3.0.0. Update your NuGet package reference for Microsoft.Azure.WebJobs.Extensions.EventHubs to 4.3.0 or later.

I've been working on a azure functions project for almost a year now. Suddenly last week i started to get this error message from my IoTHubTrigger:

The listener for function 'IotHubTrigger' was unable to start. Microsoft.Azure.EventHubs.Processor: Out of retries creating lease for partition 0. Microsoft.WindowsAzure.Storage: The response ended prematurely, with at least 158 additional bytes expected. System.Net.Http: The response ended prematurely, with at least 158 additional bytes expected.

Have not found anywhere where someone has the same problem as this, someone who knows what the error is due to? The only major change the last weeks is that i went from VS 2019 to VS 2022, can it be that?

Also added "AzureWebJobsSecretStorageType": "files", to the local.settings.json file.

I'm thankful for any kind of help with this! Cheers!

I'm trying to use Spring Cloud Stream to process messages sent to an Azure Event Hub instance. Those messages should be routed to a tenant-specific topic determined at runtime, based on message content, on a Kafka cluster. For development purposes, I'm running Kafka locally via Docker. I've done some research about bindings not known at configuration time and have found that dynamic destination resolution might be exactly what I need for this scenario.

However, the only way to get my solution working is to use StreamBridge. I would rather use the dynamic destination header spring.cloud.stream.sendto.destination, in that way the processor could be written as a Function<> instead of a Consumer<> (it is not properly a sink). The main concern about this approach is that, since the final solution will be deployed with Spring Data Flow, I'm afraid I will have troubles configuring the streams if using StreamBridge.

Moving on to the code, this is the processor function, I stripped away the unrelated parts

I have to run the following Pyspark code. I am reading from eventhub, transforming the data using multiple functions (dataframe transformation) and writing the dataframe to a directory. The update_session_id function has to run for each batch, but it is not working on the data from eventhub. It just has to update a lookup table which is referenced in the transform_raw_data function, if the current_timestamp is greater than the 2 hours from the timestamp maintained in the lookup table.

How can I implement this? Currently, the update_session_id function just executes once and then doesn't execute through out the lifetime of the stream.

I would like to write custom data frame to eventhub.

I am attempting to setup an APIM endpoint that sends messages to an event hub. I also want to use managed identities in order to authorize the APIM with the event hub. Note that all resources lie in the same subscription. The setup is as follows:

• I have an APIM instance with a system assigned identity. This identity has been giving the contributor role on a subscription level.
• I have an event hub namespace and event hub, which is setup to receive the events.
• I have created an API + operation, that generates events, based on the payload and sends them to the event hub. The example below just sends some hardcoded body, I want to get it working before working on the payload.

The policy for the operation looks like this:

I'm having trouble installing the following packages in a new python 3.9.7 virtual environment on Arch Linux.

My requirements.txt file:

I have successfully connected two actual devices to Azure IoTHub (in the same Iot Hub) and would like the second device to receive the message that the first device sends. So, in a normal MQTT broker the second device just subscribes to that topic but Azure does not have a normal MQTT broker.

What I am now trying to do is write an Azure function that triggers every time a message from the first device is received in IoTHub through the Event Hub Trigger; and sends a C2D message with the received message (string) to the second device. To achieve that the second device subscribes to this topic: devices/secondDevice/messages/devicebound

Here is my function