Quantcast
Channel: Service Bus forum
Viewing all articles
Browse latest Browse all 1916

Issue: EventHub - Single ConsumerGroup read same data in same partition mutiple times

$
0
0

Hi There,

I create one worker role that deploy as EventHub receiver and using one custom ConsumerGroup.

I am sending two following message to Event hub. During sending message I did not mention in partition key.

ehnanme: XXXXXX
Sending messages to Event Hub conn2hub
Test Data - Counter0 - Time 18-May-2015 09:47:35
Test Data - Counter1 - Time 18-May-2015 09:47:41
Press enter key to stop worker.


But on receiver side, I have only one Consumer Group. But as per Worker Role logs I can see my receiver reading same data in same partition multiple times as you can see below.

Microsoft.WindowsAzure.ServiceRuntime Verbose: 500 : Role instance status check starting
Microsoft.WindowsAzure.ServiceRuntime Verbose: 502 : Role instance status check succeeded: Ready
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '14', Offset: '1728'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '15', Offset: '1728'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '9', Offset: '432'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '10', Offset: '288'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '12', Offset: '864'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '3', Offset: '1152'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '0', Offset: 1008, Data: 'Test Data - Counter1 - Time 18-May-2015 09:47:41'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '1', Offset: 720, Data: 'Test Data - Counter0 - Time 18-May-2015 09:47:35'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '4', Offset: '1152'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '2', Offset: '720'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '8', Offset: '1152'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '6', Offset: '288'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '13', Offset: '1008'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '0', Offset: '1008'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '1', Offset: '720'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '7', Offset: '864'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '11', Offset: '1440'
WaWorkerHost.exe Information: 0 : EventMessageProcessor OpenAsync.  Partition: '5', Offset: '432'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '0', Offset: 1008, Data: 'Test Data - Counter1 - Time 18-May-2015 09:47:41'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '1', Offset: 720, Data: 'Test Data - Counter0 - Time 18-May-2015 09:47:35'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '0', Offset: 1008, Data: 'Test Data - Counter1 - Time 18-May-2015 09:47:41'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '1', Offset: 720, Data: 'Test Data - Counter0 - Time 18-May-2015 09:47:35'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '0', Offset: 1008, Data: 'Test Data - Counter1 - Time 18-May-2015 09:47:41'
WaWorkerHost.exe Information: 0 : Message received.  Partition: '1', Offset: 720, Data: 'Test Data - Counter0 - Time 18-May-2015 09:47:35'

WaWorkerHost.exe Information: 0 : Working
Getting "eventHubName" from ServiceRuntime: PASS.
Getting "messageProcessorConsumerGroupName" from ServiceRuntime: PASS.

As per logs, EventHub Receiver read same data from same Partition number 0 four times and Partition number 1 four times. Can you please suggest why it is doing like this because I have to run some rule engine once our system receive data from EventHub. But in this case, same data will be processed multiple time which will be big impact on our system.

How can I resolve this issue, single consumergroup should read data only one time.

Ankush





Viewing all articles
Browse latest Browse all 1916

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>