Hi Team,
I am working on the POC for Azure Event hubs to implement same into our application.
Quick Brief on flow.
- Created tool to read the CSV data from local folder and send it to event hub.
- We are sending Event Data in Batch to event hub.
- With 12 instance of tool (Parallel), I could able to send the 6 Lacks lines of messages to Event hub within 1 min.
- But, On receiver side, to receive the 6 Lacks line of data, it taking more than 10 mins.
Need to achieve
- I would like to Match/double my egress speed on the receiver to process the data.
Existing Configuration
- The configuration I have made user are
- TU - 10
- One Event hub with 32 Partition.
- Coding logic goes same as mentioned in MSDN site - https://azure.microsoft.com/en-in/documentation/articles/event-hubs-csharp-ephcs-getstarted/
- (Only difference is, I am sending line of data in a batch.
- EventProcessorhost with options {MaxBatchSize= 1000000, PrefetchCount=1000000 }
Please do provide your input. Let me know if you need more details to be shared?
Regards
Ravindra.