Quantcast
Channel: Service Bus forum
Viewing all articles
Browse latest Browse all 1916

CheckpointAsync may possibly be skipping offsets

$
0
0

I wrote C# code to watch the data in my test Event Hub.  I wrote it to see data that would come from API Management (using the Logger feature).  I thought that there was a bug in the API Management code, but needed to prove it out so I copied some code I found that I could just place random messages into my Event Hub.  This ended up showing that it was not API Management and that the problem is in the watcher.

After investigating, it looks like using the IEventProcessor.ProcessEventsAync, whenever I checkPointAsync the offset jumps much more than I expected.

Here is an example of what I added to Event Hub

3/11/2016 1:47:16 PM > Sending message: 77ee90cb-138a-438e-b004-b009dbab38fa
3/11/2016 1:47:20 PM > Sending message: fc23242e-9650-4477-8691-f1442ad0646c
3/11/2016 1:47:20 PM > Sending message: 891192ce-b417-4a5a-b198-b943a63b22b8
3/11/2016 1:47:21 PM > Sending message: 930d1621-4b7f-4a61-bf7b-b193996d60e6
3/11/2016 1:47:21 PM > Sending message: c1aa6c67-606e-42ba-ad02-a238f46b2b76
3/11/2016 1:47:21 PM > Sending message: 8d9160f8-ab34-4b7d-8e60-a266524dcae5
3/11/2016 1:47:22 PM > Sending message: 7e4474c2-bdf9-450c-a300-e89a87bcc979
3/11/2016 1:47:22 PM > Sending message: 153284ee-b6af-441b-b74e-aef63790101f
3/11/2016 1:47:22 PM > Sending message: 77d2f6a2-5f36-4f3b-a276-663720fbfb74
3/11/2016 1:47:23 PM > Sending message: 2275e8e4-1a1c-47a1-87e4-0313dd7c5739
3/11/2016 1:47:23 PM > Sending message: 4d967915-e1ad-4e5c-92b3-1936517f2a0e
3/11/2016 1:47:24 PM > Sending message: a68d5ea7-4e09-4c14-b24c-2e9c629f0553
3/11/2016 1:47:24 PM > Sending message: a815ff52-af86-4794-b867-3cad7f21b748
3/11/2016 1:47:25 PM > Sending message: a7c8b3e2-7632-460e-9eb6-d0c5c40ec613
3/11/2016 1:47:25 PM > Sending message: 083c4221-d663-425d-9ad7-c758fdc2363d
3/11/2016 1:47:25 PM > Sending message: 79fff1c9-b364-41c2-ae3f-27e0b141f92d
3/11/2016 1:47:26 PM > Sending message: 0b42e03d-b830-474d-91fb-b920da20f713
3/11/2016 1:47:26 PM > Sending message: 8377e88b-3215-403f-ac40-54db14ff6ee7
3/11/2016 1:47:27 PM > Sending message: bf48c309-67ca-44f4-8544-92d56ae2c557
3/11/2016 1:47:28 PM > Sending message: 8f36ec09-82d4-4d5f-a92a-7d2997fb504a
3/11/2016 1:47:29 PM > Sending message: c83a2a42-9ad7-4cf2-b737-6aa5d3665987
3/11/2016 1:47:31 PM > Sending message: 3f29c6a2-dd73-4f5f-af0d-c3521fd00349

Here is what the watcher reported

offset 12968
Partition: '3', 'Data= 77ee90cb-138a-438e-b004-b009dbab38fa
offset 13048
Partition: '3', 'Data= c1aa6c67-606e-42ba-ad02-a238f46b2b76
offset 13128
Partition: '3', 'Data= 77d2f6a2-5f36-4f3b-a276-663720fbfb74
offset 13208
Partition: '3', 'Data= a815ff52-af86-4794-b867-3cad7f21b748
offset 13288
Partition: '3', 'Data= 0b42e03d-b830-474d-91fb-b920da20f713
offset 13368
Partition: '3', 'Data= c83a2a42-9ad7-4cf2-b737-6aa5d3665987

Here is the code in the ProcessEventsAsync

async Task IEventProcessor.ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
        {
            Console.WriteLine("offset {0}", context.Lease.Offset.ToString());
            foreach (EventData eventData in messages)
            {
                string data = Encoding.UTF8.GetString(eventData.GetBytes());

                Console.WriteLine(string.Format("Partition: '{0}', 'Data= {1}", context.Lease.PartitionId, data));
            }

            await context.CheckpointAsync();
        }
Is there a bug or am I doing something wrong?



Viewing all articles
Browse latest Browse all 1916

Trending Articles