Quantcast
Channel: Service Bus forum
Viewing all articles
Browse latest Browse all 1916

Sporadic MessageLockLostException

$
0
0

Every once in a while (1 in thousands of messages) I am getting a MessageLockLostException when trying to complete a message.

I have a queue defined with lock duration set to 30s:

In code I create a listener with the following options:

var messageHandlerOptions = new MessageHandlerOptions(exceptionReceivedHandler)
{
   MaxConcurrentCalls = 10,
   AutoComplete = false,
   MaxAutoRenewDuration = TimeSpan.FromSeconds(180)
};

var queueClient = ResolveQueueClientByQueueConfigName(queueConfigName);

_logger.Info($"Listening on queue {queueClient.ServiceBusConnection.Endpoint}{queueClient.QueueName}");
queueClient.RegisterMessageHandler(MyMessageHandler, messageHandlerOptions);

And in MyMessageHandler I complete the message after successful processing:

private static async Task CompleteAsync(IReceiverClient queueClient, string lockToken)
{
   await queueClient.CompleteAsync(lockToken);
}

private async Task MyMessageHandler(Message message, CancellationToken cancellationToken)
{
   //Processing omitted
   await CompleteAsync(queueClient, message.SystemProperties.LockToken);        
}

Like I said, this works fine for most of the messages, but once in a while I get the following:

2020-03-27 10:31:01,004 [64] INFO Received message: MessageId:7da4f7ec-9e27-4951-83f5-7f29d8fc93a8 2020-03-27 10:31:01,005 [64] INFO Entity Framework Core "3.1.0" initialized '"SmsDataContext"' using provider '"Microsoft.EntityFrameworkCore.SqlServer"' with options: "None" 2020-03-27 10:31:01,087 [109] INFO Sending message to smssender, CorrelationId: 7da4f7ec-9e27-4951-83f5-7f29d8fc93a8, label: SmsSender 2020-03-27 10:31:05,205 [158] INFO Completing message '7da4f7ec-9e27-4951-83f5-7f29d8fc93a8'2020-03-27 10:31:05,206 [158] ERROR Message handler encountered an exception Microsoft.Azure.ServiceBus.MessageLockLostException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance. at Microsoft.Azure.ServiceBus.Core.MessageReceiver.DisposeMessagesAsync(IEnumerable`1 lockTokens, Outcome outcome) at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.CompleteAsync(IEnumerable`1 lockTokens) at Bcp.AzureServiceBusHelper.ServiceBusClientHelper.CompleteAsync(String queueName, Message message) at Bcp.Sms.Validator.ServiceHost.HostedService.ProcessMessagesAsync(Message validatorMessage, CancellationToken token) in *** at Microsoft.Azure.ServiceBus.MessageReceivePump.MessageDispatchTask(Message message). Context: Endpoint= prdhybrid.servicebus.windows.net; Entity Path: smsrequestvalidation; Executing Action: UserCallback2020-03-27 10:31:30,949 [109] INFO Received message: MessageId:7da4f7ec-9e27-4951-83f5-7f29d8fc93a8

So in this case the lock was taken at 10:31:01 and the message complete failed 4 seconds later at 10:31:05, well within the 30 second lock duration.

The last log line indicates the message was retried after 30 seconds at 10:31:30 which tells me that the lock should have been valid at 10:31:05.

Could it be that the lock contained in message.SystemProperties.LockToken is corrupted? Also, could this be due to a short network interruption? I would imagine the LockToken stays valid in this case.

This application is written in .net core 3.1 using Microsoft.Azure.ServiceBus 4.1.2

Thanks in advance for the help!



Viewing all articles
Browse latest Browse all 1916

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>