I have several windows services instances accessing its own namespaces in same one service bus host. But lately most of services start getting strange errors when trying to work with service bus (the first point where each service goes when it is starting is creating a topic for itself). But I still able to access same namespace manually with the very same connection sting via servicebus explorer.
I see several errors in event viewer like:
MessagingEntityAlreadyExistsException
Microsoft.ServiceBus.Messaging.MessagingEntityAlreadyExistsException:The remote server returned an error:(409)Conflict.SubCode=40900.Conflict.TrackingId:357526f0-1635-4e16-bcf0-d1c963d65184_GWIN-NKD9IS8A8GG,TimeStamp:4/8/201610:07:59 AM --->System.Net.WebException:The remote server returned an error:(409)Conflict.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.ServiceBus.Messaging.ServiceBusResourceOperations.CreateOrUpdateAsyncResult`1.<GetAsyncSteps>b__14(CreateOrUpdateAsyncResult`1 thisPtr,IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
--- End of inner exception stack trace ---
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at AFT.RegoV2.RegoBus.Bus.WindowsServiceBus.<>c__DisplayClass24_0.<EnsureTopicExists>b__0() in C:\TeamCity\buildAgent3\work\218e2fa0847c410\RegoBus\Bus\WindowsServiceBus.cs:line 263
at AFT.RegoV2.Shared.Synchronizatio...
But I am 100% sure that there is no topics in that namespace. I checked connection string, checked namespace, removed it manually before it and so on.
Next one is:
SecurityTokenService warning:System.DirectoryServices.AccountManagement.PrincipalServerDownException:The server could not be contacted.--->System.DirectoryServices.Protocols.LdapException:The LDAP server is unavailable.
at System.DirectoryServices.Protocols.LdapConnection.Connect()
at System.DirectoryServices.Protocols.LdapConnection.SendRequestHelper(DirectoryRequest request,Int32& messageID)
at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request,TimeSpan requestTimeout)
at System.DirectoryServices.AccountManagement.PrincipalContext.ReadServerConfig(String serverName,ServerProperties& properties)---End of inner exception stack trace ---
at System.DirectoryServices.AccountManagement.PrincipalContext.ReadServerConfig(String serverName,ServerProperties& properties)
at System.DirectoryServices.AccountManagement.PrincipalContext.DoServerVerifyAndPropRetrieval()
at System.DirectoryServices.AccountManagement.PrincipalContext..ctor(ContextType contextType,String name,String container,ContextOptions options,String userName,String password)
at System.DirectoryServices.AccountManagement.PrincipalContext..ctor(ContextType contextType)
at Microsoft.Cloud.ServiceBus.Common.Security.SecurityExtensions.IsDomainController()
at Microsoft.Cloud.ServiceBus.Common.Security.SecurityExtensions.TryOrDefault[T](Func`1 func, Boolean trace)
But strange thing that I can take the connection string from log or debug and connect to service bus via Servicebus Explorer freely. And at the same time service is not able to connect using the very same string.
And third one is:
TrackingId: f5bb8bfe-a5c2-4b86-b982-4beaca4defba,SubsystemId:NoSystemTracker,Gateway:Management resource manifest is marked as broken.Details:Cleanup:SetState='Broken'for1 resources
I tried google all of them, tried all receipts that I could find but still no luck. in specifics i tried:
-) restart servicebus farm
-) reboot entire server where servicebus is installed
-) remove and create namespace manually and with our deploy scripts
-) tried both oauth and win ways of authentication
-) both available transport types (amqp and netmessaging)
Code which services use to create topic:
using(var scope =CustomTransactionScope.GetTransactionSuppressedScope()){
_syncService.Execute("WindowsServiceBus_"+ topicName,()=>{var namespaceManager =GetNamespaceManager();if(!namespaceManager.TopicExists(topicName)){var topicDescription =newTopicDescription(topicName);
namespaceManager.CreateTopic(topicDescription);}});
scope.Complete();}
_syncService is just a way to ensure that this method will be executed in one thread only for each topic
Any advice is very appreciated.