Me
Massaging operations?
- Brokered connections?
- Relay message volume?I created a simple console app that creates a topic, allows you to send a message to that 1 topic; and also created the client to listen in and subscribe to that topic, the subscription name is dynamically generated.; its all based on the sample code thats available.
If I run the application (call it runA), I can enter a message, and the listener will post it back on the console - this works.
If I run the application (call it runB), I can enter a message, and the listener will post it back on the console, and also show it on runA - this works.
is the topic-subscriber max count still 2000? If I wanted to create a simple chat application with N listeners is there a better way to do this?
Can we scale up and scale down the throughput units using SDK when the throughput units are more than the what the Azure portal will let you set? So, if I call support to increase the throughput units on a service namespace and need to scale it down on an hourly basis using the SDK then is that possible?
Twitter: @SharePointLife
I've been looking at the example code for using apache qpid with service bus: https://azure.microsoft.com/en-gb/documentation/articles/service-bus-java-how-to-use-jms-api-amqp/
If I download the jar files linked by this article I can get the example code to run, however if I use the latest qpid libraries from https://qpid.apache.org/download.html then the example fail - in particular I get the error that amqps is not a valid protocol.
I'm not sure if I'm missing something but I would have assumed that the latest apache libraries should work to connect to Service Bus ?
While sending the data to Eventhub we can define the partition key with message and internally a hashing function will map the key to a specific partition ID.
On the consumer side , the consumer group will need to read from a specific partition id value. Now if the Partition id is not known then how will I read the message? How will I map the consumer to a partition?
According to MSDN :
"When using partition keys, the event publisher is only aware of its partition key, not the partition to which the events are published. This decoupling of key and partition insulates the sender from needing to know too much about the downstream processing and storage of events. Partition keys are important for organizing data for downstream processing, but are fundamentally unrelated to partitions themselves."
And for Consumer:
" A consumer must connect to a specific partition."
This seems quite contradictory.
I am getting data from 50 different clients every minute 50*1440*31 = 2232000.I need the data to be processed and updated real-time.Each client data to be processed synchronously.All the components i am using is Azure like (Web, DB, Services)
So initially i am doing processing once i get the message and respond back to client.Due to delay in the processing on server.Client response is getting delayed.So i looked for 3 options to give immediate response.
a) Using Service Bus Queue.
b) Running using Parallel Processing (Task.StartNew()).
c) Store the Message and Process it later using worker role continuously.
So i started using Service Bus Queue.
I added one queue and try to load all the 50 clients data into one then process it from queue using Azure Worker role.Then queue length is getting increasing due to delay in processing of message for each client.Each msg processing is taking some where around 15sec.But i need the data to be processed real-time with in that minute the message receives.
So i added separate queue for each client to load the message specific to the client.By using azure worker role i am processing the messages from these 50 queues using 50 queue clients and try to process the messages like as shown below
public override void Run()
{
Trace.WriteLine("Starting processing of messages");
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = true;
options.MaxConcurrentCalls = 1;
options.ExceptionReceived += LogErrors;
_client1.OnMessage((receivedMessage) =>
{
ProcessMessage(receivedMessage);
}, options);
_client2.OnMessage((receivedMessage) =>
{
ProcessMessage(receivedMessage);
}, options);
.
.
.
_client50.OnMessage((receivedMessage) =>
{
ProcessMessage(receivedMessage);
}, options);
CompletedEvent.WaitOne();
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// Create the queue if it does not exist already
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.QueueExists(QueueName1))
{
namespaceManager.CreateQueue(QueueName1);
}
if (!namespaceManager.QueueExists(QueueName2))
{
namespaceManager.CreateQueue(QueueName2);
}
.
.
if (!namespaceManager.QueueExists(QueueName50))
{
namespaceManager.CreateQueue(QueueName50);
}
_client1 = QueueClient.CreateFromConnectionString(connectionString, QueueName1);
_client2 = QueueClient.CreateFromConnectionString(connectionString, QueueName2);
.
.
_client50 = QueueClient.CreateFromConnectionString(connectionString, QueueName50);
return base.OnStart();
}
public override void OnStop()
{
// Close the connection to Service Bus Queue
_client1.Close();
_client2.Close();
.
.
_client50.Close();
CompletedEvent.Set();
base.OnStop();
}Clarifications required:
a) Is the above code works parallel execution of data from multiple clients?If not please let me know what needs to change in code
b) Is this the best way to process the message from 50 queue clients with in that minute?
c) For real-time data processing is it better to use queues in Azure Service Bus or any other options we can use in Azure?
d) One more thing need clarification is how the Azure Usage costs for this intermediate layer added as Service Bus Queue for 50 Queues for the total number of Messages and the Worker role calls for every minute for a monthly transactions around 50*1440*31 = 2232000.If the monthly costs are not more than $100 per month for message storing and message processing using worker role as above we can look for retaining this option of Service Bus Queue.Other wise i need to look for other options in Azure to process it for cost optimizations.I did not get very clear on Service Bus Brokered Connection and costs on Azure Website.That why i am asking about it.
e) How reliable this Azure Service Bus Queue.Once we added all data to be Service Bus Queue.This component needs to be available all time during the day for whole year.This will be critical component for us.If this Service Bus Queue is down for some reason then whole system is down.Such a mission critical.Is it good to use Service Bus Queue for this kind of scenarios?
f)Is there any thing we need to do to make the Service Bus Queue up all the time for the whole year.Is there a way we can get notification email if the service bus queue goes down.To handle processing in different manner.
Please suggest some one from Azure Team or people who is using Azure in these kind of scenarios in their applications.
Thanks in advance
I accidently removed the certificates and now I cannot remove the Service Bus, the error I get is;
This computer is part of a farm. To uninstall, first leave the farm either using 'Leave from Farm' option in Service Bus Configuration tool or Remove-SBHost cmd-let in PowerShell."
When trying the Remove-SBHost powershell command I get this error:
"Remove-SBHost : Certificate requested with thumbprint EC7608B922349F8A8E22SE73RT5216E6B59876G5 not found in the certificate store LocalMachine\My."
How can I uninstall it now?
I am attempting to create an Event Hub. When I do, the error "Failed to create Event Hub 'myeventhubname'." is displayed. Expanding the details reveals the message "Please try again. If the problem persists, contact support."
There aren't a lot of ways to stray when using Quick Create to create an event hub. The only option is the name of the event hub. What's going on?
I've been researching event hubs and I'm trying to find a good way to auto scale out my event consumer worker roles. So far, I've only found documentation that suggests using CPU usage as a metric to auto scale, but that obviously won't work if my worker roles are I/O bound. What I really want is some way to measure the number of unprocessed messages for a consumer group which would indicate that I'm falling behind in processing incoming messages for that group, but I haven't found a way to measure this. Is there some way to do this? Or am I going about this the wrong way?
Thanks in advance for any suggestions you may have.
From @chorpie via Twitter:
Any idea why NotificationHubClient.CreateRegistrationIdAsync() would return a 404? I'm using C# and the provided connection string for DefaultFullSharedAccssSignature generated from the Azure Portal.
https://twitter.com/chorpie/status/636959718989164544
Thanks!
@AzureSupport
Referring to link https://msdn.microsoft.com/en-us/library/jj192996.aspxI found the performance counters for Windows Service Bus 1.1 but surprisingly I am not able to find the Threshold for each counters.
Is the threshold is not yet derived for Windows Service Bus counters or I am not able to trace.
Can someone help me to find the threshold for these performance counters.
Hi,
I'm calling the Service Bus REST API to get data from a Topic Subscription from an IoT device that only supports GET and POST verbs.
Tried to use the X-HTTP-Method to send a DELETE but it seems to fail.
https://nsnsnsns.servicebus.windows.net/topic1/subscriptions/subscription1
Couldn't find any reference to this in the documentation...
Can someone please confirm if this is supported or not?
Thanks,
www.eclo-mobile.com
There seems to be two ways to create a new MessagingFactory - Create() and CreateFromConnectionString(). I have been using the later, but now I need to adjust the OperationTimeout which you can only seem to do using Create() - and that requires a TokenProvider.
So, how can I set the OperationTimeout using CreateFromConnectionString(), or conversely, how can I use Create() if I have a connection string?
Thanks for any help,
Bob
Hi,
This morning we experienced a very slow connection towards the service bus while trying to troubleshoot dead letters. We are using the Service Bus explorer to trouble shoot deadletters. For about an hour we couldn't get (decent) access to the service bus queues and topics. Also we noticed quite some delay (minutes) in processing events from the bus. So more or less we were stuck for a certain moment and couldn't answer (support) questions from our users.
Any advice on this?
Best regards,
Bart
bart
Hi,
We use Azure Service Bus queue to send/receive messages between two different applications in our solution.
We also need to have a development, stage and production deployments running.
As you can imagine, we must somehow isolate messages for all these three deployments.
We use automated build/deployment so we must automate thing as much as possible.
Any ideas?
.NET Software developer for industrial internet and automation system.
Hello,
Does anyone have any insight on whether the on premise version of the service bus will be supported in the foreseeable future? It seems that it hasn't been updated since 2013, even though a yearly update was promised. This gives me the impression this product will overcome the same fate as other on premise products such as App Fabric, i.e. will be discontinued. Would it be smart to use this product for new development? Is there any road map?
Hi
I am trying to access my SB queue named ifslabscustomer_ifsapplicationsqueue using a WorkerRole in the cloud but getting the following exception:
A first chance exception of type 'System.ServiceModel.FaultException`1' occurred in Microsoft.ServiceBus.dll
A first chance exception of type 'System.TimeoutException' occurred in Microsoft.ServiceBus.dll
I have a standard implementation as follows:
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// Create the queue if it does not exist already
string connectionString = CloudConfigurationManager.GetSetting("ServiceBusQueueConnectionString");
string queueName = CloudConfigurationManager.GetSetting("ServiceBusQueueName");
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.QueueExists(queueName))
{
namespaceManager.CreateQueue(queueName);
}
// Initialize the connection to Service Bus Queue
Client = QueueClient.CreateFromConnectionString(connectionString, queueName);
return base.OnStart();
}
public override void Run()
{
Client.OnMessage((receivedMessage) =>
{
Trace.WriteLine("Received message");
}
CompletedEvent.WaitOne();
}
Bert Jackson
Is the retry policy set by default when using the service bus with the latest sdk?
I assume you do not need to set the retry policy, unless you want to change the behaviour of the default policy?
Thanks,
Andrew
In our Project, we use two EventProcessorHost (different host name, same consumer group: hostA and hostB) to consume data from an EventHub, we send 100 messages(message 1 ~ message 100) to the hub, below is our processor code:
publicasyncTask ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> events)
{
foreach (var eventData in events)
{
//handle the event data
}
await context.CheckpointAsync();
}
now we run two host to consume data, at first, only hostA can got data (message1, 2, 3, …),but when hostA got message50, then we got the
ReceiverDisconnectedException,then hostA cannot got data any more.
Then hostB start to consume data, but it not start with message50, it start with message40, it also can get message40-message50 which were got by hostA. We already call check point for each message when we got the message. If we close the HostB, the hostA also not got the message yet. If we run another host, host will got the data which was not consumed.
So how to make the HostB got data start with message50 when hostA got theReceiverDisconnectedException?