Is it possible to get the end of the partition?

My case is:

I manage the offset manually. I want to skip the last events in my partition and start from after the last current event.

I know the only way to do it, it is to receive all messages from my last offset. That means a lot of unnecessary network traffic.

What I want is some method GetLastMessageOffset() or something like that.

Is there such method?

Leonid Ganeline [BizTalk MVP]

I'm using the EventProcessorHost for receiving messages.

I want to initialize the EventProcessorHost and start receiving the last messages.  

I didn't find a method to set up an offset. So now I have to receive all messages in EventHub to position on the last messages.

Seems I can use REST API to modify the BLOB storage, which holds the offsets for all groups / partitions. 

Is there a .NET method to set up the offsets?

Leonid Ganeline [BizTalk MVP]

I know it is not directly related to the EventHub but to the testing of asynchronous processes, but I'm sure you also solved this problem.

I want to test the EventHub solution with both, senders and receivers. Receivers are implemented with EventProcessorHost.

If it is implemented with Direct receiver, the testing is straightforward:

1. call a send message
2. call a receive message
3. compare both messages.

But for EventProcessorHost it doesn't work.

1. call a send message...

- but there is no call to receive. The receive is implemented a s callback to ProcessEventsAsync.

Additional complexity is the ProcessEventsAsync does not return an EventData.

Do you have an idea how to implement testing? Maybe there are samples for it?

Leonid Ganeline [BizTalk MVP]

Hi,

I want to use the Azure Service Bus Topics. I have an app that needs to subscribe to the Service bus Topic. The Nuget-Package for Azure Service Bus is not applicable for Windows Store Apps (I get an error during install), thats why I used the Windows Azure Service Bus managed for Windows Store. Unfortunatelly this API throws a Serialization Exception when I create a subscription. What can I do to use the Service bus in Windows store apps?

Thats how I try to create the subscription:

await Microsoft.WindowsAzure.Messaging.Subscription.CreateAsync(TOPIC_NAME, SUBSCRIPTION_NAME, CONNECTION_STRING);

Hi,

I need some help renewing brokeredmessage lock duration for service bus queue. Can you provide me a sample code snippet which will renew the lock duration for a brokeredmessage when message is about to expire?

Thanks

Subhendu

Thanks Subhendu DE - DotNet Artisan If a post answers your question, please clickMark As Answer on that post. If you find a post helpful, please click Vote as Helpful

Azure Service Bus is completely down in south central and microsoft is not really acknowledging it accurately on their health status page at http://azure.microsoft.com/en-us/status/ or fixing it - when will it be resolved and how is the best way to report outages?

How can we retrieve the number of provisioned service bus namespaces from a subscription programmatically. Are there any published libraries (NUGET packages) from Microsoft to get this information.

Thanks,
Sankaranarayanan MG

Hi,

I found an example online of using SAS to send to an Event Hub via JavaScript on a device.

I got this script to successfully post events, and then it stopped working.

Chrome Developer Tools Network says 401 (40104 Invalid authorization token audience.)

The Console message is:

 XMLHttpRequest cannot load https://devhub01-ns.servicebus.windows.net/devhub02/publishers/pd1/messages. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:16532' is therefore not allowed access. The response had HTTP status code 401.

What is strange is that it worked, and then stopped. Using this tool, I generated a new SAS token (a few times) , and the issue persists, despite the fact the a new SAS token seemed to be what got it working in the first place.

Here is the JavaScript (with the serviceNamespace and HubName changed to disable the SAS):

<input type="text" value="9" id="temp" /><input type="button" value="Send Temperature" onclick="sendTemperature()" /><label id="status"></label><script type="text/javascript">
function sendTemperature() {
    var sas = 'SharedAccessSignature sr=https%3a%2f%2fdevhub01-ns.servicebus.windows.net.servicebus.windows.net%2fdevhub02%2fpublishers%2fpd1%2fmessages&sig=gycTCE6nuqkbdG4GbDhfDkJ%2bLlZD%2b2s5X%2b12BSYCLcw%3d&se=1420704992&skn=Admin';

    var serviceNamespace = "DevHub01-ns";
    var hubName = "devhub02";
    var deviceName = "pd1";

    var xmlHttpRequest = new XMLHttpRequest();
    xmlHttpRequest.open("POST", "https://" + serviceNamespace + ".servicebus.windows.net/" + hubName + "/publishers/" + deviceName + "/messages", true);
	xmlHttpRequest.setRequestHeader('Content-Type',"application/json");
    xmlHttpRequest.setRequestHeader("Authorization", sas);


    xmlHttpRequest.onreadystatechange = function () {
        if (this.readyState == 4) {

            if (this.status == 201) {
                document.getElementById('status').innerText =
                    'Sent: ' + document.getElementById('temp').value;
            } else {
                document.getElementById('status').innerText =
                    this.status;
            }
        }
    };
    xmlHttpRequest.send('{"temperature":"20","area":"Frankfurt"}');
}</script>

Please let me know if you see a reason why this currently doesn't work.

Thanks,

Aron

Hi,

I want to use the ScheduledEnqueueTimeUTC property of the BrokeredMessage to send the message to a service bus queue because I want this message to appear only in a particular delay (1 hour later).

I have this code to set the property:

BrokeredMessage outboundMessage = new BrokeredMessage();            outboundMessage.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddHours(1);

With the ServiceBus explorer, even if I set this property, I'm always able to receive the message !

Is it really working with Service Bus for Windows Server 1.1 ?

Thanks.

This method would seem to be useful for reading multiple message from a queue but I've yet to get more than one.

var messages = await client.ReceiveBatchAsync(50);
await client.CompleteBatchAsync(messages.Select(m => m.LockToken).ToArray());

Where client is a QueueClient. I've tried this using both ReceiveMode settings on the client (i.e. PeekLock and ReceiveAndDelete) and I consistently get only a single message back. I'd like to read all currently available messages from the queue in one pass and without something like a ReceiveBatchAsync then I'd have to await client.ReceiveAsync inside a loop until a null message was returned.

Is it possible to make scales for Incoming and Outgoing messages the same?

Now it looks unnatural. See the picture: outgoing messages seems less than incoming (which is, of course, is not the case). 

Understood, there are several metrics on one graph (counters, sizes, etc.). But it is really confusing if incoming and outgoing message counters in queue have different scales. (BTW It is not related to the error messages).

Leonid Ganeline [BizTalk MVP]

We know the EventHub has the filter function. I'm designing a new solution for a customer and it looks like EventHubs are great for sending and receiving our near-realtime data. Downside is when receiving the data, we receive all data of all our devices in the world. Most of the time, our clients only want to see data of one (or a few) device. We could of course filter the data by ourselves, client side, but this would cost a lot of bandwith.

From theFeedBack, it said Filters will be tied to Consumer Groups. And I Check the Consumer Group Class, it has the Create Time and Update Time, but it has no set Function,

So how to use the Consumer Group to filter Event Data during the Receiving data?

Because in our solution, we use the EventHubHost to consume data, if our service bus worker role restart, we will receive all data in the EventHub, but we only want to receive the latest data,

If we use the EventHubHost to consume data, can we also to filter data? If yes, how to do?

Thanks very much!

Now in our project, we have a Cloud Service WorkRole, in the work role we create an EventProcessorHost in the Run method to consume data,

       publicoverridevoid Run()

       {

          _host = newEventProcessorHost(

               eventHubInfo.HostName,

               eventHubInfo.HubName,

               eventHubInfo.ConsumerGroupName,

               _configurationSetting.ServiceBusConnectionString,

               _configurationSetting.StorageConnectionString);

               ......

       }

We send 100 message to the EventHub (message1 ~ message 100), we can get 100 messages by the _host(host1), it works well.

Eg.But when we get message 50, the work role restart for some reason, it will run the Run method, we will get a new host (host2) to consume data, the host2 will get the message1~message 100,

Do we have a way to make the host2 follow host1, it means the host2 will get Message51~message100, if cannot, we know the event hub has the filter function,

do we have a way for the host2 filter to make it only receive Message51~Message100 using the EventProcessorHost ?

According to documentation the maximum length of a Queue or Topic path is 260 characters.

We have a integration test assuring this. This has always been working. However this now suddenly breaks.

NamespaceManager.GetQueue() now throws

The token provider was unable to provide a security token while accessing 'https://xxxxxx-sb.accesscontrol.windows.net/WRAPv0.9/'. Token provider returned message: 'Error:Code:400:SubCode:T0:Detail:ACS50000: There was an error issuing a token. ACS90013: Object exceeded its maximum length.:TraceID:a40c58e2-6c64-4356-8d2f-d03f608c341d:TimeStamp:2015-01-06 12:04:08Z'. (System.UnauthorizedAccessException)

After some investigation the limit now seems 207 characters.

Is this a breaking change in a recently rolled out Azure service? Is this a bug? Or am I missing something here?

Adding further to the inconsistency.

We also have a test to ensure, when calling NamespaceManager.GetQueue() with 261 characters, we get an ArgumentOutOfRangeException. This still is true.

FYI we have code creating SB paths dynamically. In an effort to get a stable architecture, we need to know this exact limit.

References:

Using: Microsoft.ServiceBus.dll v2.2

http://msdn.microsoft.com/en-us/library/azure/hh767287.aspx

Regards,

Marco

Hi - I need help on using Azure event hubs in the following scenario. I think consumer groups might be the right option for this scenario, but I was not able to find a concrete example online.

Here is the rough description of the problem and the proposed solution using the event hubs (I am not sure if this is the optimal solution. Will appreciate your feedback)

I have multiple event-sources that generate a lot of event data (telemetry data from sensors) which needs to be saved to our database and some analysis like running average, min-max should be performed in parallel.

The sender can only send data to a single endpoint, but the event-hub should make this data available to both the data handlers.

I am thinking about using two consumer groups, first one will be a cluster of worker role instances that take care of saving the data to our key-value store and the second consumer group will be an analysis engine (likely to go with Azure Stream Analysis).

Firstly, how do I setup the consumer groups and is there something that I need to do on the sender/receiver side such that copies of events appear on all consumer groups?

I did read many examples online, but they either use client.GetDefaultConsumerGroup(); and/or have all partitions processed by multiple instances of a same worker role.

For my scenario, when a event is triggered, it needs to be processed by two different worker roles in parallel (one that saves the data and second one that does some analysis)

Thank You!

- Amrish

public class ClassTests
{
    [Fact]
    public void test()
    {
        var inputString = "{}";
        var inputBytes = Encoding.UTF8.GetBytes(inputString);
        var payloadJsonStream = new MemoryStream(inputBytes);
        var brokeredMessage = new BrokeredMessage(payloadJsonStream, true);
        brokeredMessage.ContentType = "application/json";

        var outputString = brokeredMessage.GetBody<String>();

        Assert.Equal(inputString, outputString);
    }
}

The primary problem here is that BrokeredMessage.GetBody<>() attempts to deserialize XML even if the content type is set to anything other than text/plain.  In this example, I set the content type to "application/json" and try to pull a string out of the body but an exception is thrown complaining that the format of the message is not correct because it isn't valid XML.

I can work around this by calling BrokeredMessage.GetBody<Stream>() and the manually dealing with a stream of bytes, but this is not ideal.  While I can appreciate not supporting a wide array of deserializers besides the XML deserializer, it seems unreasonable to treat String deserialization the same as object deserialization, especially in the face of an unknown content type.

In particular, this is problematic because as an application developer my choices are to set an incorrect contentType on the message so that the SDK is easier to work with (text/plain and GetBody<String>) or do more work on the consumer and producer ends and deal with everything as streams.

Ideally, I would like to be able to provide custom deserializers function, not deriving from XmlObjectSerializer.  Then I can call GetBody<String>(MyCustomDeserializerMethod) and get out something more reasonable.  At the least, I would like a way to tell ServiceBus to not assume everything is XML serialized.

public class EventProcessor : IEventProcessor
{
    private readonly IEventService _eventService;
    private readonly ILogger _logger;
    private readonly Lazy<RetryPolicy> _retryPolicy;
    private readonly IConfigurationProvider _configurationProvider;

    public EventProcessor()
    {
        try
        {
            _eventService = ContainerProvider.Current.Container.Resolve<IEventService>();
            _logger = ContainerProvider.Current.Container.Resolve<ILogger>();
            _configurationProvider =     ContainerProvider.Current.Container.Resolve<IConfigurationProvider>();

        }
        catch (Exception exception)
        {
            _logger.WriteError(string.Format("Error occured when intializing EventProcessor: '{0}'", exception));
        }
    }

    public Task OpenAsync(PartitionContext context)
    {
        return Task.FromResult(0);
    }

    public async Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> events)
    {
        var eventsList = events.ToList();
        EventData lastEvent = null;
        foreach (var eventData in eventsList)
        {
            _logger.WriteVerbose(string.Format("Consumming {0} events...", eventsList.Count()));
            _eventService.ProcessEvent(eventData);
            lastEvent = eventData;
        }

        if (lastEvent != null)
        {
            await AzureServiceBusRetryPolicy.ExecuteAsync(async () => await context.CheckpointAsync(lastEvent));
        }
    }

    public async Task CloseAsync(PartitionContext context, CloseReason reason)
    {
        _logger.WriteInfo("EventHub processor was closed for this reason: " + reason);

        if (reason == CloseReason.Shutdown)
        {
            await AzureServiceBusRetryPolicy.ExecuteAsync(async () => await context.CheckpointAsync());
        }

    }


}

Hi,

I am using Service Bus for Windows Server 1.1. I have a couple of applications using queues and topics etc, one is a server side app, the other is meant to run on the client.

Currently, SB is using its self sign certificate and as such I have to use PowerShell to export the certificate (pfx and crl files) in order to install on the client. We don't really want to have to install a certificate on each client that will ultimately be talking to our hosted service bus.

We already have a SSL certificate (issued by GoDaddy). Can I tell Service Bus (which is installed on the server containing the SSL certificate) to use our SSL certificate instead of the auto generated one it installed with?

If I did this, would this mean I no longer have to install anything (certificate wise) on the clients that want to subscribe to the topic?

By the way, I am not using Domain Auth as the client machines will not be members of my server domain. I am using SAS instead.

Thanks for any help.

Graham

Hi,

I am using Service Bus for Windows Server 1.1. I have a couple of applications using queues and topics etc, one is a server side app, the other is meant to run on the client.

Currently, SB is using its self sign certificate and as such I have to use PowerShell to export the certificate (pfx and crl files) in order to install on the client. We don't really want to have to install a certificate on each client that will ultimately be talking to our hosted service bus.

We already have a SSL certificate (issued by GoDaddy). Can I tell Service Bus (which is installed on the server containing the SSL certificate) to use our SSL certificate instead of the auto generated one it installed with?

If I did this, would this mean I no longer have to install anything (certificate wise) on the clients that want to subscribe to the topic?

By the way, I am not using Domain Auth as the client machines will not be members of my server domain. I am using SAS instead.

Thanks for any help.

Graham

Hi there,

We are using Windows Server Service Bus 1.1 to distribute some sensitive information and would like to implement some sort of auditing around the bus. As far as we understand, Service bus does not provide any form of auditing out of the box and does not generate any logs (would be happy if someone corrects me here)

One way of auditing the payload is using "ForwardTo" auditing subscription to write to an audit topic but this does not tell us "who" sent the message. We can harness custom message properties but then it means we rely on the client to provide us this information. We can use a separate WCF logging service but then i would have to worry about distributed transactions.

Is this audit information available in any form am not aware of, any events that we could possibly intercept?

Any pointers will be appreciated. Thanks in advance.

http://btsbee.wordpress.com/