Quantcast
Channel: Service Bus forum
Viewing all 1916 articles
Browse latest View live
↧

Azure Event Hubs - Error Message when passing it as parameter "Overloaded method value json with alternatives"

July 12, 2019, 5:19 pm
$
0
0

We are sending a file path as message through azure event hub and when passing received messages to spark.read.json as parameter getting an error  message"overloaded method value json with alternatives:"

========================================

import org.apache.spark.sql.types._
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions._
import org.apache.spark.eventhubs._
import org.apache.spark.eventhubs.ConnectionStringBuilder
import org.apache.spark.sql.types.{IntegerType, StringType, StructField, StructType}
import org.apache.spark.sql.DataFrame


val connectionString = ConnectionStringBuilder("Endpoint=sb://tst.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=dfgdfgdfOcXdfgdfghT3dsfgdfgsdRoIM=").setEventHubName("tsthub").build

val customEventhubParameters =EventHubsConf(connectionString).setMaxEventsPerTrigger(5)

val incomingStream = spark.readStream.format("eventhubs").options(customEventhubParameters.toMap).load()
val rawData = incomingStream.selectExpr("cast(body as string) as json")

val filepath = incomingStream.withColumn("Offset", $"offset".cast(LongType))
      .withColumn("Time", $"enqueuedTime".cast(TimestampType))
      .withColumn("Timestamp", $"enqueuedTime".cast(LongType))
      .withColumn("Body", $"body".cast("String"))
      .select("Body")
filepath.printSchema
//filepath="\\usimachine\wklyreport.json"
val CHG_DF = spark.read.json(filepath)
CHG_DF.show()

val query = filepath.writeStream.format("console").start()
query.awaitTermination() 

Regards

Suresh SM

 

↧
Search

Getting a "Microsoft.Azure.ServiceBus.ServiceBusCommunicationException" when writing a simple test

February 13, 2019, 8:34 am
$
0
0

Hello,

I am trying to reproduce the simple example explained here about the use of Service Bus: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues

However, I am getting the following exception

Exception thrown: 'Microsoft.Azure.ServiceBus.ServiceBusCommunicationException' in System.Private.CoreLib.dll

with the message:

Exception: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond ErrorCode: TimedOut
 Stack Trace:    at Microsoft.Azure.ServiceBus.Core.MessageSender.OnSendAsync(IList`1 messageList) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageSender.cs:line 562
   at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\RetryPolicy.cs:line 85
   at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\RetryPolicy.cs:line 107
   at Microsoft.Azure.ServiceBus.Core.MessageSender.SendAsync(IList`1 messageList) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageSender.cs:line 252
   at ServiceBusHelloWorld.Program.SendMessagesAsync(Int32 numberOfMessagesToSend) in C:\_dev\ServiceBusHelloWorld\ServiceBusHelloWorld\ServiceBusHelloWorld\Program.cs:line 58

in this line (SendMessagesAsync() method):

// Send the message to the queue
await queueClient.SendAsync(message);


Could it be an non-open ports issue? If so, what port should be used to handle the ServiceBus' response?

↧
↧

Newbie Question - sending VM TAG Changes to Service Bus

July 18, 2019, 7:50 am
$
0
0

Hi All - new to this forum. Really appreciate any assistance. I have spent way too many hours recently trying to do what I thought should be pretty simple.

I want to subscribe to an event stream that will near-realtime (programatically) notify me of add/remove/change of any TAGS on any VM in my resource group.

If someone could point me in the right direction, that would be amazing. I have tried following various 'guides' but with no success. Appreciate your assistance!

Rossi

↧

Azure Service Bus Queue Portability

July 22, 2019, 11:45 am
$
0
0
Is there a way to communicate with the Azure queue endpoint using AMQP 1.0 that does not require using the Microsoft.Azure.ServiceBus.dll? I am sending CFX messages successfully to a RabbitMQ broker using AMQP 1.0 and want to transition to using an Azure Service Bus queue but still want to use the CFX endpoint instead of Azure. Thanks
↧

sending azure monitoring alert to event grid or other event service

July 23, 2019, 6:47 pm
$
0
0

I want sending azure monitoring alert to event grid or other event service to use by other on Prem application. I have tried sending alert to action group as webhook which I use the event bus end point URL as http url, but even tough the alert trigger I don't see and message on event Bus say just zero. not sure I missed or other easy method available to send the alert to event queue?



Thanks

 

raindrop18


↧
↧

Hybrid Connection Manager UI Setup

June 25, 2017, 12:21 pm
$
0
0

Hi,

I logged the following with @AzureSupport Twitter:

"I have just downloaded the hybrid connection manager ui. I am stuck in an endless loop of 'Configure another Hybrid Connection' > log in > Nothing is listed > 'Configure another Hybrid Connection' > repeat ... I have tried on Windows 10 and Windows Server 2016. All I want to do is set up a hybrid connection relay"

The conclusion the agent came to was that Server 2016 is not supported but this doesn't sound right to me. Could I be missing some pre-requisites on the server?

Thanks in advance

Alex

↧

read eventbus queue message

July 27, 2019, 9:57 am
$
0
0
 I want to check/ test the actual message on service bus, is there any curl or cli to read the actual message. I want to see before other system read the queue. 

raindrop18

↧

How to receive pending message in Azure Service Bus queue?

July 30, 2019, 1:34 am
$
0
0

Hi All,

I am working on receiving a message from Azure Service Bus queue.
I have created a queue (SampleQueue) and I am able to send the message successfull in that queue via POSTMAN using a SAS token which I generate with my Java program.
I am also getting a 201 created status after hitting my service bus queue api url.

I want to receive the message pending in my Service bus queue, how can I get those messages ( java code would be preferable)?
I went through some links about receiving message (https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-java-how-to-use-queues), but this does not contain information about how I can receive and view those messages. Is there any way we can receive pending message on service bus queue?
Would be grateful for any help here.

Thanks,
Rudra

↧
Search

Eventhub sends data out of order

August 2, 2019, 12:23 am
$
0
0

When we use eventhubs we have a serious problem that events are sent out of order,  1 batch goes before another batch. This happens mainly if there are 2 paralel ampq connections. Most likely the ampq sends data and cofirms it's OK but it's not yet propagated to the partitions. (Disclaimer the partition ID is set correctly). The multiple connections can arise in following situations:

  1. Send Batch,  Disconnect/Fast reconnect/ Send batch
  2. I have a stateless service that is replicated  S[1,2,3,..]. I have a client C that invokes something on S. This operations has a side effect that it sends something to eventhub.  If C sends something to S(1),  waits for confirm,  then sends something to S(2) quickly (despite waiting for confirm of all operations), different ampq connections are used for eventhub (as each S replica has its own AMPQ connection) and messages come out of order. The C invokes S must be done quickly, some load balancer randomly selects S(1,2,3,...). We are using linkerd and grpc.

I replicated this issue in golang and python. I would attach a test case but I am unable to because of your forum policy. This is a serious problem with event streaming which I hope will be fixed as soon as possible.  (Also I know this is Service Bus forum but I couldnt find eventhub topic)



↧
↧

Hybrid Connection Manager UI Missing

August 3, 2019, 4:41 pm
$
0
0

I recently installed the hybridconnectionmanager on a new SQL server (2017 Standard on Windows DC 2016).   I'm trying to setup a Hybrid connection and it appears the UI for the connection manager no longer exists.

Is this true?

↧

can we externally subscribe to azure service bus topic externally and read/receive the messages published to that topic?

August 4, 2019, 9:45 pm
$
0
0

Hi Microsoft azure team,

i am working on a project which involves usage of microsoft azure cloud services.

1.i have connected a raspberry pi + DHT11 sensor to the cloud and sent the sensor data to the iot hub and from there i routed the messages to service bus topic. now i want to subscribe to that topic from another device (esp32) exterally and read/receive those messages and take further actions on that device like lighting up an led based on the messages.

i have searched a lot for the same but couldn't find any content that would assist me to achieve the issue. can you please help me out?

the above mentioned is the specific issue i am facing...i would like to give details about my project and can you please suggest me ways in which i can achieve that?

main objective: i want to measure the room temperature of my room and then send the data to the cloud (here iot hub) next i want to see if the temperature is greater than say some 25 degrees then i would want to turn on an led on my device i.e in simple words switching on/off an led based on some condition. can you please give a overview on how to achieve this ?

as of now i am trying to do it this way..

(device + sensor) -> (send data to iot hub) -> (route it to service bus topic) -> (write a azure function...in which it evaluates the condition and sends 'yes' or "1" it temperature is greater than 25 and 'no' "0" if otherwise) -> (send to another service bus topic) -> (using an external device i want to connect/subscribe to the azure service bus topic and read messages and perform an action like turning on an led if the message if "yes" "1" and turning off an led if the message is "no/"0").

this is algorithm i am using to achieve the result ? is it possible to achieve the result in the above specified way ?

please do suggest any other method to achieve the result.

Thanks !

↧

Exceptions on Renaming Queue

August 6, 2015, 2:33 pm
$
0
0

If I attempt to rename a queue to a name that I have used previously, but no longer exists it throws a 500 exception.

The remote server returned an error: (500) Internal Server Error. SubCode=50000. Internal  Server Error. TrackingId:a8a4c20b-c407-4b5a-8ddf-6e6db1698361_G41,TimeStamp:8/6/2015 9:27:17 PM

How to recreate is Create a Queue named "QueueA" and another called "QueueB", delete "QueueB" and attempt to rename "QueueA" to "QueueB".

↧

503 Error on service bus message send from Logic Apps.

August 9, 2019, 2:12 am
$
0
0

RunID: 08586362653094527714745890400CU34
Subscription: 0d440ec8-b273-4af9-b797-b209e72c0399

Raw outputs

{
    "statusCode": 503,
    "headers": {
        "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
        "X-Content-Type-Options": "nosniff",
        "X-Frame-Options": "DENY",
        "Timing-Allow-Origin": "*",
        "x-ms-apihub-cached-response": "false",
        "Cache-Control": "no-store",
        "Date": "Fri, 09 Aug 2019 09:07:26 GMT",
        "Content-Length": "266",
        "Content-Type": "application/json"
    },
    "body": {
        "error": {
            "code": 503,
            "source": "logic-apis-westeurope.azure-apim.net",
            "clientRequestId": "94014b57-f71b-4695-b595-e41b5c26e5db",
            "message": "The response is not in a JSON format.",
            "innerError": "The service is unavailable."
        }
    }
}

↧
↧

LOCKUNTIL?

August 9, 2019, 8:00 am
$
0
0

Hi,

I am trying to develop a real time system which utilizes a Service Bus. I have a Trigger on a Service Bus Queue which takes each message and fires off an Azure Function. Please see my attachment. My question is that they processing between messages is much slower than what I need. What does the LockUntil do and how can I configure this? OR is this a property of the Trigger which fired with each message?

Thanks so much!
Mike

Mike Kiser

↧

Azure Service Bus Premium - Geo-Recovery

August 12, 2019, 11:02 am
$
0
0

I have a Premium Service Bus namespace in one region, and have created another one in another region, as a secondary. I have enabled Geo-Recovery on the primary, and have configured the pairing with the secondary. I ran a test to continuously send messages to a Topic, and I have a receiving application subscribed to it. The sender will send "Sending message: Message {number in sequence}" and receiver will display "Received message: SequenceNumber:{SB assigned sequence number} Body:Message {number in sequence}". However, when I tried to initiate a Failover to the secondary through the Portal, I noticed that though the sender kept on sending messages, the receiver dropped some messages while completing the failover. Please see below:

Logs from Sender:

Sending message: Message 244
Sending message: Message 245
Sending message: Message 246
Sending message: Message 247
Sending message: Message 248
Sending message: Message 249
Sending message: Message 250
Sending message: Message 251
Sending message: Message 252
Sending message: Message 253
Sending message: Message 254
Sending message: Message 255
Sending message: Message 256
Sending message: Message 257
Sending message: Message 258
Sending message: Message 259
Sending message: Message 260

Logs from Receiver:

Received message: SequenceNumber:255 Body:Message 244
Received message: SequenceNumber:256 Body:Message 245
Received message: SequenceNumber:257 Body:Message 246
Received message: SequenceNumber:258 Body:Message 247
Message handler encountered an exception Microsoft.Azure.ServiceBus.UnauthorizedException: Connection rejected after GeoDRFailOver. TrackingId:7bb0b78d-2bf5-4807-8bcb-c831b00c6692, SystemTracker:AmqpGatewayProvider, Timestamp:2019-08-12T17:42:38
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<OnReceiveAsync>d__86.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<<ReceiveAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.ServiceBus.RetryPolicy.<RunOperation>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.ServiceBus.RetryPolicy.<RunOperation>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<ReceiveAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<ReceiveAsync>d__62.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.ServiceBus.MessageReceivePump.<MessagePumpTaskAsync>d__11.MoveNext().
Exception context for troubleshooting:
- Endpoint: rjpremium.servicebus.windows.net
- Entity Path: topic1/Subscriptions/sub1
- Executing Action: Receive
Received message: SequenceNumber:1 Body:Message 254
Received message: SequenceNumber:2 Body:Message 255
Received message: SequenceNumber:3 Body:Message 256
Received message: SequenceNumber:4 Body:Message 257
Received message: SequenceNumber:5 Body:Message 258
Received message: SequenceNumber:6 Body:Message 259
Received message: SequenceNumber:7 Body:Message 260

The messages between 247 and 254 are dropped. Though the sender sent all those, the receiver never received those messages. If I enable Geo-Recovery, ae these messages should also be received by the receiver?

↧
Search

How to log messages?

August 19, 2019, 10:59 am
$
0
0
I'm trying to figure out how to get a log of messages in the ServiceBus in log analytics, something that would list message events and some info about them. I cannot figure out how to do this from any of the documentation; it just mentions how to get Azure metrics and diagnostics logs.
↧

Service Bus Queue Replication

August 19, 2019, 7:22 pm
$
0
0

Hello

What I would like to do is use a Queue which is part of our Production Application to assist me in testing a new Application. In other words I would like to have this one queue, lets call it the Blue Queue, replicate (duplicate) the same messages into another Queue, let's call it the Green Queue, where I can use the same message for testing a new System. Does anyone know if/how this can be done (hopefully easily LOL)?

Thanks!
Mike Kiser

Mike Kiser

↧
↧

Cant access the data from Service bus Queue from Tibco using Apache qpid library when events only coming from stream analytics

August 21, 2019, 2:10 am
$
0
0

Hi,

From Last week we are facing issues to access data coming from stream analytics to Service bus Queue.

The adapter we use at our side to read from service bus queue is a JMS application. It use Apache qpid library which connects via AMQP 1.0 protocol to service bus queue.The adapter task is to read from service bus queue, enrich the data and publish the result on IBM MQ for further internal processing.We use Tibco BusinessWorks as integration tool which supports the JMS API.

It was working perfectly fine. On last Thursday we restarted stream analytics, after that it stopped work.

Tried using below steps:

1: Service bus explorer – Tibco able to read message without error

2: Test application using C# - Tibco able to read message without error

3: Events from Stream analytics - Tibco able to read with error

Can someone help me on this.

Thanks,

Snigdha

↧

Payload Format Recommendations

August 21, 2019, 4:16 am
$
0
0

In Microsoft's documentation about Messages, payloads, and serialization it is recommended to use JSON and Apache Avro as payload formats for structured data. What is the reasoning behind this recommendation? Both these formats use dynamic typing and are generally slower in marshalling/unmarshalling compared to statically typed payload formats such as Thrift or protobuf.

Associated GitHub Issue: Azure/azure-service-bus/issues/297

Sorry for no actual links, my account has not been verified yet.

↧

Reliable Communication from On-Prem .NET services to Azure Service Bus

June 11, 2019, 1:38 pm
$
0
0

Hi,

We have a number of Azure WebJobs that integrate with on-prem apps using Service Bus in a store-and-forward scenario. The overall architecture is that we receive HTTP messages that come in from the Internet through API Management, straight to Service Bus Queues, and then the WebJobs forward them to .NET Core APIs running on prem (again, going through API Management). That's great, because Azure Service Bus is reliable by its very nature; if our WebJob can't talk to the on-prem API for whatever reason, we just release the peek lock, and try again in 10 minutes until it succeeds. 

My question is...what is the prescribed guidance for reliable messaging going the other way i.e. from on-prem to Service Bus? I'm considering things like "the network is down", or Service Bus itself is offline (yes, it can happen). And yes, I realize I could use MSMQ and then a Windows Service to call Service Bus, but is there a better way suggested way for this "first-mile" part of an on-prem to Azure integration? 

Thanks!

PBR

↧
Viewing all 1916 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>