Frequent timeout exceptions from ServiceBus queues

I've got a .NET application which uses ServiceBus queues and it is experiencing frequent timeout exceptions from ServiceBus-related classes. The app works fine on few developer machines but I'm seeing these exceptions on the server where this is supposed to run. The exceptions are:

A timeout has occurred during the operation. 2b122421-8715-403d-97fc-3d1261581dbd_G24
System.ServiceModel.FaultException`1 was unhandled by user code
  HResult=-2146233087
  Message=A timeout has occurred during the operation. 2b122421-8715-403d-97fc-3d1261581dbd_G24
  Source=Microsoft.ServiceBus
  Action=http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher/fault
  StackTrace:
    Server stack trace: 
       at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.ThrowIfFaultMessage(Message wcfMessage)
       at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.HandleMessageReceived(IAsyncResult result)
    Exception rethrown at [0]: 
       at Microsoft.ServiceBus.Common.ExceptionDispatcher.Throw(Exception exception)
       at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
       at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.DuplexCorrelationAsyncResult.End(IAsyncResult result)
       at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)
       at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__4(RequestAsyncResult thisPtr, IAsyncResult r)
       at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
  InnerException: 

Now, because this works on some machines I suspect that the problem is not likely in code or the ServiceBus itself but in the connectivity somehow. Yet the internet connection on the server should be fine.

I tried to force the ConnectivityMode to both Http and Tcp but that didn't help - I think it is using net.tcp when on AutoDetect.

I managed to turn on WCF activity and message logging to get some traces on this. That does produce some log but I haven't found any errors in those logs. Is there some other logs I could look into to get some insight on this?