Hello,
There's a few scattered posts and topics around the web on this, so I thought I'd consolidate the information in one place. There's no suitable Microsoft Connect product issues page, so I'm unable to post this somewhere else - but hopefully the below can save someone else the weeks of on/off troubleshooting I had to endure and time with .NET Reflector pulling apart the Windows Fabric libraries.
Service Bus 1.1 Does Not Work with Any Certificates that Use SHA512 Thumbprint Algorithm.
When a user attempts to configure a service bus issued with a certificate that is generated from a Windows 2012 series CA, the default settings are to thumbprint the generated certificate with a SHA512 hash. This is a problem because Windows Server's implementation of TLS 1.2 does not have the SHA512 algorithm in it's ciphers suite, and this TLS 1.2 by default is the mechanism used to secure the transport for the Windows Fabric.
How the Problem Manifests
Users who have this issue will find that on a single-node configuration, the Service Bus farm will start up without issue. However, adding a second node to a Windows Fabric ring/SB farm will fail. Subsequent attempts to restart any node services with more than one server in the 'ring' for the Windows Fabric will result in the Message Broker service hanging. You can identify that the Windows Fabric is not operating by checking 'netstat -a -n' and looking for TCP Port 9003 in the listening state. You'll find that on a single-node setup, it'll be present and listening on 127.0.0.1, but attempting to add a second or third node will result in a hung service.
Fixing the Problem
If you have the means to generate a certificate from a CA that uses the SHA256 thumbprint mechanism, great, go nuts and re-issue your certificate. If you are using a Windows CA though, you'll not be able to change your thumbprint selection without reinstalling the CA role. The easiest way to change this for issuing a certificate is:
- Open RegEdit and navigate to: HKLM\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration\<CA Common Name>\CSP
- Change the REG_SZ string value 'CNGHashAlgorithm' to the value SHA256.
- Stop and Start the Certificate Service.
- Re-issue your service bus farm certificate.
- Install new certificate on SB Farm nodes
- Stop all instances of the service bus services.
- Use Set-SBCertificate to change the certificate to use the new thumbprint value.
- Start-SBFarm
At step 6 you'll need to stop all instances of all farm services - but use Task Manager to kill any hung 'Service Broker' instances. This is not an officially supported means of changing your CNG, so as always, back up your CA database first and revert the registry to it's original values after you've issued the specific certificate.
How Issue Was Found
Hex-Rays, IDA Pro, Reflector and a lot of cans of red-bull. Working backwards from the Service Bus and Windows Fabric event logs. The logs indicated that something was having issues connecting to Port 9003, so I worked backwards through the service code trying to find what was supposed to start listening on that port, and why it wasn't in a multi-node setup. For a single-node setup there's only one server in the fabric ring, so it appears there's some optimisation that occurs to avoid going down to the network stack and hitting this issue.
Other Areas Affected
Lync 2013 uses Windows Fabric. I'd had horrible issues setting this one up at home too, and it turned out to be the same issue. When I'd found the solution it looks like Lync users may have stumbled onto the same problem - though people suffering from the Service Bus issue by and large haven't found solutions yet.