I have recently upgraded my service bus library to version v3.0.1 and noticed a strange behavior.
What I did was I performed load test against my web app (which will raise new event to the bus internally) and recycled the application pool in the middle of the test. Almost 100% of the time the service bus would stuck at somewhere for a very long period of time.
Because of overlapped recycle, IIS would spawn a new w3wp process to serve new requests and the existing w3wp process remains in the background until it finishes serving existing requests. The threads in existing w3wp process would stuck and strange thing is even the new w3wp process is also showing the same behavior.
Since every thread is stuck so IIS had to keep spawning threads to serve new requests. The thread count would increase until the point IIS no longer able to take it where it would start rejecting incoming requests.
I have upgraded to v3.0.9, but the problem persists. I have then downgraded to v2.6.5 and seems like the issue is not there.
Do we have any changes done in v3 that could potentially cause this issue?
In case you want to replicate the issue, here are the steps:
- Create a web api project and use service bus v3.0.X. Our code raises event to a topic but potentially queue is also affected.
- Perform load test against the web api.
- After running for 1 - 2 minutes, recycle app pool and in task manager you would see the w3wp thread count start increasing.