Friday, May 5, 2006

TCP/IP

TCP/IPYou are proud of your well tested web application, and it runs just fine on your own development machine. However, when you deploy it on a Windows XP staging server and perform load testing, you start getting, from time to time, this error message: "Server access Error: Address already in use". This does not happen in the browser, but on the server. Yes, your web application connects to a server, say making REST API call through HTTP. So that must be the source of the problem: sometimes, the connection from your application to the server just fails.

Not all the connections fail, say only 1 out of 10 fails. And this only starts after the server has about 3000 requests. Yes, you can't see a single error for the first 3000 requests. This is when you start thinking about cosmic rays as a possible explanation.

Cosmic rays are not the answer, the way TCP/IP is implemented in Windows is. When an application connects to an HTTP server, a connection is made to port 80 on the server, but because this is TCP/IP, a port is also open on the machine that runs the application. That port number is assigned by the operating system and is by default in the 1024-5000 range on Windows. With TCP/IP, after you close a connection, the local port won't be reused for a few minutes.

If your application is under load, you can easily run into the situation where all the ports between 1024 and 5000 have been used and closed already, which leaves no port available to establish new connections. Hence the errors.

One simple solution is to tweak Windows and make this range larger. Changing it to 5000-65534 is very likely to take care of the issue. You can do this by changing the MaxUserPort parameter in the Windows registry. This also gives an additional reason to do connection pooling (and some will say to deploy your web application on a server that runs UNIX instead of Windows).

No comments:

Post a Comment