ClusterFunk

In my first post on this subject we look at how network routing effects the configuration of publishing of web applications. In this post I consider how load balancing effects applications and discuss some of the problems

Here is an example nTier platform. This is a common approach to web application publication using layered security to minimise the effects of compromise of any one area of the solution. 

In this example an ISA Server farm acting as a reverse proxy hides the internal infrastructure behind a single IP address. If you need to publishing multiple HTTPS domains you will need an IP for each (SSL) site. In this example each site is resolving to an Windows NLB VIP address hosted by the ISA Farm. 

The ISA Servers also act as the perimeter firewall with an External and Internal NIC configuration for true network segmentation. ISA Server 2006 provides basic load balancing functionality, the solution uses this capability to publish the web servers. ISA has a wizard to create a Web Farm. 

Web Farms

A collection of servers is organised into a Web Farm. ISA has two techniques for balancing the traffic to the servers in the Web Farm. Both techniques relies on round robin to balance requests. As such its a very basic mechanism and doesn’t distribute load just requests against the web servers. 

1) Session Affinity
ISA inserts a cookie into the HTTP payload creating a session id for each client requests. All subsequent request from the host includes the session cookie which ISA uses to direct the client to the same web server. This technique relies on a browser that is HTTP v1.1 if it doesn’t or cookies are disabled then ISA cannot use this method. 

2) IP Based
ISA uses the Client IP to directs the request to a specific web server. This technique is problematic if your clients are behind multiple proxies 

Session State

Stateless

Web applications are either statefull or stateless. HTTP is by definition stateless. A client makes a request against a server. A TCP/IP connection is created, the server responds the connection is closed and there is no persistent connection between the client and server. This is fine for static content such a readying this blog.

Stateless Application

Request

Response

Transaction is completed and after a timeout period the connection is closed.

Client Sends another Request and a new connection is created…..

Statefull Application (in load balanced environment)

Many modern web based applications are statefull, that is they need to maintain a logical link between client and specific server. e.g. shopping based activity were you want to pay for your goods. Frequently (due to PCI compliance) you are connected to a 3rd party to process credit card payments. Once this transaction is completed you are then returned to the original site for order confirmation. Without session state been maintained the application server processing your purchase may or may not be the one that continues the process.      

This example focuses on the Application state at the App Server Tier but the issue of maintaining state has to be addressed at each point that there is a load balancing decision between the client and the application tier. 

Typical statefull application in stateless configuration.

Request

Requests back and forth between client and server until a period of inactivity at the client results in a timeout of the TCP connection.

Connection Time Out

Next Request is directed to a different application server (AP1) and the request is unable to be processed resulting in an error on the clients system.

Using infrastructure to maintain session state.

We can address this in a number of ways:

Hardware load balancing can provide session affinity based on the origination client IP or MAC address. This is successful for simple load balanced configurations. However it is problematic for multiple tier load balanced configurations as the second tier always receives requests from a limited number of hosts at the first tier which can easily result in a uneven loading across application servers. 

Application Layer load balancing

Using Cookies for session affinity
The load balancing solution inserts a cookie into the request header, which is used to identify specific user and maintain the relationship between a client and server. This is ideal for the multiple tier load balanced environment.

It does impose the requirement that the client supports HTTP v1.1 so its not going to work for most mobile users or users that disable cookies.  

Protocol Inspection
One such method uses http session id (Session Identification URI to give it its proper name) to maintain session state between client and application server. Alternatively you can insert into the http header a value of your own choosing on which to make load balancing / session affinity decisions.

SSL (HTTPS) and Load Balancing.

When you deal with SSL traffic, it is encrypted between the client and the destination web server. It removes the opportunity to inspect the content of the request / response. Particularly useful then is the ability to load balance based on SSL session ID. Most solutions (ZXTMs and Cisco ACE for example) allow you to do this.  

You could terminate the SSL encryption on the edge of your environment and pass through HTTP traffic internally re-encrypting the traffic as it leaves you environment. Alternatively you could decrypt and then re-encrypt with internal and External SSL certs. You need to consider the load that the SSL offload will have on your solution and also consider the needs of you organisation. Hardware based solutions such as the Cisco ACE modules are licensed based on a number of SSL transactions in combination with network I/O so you will need consider the costs associated with the solution you choose. 

Development led options

There are a number of ways that this can be addressed by the developers of the application. 

Record Session State in Cookie/s

Using this method it doesn’t matter which server receives the request as the session state is recorded in the cookie. This is limited by the number and size (payload) of cookies that can be added to the http header. It requires the application to be developed to accommodate this approach so needs to be designed into the web application. The cookies can add significantly to the amount of data that is transmitted and also increase the processing overhead on the web servers. 

Record Session state in Database

The session state can be written to a database by the application server. The session id is then used to retrieve the session sate. A suitable database tier is required and obviously this has to be designed into the application from the start.

Other Considerations

The return path is equally susceptible to problems relating to state as highlighted below. This needs to be accommodated  in your application / infrastructure design.

Response

Which is the best method to adopt?

Obviously you need to consider the platform, the applications and the infrastructure. If you are managing a simple web application such as a basic MOSS deployment with a pair of load balanced web servers your requirements are considerably different to the delivery DRM protected video assets that need to be restricted based on location of the user making the request. 

If you are managing a complex multi-tier load balanced environment you are likely to be maintaining a highly dynamic set of web based applications. The business will be constantly responding to the environment in which it operates which is likely to include frequent development of the applications, changes to the environment and often with very aggressive delivery deadline.

In my experience the key to successfully managing such environments is to be able to respond quickly and utilise solutions that are versatile. 

If you have access to product such as the excellent Zeus ZXTMs you have the opportunity to inspect and manipulate the client request and server responses directly via traffic script. Its possible to make decision based on a huge number of parameters, providing extremely granular control of the data flowing through your network, manage service levels and respond to requests differently depending on the load on the platform.

Combined with the Load balancing capabilities where decisions can be tailored based on any number of factors such as response times, time of day, requested resource or even geographic location of originating request. You have the tools to be able to operate effectively in such fast moving dynamic environments. This is why I am such a fan of the ZXTMs. They are a software solution that can be deployed very rapidly, tailored very easily by system admin and developers without input from networks. They put the control of the application function into the hands of the guys that are interested in it (no offence to the network guys out there ) and they don’t cost more money if you want to increase the load that they handle.