This post relates to RHEL 5.x

After you install the OS

Set up user accounts

useradd username
passwd password

I create an account called remote that I can us to login via SSH.

Network

Set up networks to provide access to internet

For example here is my VM config ( /etc/sysconfig/network-scripts/ifcfg-eth0 file for eth0 network interface:)  use nano or vi to edit this as required. 

DEVICE=eth0
BOOTPROTO=static
BROADCAST=192.168.1.255
HWADDR=00:0F:EA:91:04:07
IPADDR=192.168.1.111
NETMASK=255.255.255.0
NETWORK=192.168.1.0
ONBOOT=yes
TYPE=Ethernet

Static routes:

You would put your default gateway in “/etc/sysconfig/network” using “GATEWAY=x.x.x.x”

You would typically add static routes into a startup file the system will read on boot e.g.

/etc/sysconfig/network-scripts/route-eth0

#Route Description

10.8.0.0/24 via 10.0.0.1 dev eth0

Once configured you can get the system to re-read the files as follows:

“service network restart” (redhat specific)

Or

“/etc/init.d/network restart” (Works with just about any Unix box)

RHEL 5 Registration

rhn_register

Follow onscreen dialogue to register RHEL – You obviously need to have purchased a subscription.

Install Java

yum install java

its that easy

General OS Update

To update RHEL simply type Yum update

ZXTM pre Install

This section assumes that you are doing the install remotely from Windows machine. You must have port 22 access through any firewalls between zxtm and remote console.

Download following utils

winscp http://winscp.net/eng/download.php

putty http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Use winscp to upload ZXTM.tar file to /home/remote

Use putty to SSH to host as remote

Then sudo to root

Untar the zxtm install file

Tar –zxf /home/remote/ZXTM_51r1_Linux-x86.tgz

This creates a folder called ZXTM_51r1_Linux-x86

Navigate to the folder it contains zinstall

Type ./zinstall

Once the install is completed you may need to configure the firewall (iptables) to allow access to the administration port.

RHEL Firewall

iptables – open ports required for administration

e.g. this command allows access to ZXTM default admin ports from 192.168.1.1

iptable –A INPUT –d 192.168.1.1 –p –tcp –m tcp –dport 9080:9090 –j ACCEPT

/etc/init.d/iptables save

List command allows inbound connections on 9090

iptables –A INPUT –p tcp –m tcp –dport 9090 –j ACCEPT

Obvious but if you are installing ZXTMs into existing platform consider the infrastructure such as firewalls and routers. You may require static routes on the ZXTM “internal” interfaces to route to you web servers. Other consideration is to make sure that the ZXTM interfaces can ping the gateway address set in the network configuration.

Useful ports to open on any firewall/s in related infrastructure. You may also need to configure iptable on zxtm hosts as well.

SNMP (161) – to infrastructure server (monitoring).

SSH (port 22) – between administrative console & infrastructure server (back up).

RADIUS (1812) – to Radius server if required

HTTP (80) – to all relevant servers via internal interface and to external to internet/network that the clients come from

HTTPS (443) – to all relevant servers via internal interface and if providing SSL passthru to external to internet/network that the clients come from   

DNS (53)

ZXTM specific OS

Areas of the OS to be familiar with from a ZXTM configuration perspective (assumes Redhat RHEL)

Zeus Install directory

/usr/local/zeus/zxtm/

Logs:

/usr/local/zeus/zxtm/log/

They include

errors – this is were log.info() output is logged

audit – Log shows security/change activity viewed via Diagnose/Audit Log

Plus any logs that you have created for virtual servers (Activity/View Logs/Virtual Server Request Logging)

Extra Files

This is were extra files live such as IP white list or html assets that you want to have served by the ZXTM in the event of loss of connectivity to or outage of web servers.

/usr/local/zeus/zxtm/conf/extra

Config Script

If you need to make changes to core ZXTM install the config script is located in

/usr/local/zeus/zxtm/

to execute type ./configure

Debugging

To view logs in real time for debugging

tail –f /usr/local/zeus/zxtm/log/errors

Hope this is useful

If like me you have spent most of you IT life working with a Windows environment you have never really had to consider the case that you write scripts in. The odd login script or batch file aside its not the mainstay of the work concentrates on GUI environments.

While working on a particular task recently I spotted this little issue with issuing a 302 redirect with traffic script.

Linux is case sensitive so login.aspx is not the same as lOgin.aspx

To this end its important that you consider case If you are using ZXTMs to terminate SSL and restrict access to resources served from a none Linux based web servers.

e.g.

This script looks for any URL containing login.aspx, signup.aspx, /thismustbessl/userdetailseform.aspx, /admin/ for the website www.website.net.

$url = http.getRawURL();
$host = http.getHeader (“host”);

if (($host == “www.website.net“) && (string.contains($url, “Login.aspx”)) || (string.contains($url, “Signup.aspx”)) ||

(string.contains($url, “/ThisMustBeSSL/userdetailseform.aspx”)) || (string.contains($url, “/admin/”))) {

http.sendResponse( “301 Moved Permanently”, “text/html”, “”, “Location: https://”.$host . $url);
}

So this script does what we need right? Wrong

If you request http://www.website.net/ThisMustBeSSL/userdetailseform.aspx the script matches all conditions and the redirect will be issued to make the site HTTPS.

However if you request http://www.website.net/thismustbessl/userdetailseform.aspx

The traffic script will not match and the page will be served as HTTP. Disaster!

To avoid this occurring a minor but crucial change is required. First do a string conversion on the url, I force the url to be lowercase but you could equally use uppercase if you wish. Then make sure that all of the strings you are comparing are also the same case (lowercase in my example). This will allows match regardless of the case that the original request is submitted as.

$url = http.getRawURL();
$host = http.getHeader (“host”);

$s = string.lowercase($url); # set $s to equal lowercase $url
$url = $s;  ~ now set $url to equal

if (($host == “www.website.net“) && (string.contains($url, “login.aspx“)) || (string.contains($url, “signup.aspx“)) ||

(string.contains($url, “/thismustbessl/userdetailseform.aspx“)) || (string.contains($url, “/admin/”))) {

http.sendResponse( “301 Moved Permanently”, “text/html”, “”, “Location: https://”.$host . $url);
}

Now everything is cool

Happy Days….

When you write traffic script make sure that you comment you scripts with plenty of debug information to facilitate testing.

Example:

$responseLocation = http.getResponseHeader(“Location”);
$body = response.get();
$responseCode = http.getResponseCode();

log.info( “Location at Zone A ZXTM is : ” . $responseLocation );
log.info( “Body is : ” . $body );

# Test for HTTP 302, Location is HTTP and body contains HTTPS
if (($responseCode == 302) && (string.startsWith($responseLocation,”http://”)) && ( string.contains($body, “a href=’https://”)) ){

# Rewrite location header
$responseLocation = string.replace($responseLocation, “http:”, “https:”);
log.info( “Location has been rewriten to : ” . $responseLocation );
http.setResponseHeader( “Location”, $responseLocation );

}

View Log

Then when you are testing your scripts SSH to the ZXTM and run the following command to view the log as events are written to it.

tail –f /usr/local/zeus/zxtm/log/errors

You can then view the log as you test to check that your script is behaving as expected.

Once complete rather than amending your script, which could potentially introduce bugs. change the logging level in the ZXTM config to not log info messages.

You simply turn this on and off as required for testing.

Happy Debugging

Zeus Knowledge hub has an article here but I thought I would elaborate a little for the benefit of the Windows Admin’s

This assumes that you have a Windows machine on which to do the conversion.

1) Install OpenSSL

Windows binary here http://www.slproweb.com/products/Win32OpenSSL.html

2) Export Private Key from .PFX

Once you have installed openSSL

Do the following:

Copy your .PFX file to local file system on the windows machine you have installed OpenSSL on -

If you follow default install navigate to c:\openssl\bin\ and enter

openssl.exe pkcs12 -in <drive\path\name.pfx> -nodes -out drive\path\name.pem>

e.g. openssl.exe pkcs12 -in C:\cert\govuk.pfx -nodes -out c:\cert\PKgovuk.pem

Type the password for the PFX file

You should see a .pem file for the private key in your folder.

3) Export Certificate

Now repeat the process but this time use following syntax to export the certificate

openssl.exe pkcs12 -in C:\cert\govuk.pfx -nokeys -out c:\cert\Certgovuk.pem

You should see this:

and a new file

4) Import into ZXTM

Open admin console and navigate to catalogue \ SSL \ Server Certs

Select Import Certificate

Give your cert a name and populate the location of your cert and private key .pem files. Click “Import Certificate”

You should now see following.

It is more than likely that you will require an intermediary Certificate to complete the key chain.

5) Intermediary Certificate

If the Cert requires an Intermediary to complete the certificate chain do the following:

Download the appropriate certificate from the issuing Certificate Authority. In this example the CA is global sign

Cert is Here:

http://www.globalsign.com/support/intermediate/domainssl_intermediate.php

VeriSign here: http://www.verisign.com/support/install2/intermediate.html

And Thawte requires login here: http://www.thawte.com/roots/index.html

Download the intermediate certificate, this is usually via copying the cert from the web page and saving in a text file. Call the file intermediate.pem

Open the Imported SSL Cert and (scroll down) select install intermediate.

Populate the box with the location of the cert and then click upload.

You should see something similar to below

6) Finished

Test by navigating to the site and verify the certificate via the browser. The Certificate should be valid and display the complete key chain.

- FIN –

Event ID: 12260      Source: Microsoft ISA Server Job Scheduler

This error may be due to a corrupted certificate or insufficient permissions to access the certificate.

Locate the reported certificate in the local machine store. If required, reinstall the certificate and its private key, or delete it.

http://www.microsoft.com/technet/support/ee/transform.aspx?ProdName=Internet+Security+and+Acceleration+Server&ProdVer=4.0.3443.594&EvtID=12260&EvtSrc=Microsoft+ISA+Server+Job+Scheduler&LCID=1033

Why?

This occurs because of the way that the certificate is applied. If you import an SSL cert from a .PFX file via the the method described below the problem occurs:

…"Place all certificates in the following store" should be selected, and below it, the Personal node should also be selected. Press "Next". Press "finish" on the next screen and your certificate has been successfully imported.

If you do this the cert is placed in the Personal node of the Current User not the Personal node of the Local Computer.

If you open the Certificates store MMC you will see the following:

This is also the case if you choose the “Automatic select option” option

The obvious thing to do to resolve this misplacement is to open the Certificate store, cut and paste the Cert from Current User into Local Computer Personal store. You open the Cert and everything looks fine the certificate chain all works and the certificate says that it has its private key. Great….

Then you go into ISA and configure your web listener and select your Cert only to find that ISA is not happy…

Notice that the Certificate is correctly installed according to the GUI, but ISA is not happy. If you look in the application log you will see an error 12660.

To resolve this is:

First delete the Certificate in the Local Computer Personal store

Then right click on the Local Computer Personal store and select Import

Navigate to your PFX file

Follow the wizard

Bingo 

Here is an example errors 

This is what you see if you allow auto placement or follow the instructions without installing via an import in the certificate store. Notice that the Private Key is correctly installed but the Certificate store is wrong.

This is what you see if you move the Cert to the correct Local Computer Personal store. The Certificate store is correct but the Private key now shows an error.

And finally how it should all look

The event id 12660 also mentions permissions. You need to check that the certificate store is accessible by the system account. In order to check this navigate to \Documents and Settings\All Users\Application Data\Microsoft\crypto check that SYSTEM has full control on this folder. 

Further info

How to install and use certificates for SSL connections in ISA Server 2006
http://support.microsoft.com/kb/840614

Troubleshooting SSL Certificates in ISA Server 2004 Publishing http://technet.microsoft.com/en-gb/library/cc302619.aspx

STOP PRESS!

The lovely people at Zeus have just announced today (23rd March) that they are making fully featured * ZXTM licences available for Dev use. This is great news. While the ZXTM was always available in a 30 day trail it was limited in its configuration capability and lacked the fully capabilities that make this such an awesome product.

* The dev licence is limited in terms of number of connections that it will handle.

“The launch of the ZXTM Development License program, is specifically designed to make ZXTM more accessible to the application development community.

ZXTM is unique in being able to provide this community with the platform to develop sophisticated applications, in physical, virtual and cloud environments.

With a ZXTM Development License, any application developer will be able to develop and test their applications using ZXTM, with no upfront costs.

In summary, the ZXTM Developer License:

- can be obtained at no cost

- support is not included

- the duration of the license is for 1 yr which is then renewable

- has restrictions including performance and clustering capabilities.

The ZXTM Development Licenses can only be used for development, testing, education, proof-of-concept and demonstration purposes and will be available to application developers, Zeus partners and customers in non-production environments.”

Check out the full details here

HP are hosting a free course for all you budding web 2.0 type out there

You will learn

• Understand website creation, maintenance fundamentals and key Microsoft Expression Web capabilities
• Use cascading style sheets (CSS) to create styles
• Work with CSS reporting, intelligent design renaming and CSS design time preferences
• Create accessibility compliant pages and validate your site
• Configure Microsoft Expression Web to support browsers and formats
• Understand how to create complex queries and how to present data in any way you need
• Work with ASP.NET 2.0 to create dynamic websites

and the best thing is…. its FREE!

I mentioned the expressions encoder in my post on smooth streaming here the encoder is part of the expression studio suite.

Link

http://h30187.www3.hp.com/courses/overview.jsp?courseId=17511&jumpid=em_di_465332_US_US_0_000_hpc_us_761251_across-bg&dimid=1001647526&dicid=taw_Mar09&mrm=1-4BVUP

So excuses over I have decided to have a good look adaptive streaming and Microsoft adaptive streaming capabilities in Windows IIS 7.

Adaptive Streaming What is it?

Traditionally streamed assets are encoded at a specific bitrate and presented along the lines of low, medium and high quality streams. It’s up to the users to request the appropriate stream for their connectivity. If they pick the wrong one they must stop the stream and try a different one.

Apart from the obvious issues with customer satisfaction, this raises issues with content providers. They have to provide multiple encodings of assets at differing bit rates, users can request inappropriate streams leading to waste of network bandwidth and capacity. “I need to server 10,000 stream but actually need capacity to server 10,500 dues to user error”. When you consider different container formats you quickly end up with a considerable number of stream to manage and associated infrastructure.

Move Networks have a patented adaptive streaming technology which:  

“divides video into segments called “streamlets” and encodes them using a single process for multiple bandwidths (dial-up, broadband, wireless) and platforms (computer, television, cell phone). A dual-pass variable bit rate encode is used on both live and on-demand streams, preparing video images with sharp, HD quality”

Microsoft invested an undesclosed sum in Move Networks in August last year after announcing a partnership with MN in March of the same year in relation to the SilverLight cross browser player.

Microsoft IIS 7 with Smooth Streaming 

So IIS 7 recently received adaptive streaming capability in the form of an IIS media extension called Smooth Streaming.

IIS Smooth Streaming

This diagram gives you a flavour for how the technology delivers a variable bit rate stream to the player.

Installing on IIS 7 with Smooth Streaming Windows Server 2008

Select Server role

Message about required dependencies

Intro to IIS7

Confirm installation components 
Selected ASP.NET and then let the defaults install

Quick points of interest:

IIS 7 doesn’t use a metabase anymore for configuration. Everything (components) is configured in XML configuration files. If you intend to configure IIS7 remotely this capability needs to be installed. In fact most IIS6 standard functions like support for authentication, serving default page or directory browsing needs to be configured/installed. This is good news from a security/attack vector perspective but a pain in the posterior when you are doing hands on lab type work   – can’t please all the people all the time.

Installing

Components installed results

and install log

Viewing Server Roles now shows

Ok Once IIS7 is installed lets install Smooth Streaming media extension

And that’s that. Don’t even need a reboot

You can download the demo material from Microsoft

OR

You can encode your you own using Microsoft Expression Encoder

Great Guide here

Install

GUI

Expression Encoder 2 SP1 provides built-in features that make it easy to create Smooth Streaming presentations simply by choosing Adaptive Streaming video and audio profiles and the IIS Smooth Streaming output media format

Playback and Adaptive streaming in practice

Screenshot shows player. Notice the graph at bottom left.

This shows the bit rate of the stream.

In this screenshot I have throttled the bandwidth to demonstrate the artifacts of the lower bit rate stream.

The quality is superb

Again I throttle the bandwidth

and then remove the restriction it resolves

Links

More details for IIS7 configuration

http://learn.iis.net/page.aspx/569/smooth-streaming-for-iis-70—managing-your-presentations/

Silverlight

http://www.microsoft.com/silverlight/

Deep Dive IIS 7 configuration

http://learn.iis.net/page.aspx/127/deep-dive-into-iis-7-configuration/

Expressions Encoder

http://expression.microsoft.com/en-us/cc507507.aspx

Here is an example nTier platform. This is a common approach to web application publication using layered security to minimise the effects of compromise of any one area of the solution. 

In this example an ISA Server farm acting as a reverse proxy hides the internal infrastructure behind a single IP address. If you need to publishing multiple HTTPS domains you will need an IP for each (SSL) site. In this example each site is resolving to an Windows NLB VIP address hosted by the ISA Farm. 

The ISA Servers also act as the perimeter firewall with an External and Internal NIC configuration for true network segmentation. ISA Server 2006 provides basic load balancing functionality, the solution uses this capability to publish the web servers. ISA has a wizard to create a Web Farm. 

Web Farms

A collection of servers is organised into a Web Farm. ISA has two techniques for balancing the traffic to the servers in the Web Farm. Both techniques relies on round robin to balance requests. As such its a very basic mechanism and doesn’t distribute load just requests against the web servers. 

1) Session Affinity
ISA inserts a cookie into the HTTP payload creating a session id for each client requests. All subsequent request from the host includes the session cookie which ISA uses to direct the client to the same web server. This technique relies on a browser that is HTTP v1.1 if it doesn’t or cookies are disabled then ISA cannot use this method. 

2) IP Based
ISA uses the Client IP to directs the request to a specific web server. This technique is problematic if your clients are behind multiple proxies 

Session State

Stateless

Web applications are either statefull or stateless. HTTP is by definition stateless. A client makes a request against a server. A TCP/IP connection is created, the server responds the connection is closed and there is no persistent connection between the client and server. This is fine for static content such a readying this blog.

Stateless Application

Request

Response

Transaction is completed and after a timeout period the connection is closed.

Client Sends another Request and a new connection is created…..

Statefull Application (in load balanced environment)

Many modern web based applications are statefull, that is they need to maintain a logical link between client and specific server. e.g. shopping based activity were you want to pay for your goods. Frequently (due to PCI compliance) you are connected to a 3rd party to process credit card payments. Once this transaction is completed you are then returned to the original site for order confirmation. Without session state been maintained the application server processing your purchase may or may not be the one that continues the process.      

This example focuses on the Application state at the App Server Tier but the issue of maintaining state has to be addressed at each point that there is a load balancing decision between the client and the application tier. 

Typical statefull application in stateless configuration.

Request

Requests back and forth between client and server until a period of inactivity at the client results in a timeout of the TCP connection.

Connection Time Out

Next Request is directed to a different application server (AP1) and the request is unable to be processed resulting in an error on the clients system.

Using infrastructure to maintain session state.

We can address this in a number of ways:

Hardware load balancing can provide session affinity based on the origination client IP or MAC address. This is successful for simple load balanced configurations. However it is problematic for multiple tier load balanced configurations as the second tier always receives requests from a limited number of hosts at the first tier which can easily result in a uneven loading across application servers. 

Application Layer load balancing

Using Cookies for session affinity
The load balancing solution inserts a cookie into the request header, which is used to identify specific user and maintain the relationship between a client and server. This is ideal for the multiple tier load balanced environment.

It does impose the requirement that the client supports HTTP v1.1 so its not going to work for most mobile users or users that disable cookies.  

Protocol Inspection
One such method uses http session id (Session Identification URI to give it its proper name) to maintain session state between client and application server. Alternatively you can insert into the http header a value of your own choosing on which to make load balancing / session affinity decisions.

SSL (HTTPS) and Load Balancing.

When you deal with SSL traffic, it is encrypted between the client and the destination web server. It removes the opportunity to inspect the content of the request / response. Particularly useful then is the ability to load balance based on SSL session ID. Most solutions (ZXTMs and Cisco ACE for example) allow you to do this.  

You could terminate the SSL encryption on the edge of your environment and pass through HTTP traffic internally re-encrypting the traffic as it leaves you environment. Alternatively you could decrypt and then re-encrypt with internal and External SSL certs. You need to consider the load that the SSL offload will have on your solution and also consider the needs of you organisation. Hardware based solutions such as the Cisco ACE modules are licensed based on a number of SSL transactions in combination with network I/O so you will need consider the costs associated with the solution you choose. 

Development led options

There are a number of ways that this can be addressed by the developers of the application. 

Record Session State in Cookie/s

Using this method it doesn’t matter which server receives the request as the session state is recorded in the cookie. This is limited by the number and size (payload) of cookies that can be added to the http header. It requires the application to be developed to accommodate this approach so needs to be designed into the web application. The cookies can add significantly to the amount of data that is transmitted and also increase the processing overhead on the web servers. 

Record Session state in Database

The session state can be written to a database by the application server. The session id is then used to retrieve the session sate. A suitable database tier is required and obviously this has to be designed into the application from the start.

Other Considerations

The return path is equally susceptible to problems relating to state as highlighted below. This needs to be accommodated  in your application / infrastructure design.

Response

Which is the best method to adopt?

Obviously you need to consider the platform, the applications and the infrastructure. If you are managing a simple web application such as a basic MOSS deployment with a pair of load balanced web servers your requirements are considerably different to the delivery DRM protected video assets that need to be restricted based on location of the user making the request. 

If you are managing a complex multi-tier load balanced environment you are likely to be maintaining a highly dynamic set of web based applications. The business will be constantly responding to the environment in which it operates which is likely to include frequent development of the applications, changes to the environment and often with very aggressive delivery deadline.

In my experience the key to successfully managing such environments is to be able to respond quickly and utilise solutions that are versatile. 

If you have access to product such as the excellent Zeus ZXTMs you have the opportunity to inspect and manipulate the client request and server responses directly via traffic script. Its possible to make decision based on a huge number of parameters, providing extremely granular control of the data flowing through your network, manage service levels and respond to requests differently depending on the load on the platform.

Combined with the Load balancing capabilities where decisions can be tailored based on any number of factors such as response times, time of day, requested resource or even geographic location of originating request. You have the tools to be able to operate effectively in such fast moving dynamic environments. This is why I am such a fan of the ZXTMs. They are a software solution that can be deployed very rapidly, tailored very easily by system admin and developers without input from networks. They put the control of the application function into the hands of the guys that are interested in it (no offence to the network guys out there ) and they don’t cost more money if you want to increase the load that they handle.   

I am about to go away to do some onsite consultancy for a week so I thought I would build out my home lab to model the environment that I am going to redesign. I was setting up my Linksys SLM2008 Switch, which among other things supports VLAN which I use with VLAN tagging in VMware ESX server.  Top Tip for your home lab

Anyway while reaching over to check the switch port config, I pressed the keyboard with my arm by accident and got this little pop-up on the command prompt

A quick sprint round the keyboard revealed that F7 produces this list of previous commands.