I recently built out a CMS hosting environment for a UK Airport web site. Whilst doing this I came across some issues deploying ZTM and ZLB appliance which is easy to resolve if you know how.

VM Nic Bindings

OR..  Binding VM Nic 1 to eth1 and VM Nic 2 to eth0

Let’s say for arguments sake that the networks are configured as follows 

ZTM 1 has following desired config.

The Network adapter 1 is in .76 (VLAN 301 10.0.74.64/27)
The Network adapter 2 is in .108 (VLAN 302 10.0.74.96/27)

Notice that eth1 is bound to VM Network adapter 1 and eth0 to VM Network adapter 2. Compare the MAC addresses above with the mappings below.

This is a problem because the ZTM has the wrong IP config for each NIC.

We could go into the appliance OS and change the nic config as already highlight below the VM ZTM / ZLB Appliance config is managed by the ZTM/ZLB install and you will see messages like below in the config files.

I found this out the hard way when I added hosts to the hosts file, only to see them disappear and my application stopped running. Now I know this is done on the appliance via the GUI (System>Network>DNS).

FIX 1

You can try running z-set-inital-address as below.

Open the Zeus Virtual Appliance‘s console interface (via VMware management interface)

Press Alt+F2 to switch to tty2.

Logging in as admin with the default password of admin.

Running the z-set-initial-address command.

This will prompt you for an IP address and netmask. Once the command terminates, enter the logout command, and switch back to tty1 by pressing Alt+F1. You should notice that the IP address in the URL for the admin server has changed

This did not work for me.

FIX 2

This is more of a work round than a fix. It leaves, as a colleague of mine would say  a ‘sub-optimal’ configuration

Open the VM Appliance summary tab

select Edit Settings and swap the VLAN assignments

In this case I configured 

VM Network adapter 1 in VLAN 302 and
VM Network adapter 2 in VLAN 301 

This results in this configuration.

No Default Gateway

I have already mentioned that there is a command to set the initial IP address for the GUI access to actually configure the appliance.   

The command is z-set-inital-address. Below is a screenshot that shows the default gateway is left as 0.0.0.0 after running this command.

at the console type netstat –rnv to show the gateway

If the above happens to you type route add default gw <gateway_ip>

In my case

route add default gw 10.0.74.65

and then navigate to the GUI to do the initial config as normal.

Quick look under the bonnet

The main ZTM files live in /opt/zeus

Nic config is in

/etc/network/interfaces

Hope this helps

Whilst researching my AMD CPU comparison post I found these useful AMD virtualisation  resources. I think that they will be helpful in changing opinions that have been form previously about virtualisation as a enterprise solution.

http://www.amd.com/us-en/0,,3715_15781,00.html?redir=SWOP08

VIRTUALIZING SERVER WORKLOADS
LOOKING BEYOND CURRENT ASSUMPTIONS
"And while virtualization has moved out of its traditional domain into mainstream production, conventional thinking about its capabilities and limitations still keeps many organizations from embarking on large implementations. The suitability of workloads for virtualization in production environments is often guided by assumptions that are no longer true. Because every server workload has distinct performance characteristics, understanding and taking those characteristics into consideration is necessary to determine the server’s suitability for virtualization.
Improvements in virtualization hardware and other advances in virtualization technology are making it possible to virtualize an increasingly wide range of
workloads.

In summary, organizations that decided against virtualization in the past because of concerns about CPU, memory, or I/O bottlenecks would do well to take
another look.
AMD WP Virtualizing Server Workloads.pdf

What is AMD Virtualization™ technology?

The following link has a click-able image that directs you to the appropriate section  

http://www.amd.com/us-en/0,,3715_15781_15785,00.html

I notice with great interest that power is starting to push its way to the front of the agenda when discussing any projects these days. While at Tech.Ed this year we were told that Microsoft are building a large DC in Ireland "because of ambient air temperature" Its cold most of the time . I didn’t appreciate what this meant until I read the article below which suggests allowing your DC to run at a balmy 32 C (much like any married man’s living room * ) instead of the usual 18C (65F) and use air from outside to provide cooling. I guess this is an option for some and is certainly something I will discuss with our DC manger the next time I see them.        

From: Green Checklist: 10 ways to save energy in the data centre

"Data center energy use can lead to economic catastrophe for many companies (in 2009). For example, energy use at the largest data centers grew at a 7.5 percent compound annual growth rate (CAGR) between 2000 and 2006 but will hit a 24 percent CAGR from 2006 to 2010, according to Ken Brill, executive director of the Uptime Institute. At that rate, the United States will need 30 new 1,000-megawatt power plants by 2015 just to support data center usage."

CAGR – Compound Annual Growth Rate

http://www.accelerateresults.com/category/8/article/169-green-checklist-ways-to-save-energy-in-the-data-center-

* A genius is any man who can adjust the thermostat to please his wife.

If every there was a killer app for virtualisation this is it. Spend on virtualisation to save a huge amount on power/hosting. 

I have just completed the Certified PlateSpin Analyst (CPSA) exam.

This is a cracking product for migration, DR, consolidation or technical refresh projects. It comes into its own where you need to maintain service with minimal interruption – hence the name

PowerConvert

http://www.platespin.com/products/powerconvert/

Basically you can take an image of a target source which can be an image, physical or virtual server and provision a destination source which can be a physical, virtual or image. Supports live file, block and VSS based imaging + offline PE based imaging of a variety of Windows and Linux OS inc (Windows NT4)

PowerRecon

They have a sister product that is used to analyse/benchmark you environment and then either provided details for consolidation or specification for migration/planning. Its very similar to VMware Capacity Planner

http://www.platespin.com/products/powerrecon/

My colleague has put together some excellent posts on practical examples:

http://vinf.net/2008/11/25/platespin-powerconvert-part-1-overview/
http://vinf.net/2008/11/26/platespin-powerconvert-part-2-p2v/
http://vinf.net/2008/12/01/platespin-powerconvert-part-3-v2p/

Interested but need assistance?

I work for a ioko if you are in-need of consultancy services around these or indeed any technical professional services get in touch with us here. We can help.

So Simon gave me some tips – eSATA II, big cache 16Mb is good and fast so 7200RPM. He pointed me at the Western Digital WD3200BEKT 2.5" 320GB 16Mb cache 7200RPM  we found it for around £65inc VAT + P&P.

So I tried the usual suspect for quality hardware at very competitive prices

Overclockers
Lambda-tek
MicroDirect
ebuyer
TechStore

They are all in the upper £65+ mark 

I managed to find it here for the princely sum of £57.82 inc Vat
http://www.drivemylaptop.co.uk/LaptopPartsList.aspx?pt=3

Saw this and following on from my TA Tips for Newbies thought this was useful

http://technet.microsoft.com/en-gb/bb986609.aspx

On Demand Live Meeting: TechNet Presents: MCS Talks Infrastructure Architecture– session 1, Infrastructure Architecture Overview
This first session will introduce the “MCS Talks Infrastructure Architecture” webcast series, giving an overview of the series and what’s to come. We will then move in to an overview session looking at the general architecture strategies of specific technology areas, such as adopting Microsoft Windows Server 2008 and Microsoft Windows Vista SP1 and show some real benefits from their deployment. Datacentre consolidation is also a serious consideration for many so we’ll be looking how Microsoft Hyper-V and the System Center suite of tools can be used to accelerate this and ensure long term reliability from this approach. Finally we’ll cover some of the other virtualization strategies we’re seeing, such as when and why we would look to virtualize particular workloads, how virtualization can support branch office models and remote working and finally Microsoft overall virtualization storey, including Presentation and Application virtualization.

ON DEMAND TechNet Presents: MCS Talks Infrastructure Architecture – session 2, Core Infrastructure
This session will look at core infrastructure services, focussing on the decisions commonly faced by enterprises when deploying Active Directory. We’ll look at how the improvements in Microsoft Windows Server 2008 can help rationalize the Active Directory infrastructure while still supporting complex and disperse environments. We will cover the design process for an Active Directory Domain Services (AD DS) upgrade and consolidation project, considering the key decisions which must be made and how these will benefit an enterprise environment. New features such as read-only domain controllers (RODCs) and fine-grained password policies will be covered as well as more general design principles and considerations for remote workers. Finally we will cover how it is possible to take advantage of new virtualization technologies in supporting Active Directory. TO ACCESS THE MEETING ON DEMAND PLEASE CLICK REGISTER AND FOLLOW THE STEPS

Interestingly it would seem that Emulex is putting their effort into Hyper-V based on a quick look at the site and resources available for ESX & Hyper-V. I wonder if that has anything to do with EMC?

Hyper-V

Hyper-V Virtualization – Enhanced SAN Connectivity with Emulex
http://www.emulex.com/solutions/windows_server_2008/virtual.jsp

Virtualization Self Assessment Tool – Discover your ideal SAN environment for Hyper-V
http://www.emulex.com/solutions/windows_server_2008/virtualizationTool.jsp

VMware ESX – Support Documents
http://www.emulex.com/emc/support/vmware.jsp

A resounding chorus of “YES”! was the reply. Now I got chatting to the nice gentleman from Microsoft called Andrew Fryer (it is because of him and my colleagues Simon G and Simon S that I am now writing a blog!)

Andrew who has the wonderful job title of Evangelist provided me details of a test conducted on a four instance production SQL server 2008. This is a 64 page document on SQL virtualisation (under Hyper-V) best practice.

http://sqlcat.com/whitepapers/archive/2008/10/03/running-sql-server-2008-in-a-hyper-v-environment-best-practices-and-performance-recommendations.aspx

Andrew is a very nice chap and he told me about allsorts of interesting SQL related stuff not least a hint that Microsoft are working on SQL deployment that supports sideways scale-out.   We were talking and I said that basically I arrived at the conference thinking “if you want to go virtual you use VMware ESX period”. Now I am not saying that I have had an epiphany (after all Andrew is an Evangelist) but I have to admit that Hyper-V is worth consideration. Furthermore when Hyper-V R2 is released (2010 anyone?) then VMware better get doing some pretty clever stuff or the sheer scale of Microsoft will simply overwhelm them. Right now ESX is a much slicker proposition but it has its downside and a Microsoft’s unified approach is one key area where deciding IT strategy is based on budgets rather than slick technology. So as promised Andrew here is I hope an unbiased round up of ESX 3x v Hyper-V (& R2) and to be fair I will also mention some stuff I know about ESX 4x.

VMware ESX 3x Microsoft V Hyper-V Compare and contrast

I will discuss following areas

Resilience / High Availability
Networking
Storage
Deployment
Manageability
Conclusions

I will outline the technologies available in each solution and throw in my own thoughts for good measure.

ESX = 3x (3.5 and 3i ) unless stated
HV = Hyper-V
HV2 = Hyper-V R2

Resilience

The key concern for a virtualised environment is the high availability of the physical platform on which the VMs reside. It quite clear that a physical production server tends to be fairly important in the modern IT environment and its loss is usually negated through a variety of strategies. Consider the transition from the physical to the virtual. You make a single point of failure represent a potential catastrophic failure with one physical server running 20 or more virtual machines. How do we address this?

For all solutions I assumes that VMs are located on shared storage. I am not interested in stand alone deployments for the reason mentioned above.

ESX 

The ESX suite has a number ways of achieving resilience.   

HA – High Availability
ESX server is deployed in a “cluster” configuration with each ESX server monitoring its fellow cluster members and its own console port gateway address. If a member of a cluster is isolated (network issue) or physically disappears (hardware issue) then the other member/s take ownership of the VMs and subject to resource criteria (see DRS below) start spinning up the VMs to restore service. 

Pros:

Technology is tried and tested, works and is very simple to configure.

Simple solution based on ICMP hosts ping each other and gateway. If host is isolated from cluster and gateway it shuts down VMs on the assumption that another ESX server will take on the workload. If the ESX server is isolated from its peers but can still ping gateway it continues as normal assuming that peer/s are at fault.

Cons:

Downtime can be significant, up to 15 minutes.
Although not strictly a HA issue it is possible to configure ESX (via Virtual Centre) in a way that prevents the successful migration of VMs  due to resource allocation conflicts.

VMotion

Targeted at graceful migration of VM from one ESX server to another. The technology in its simplest terms starts up a copy of the VM to be migrated on the new destination ESX server. The two VMs are then brought in CPU step with each other and when they are fully synchronised a reverse Arp is issued that updates the switch port MAC address tables resulting in all network traffic been directed to the new VM, this final step takes a few milliseconds during which there is no network connectivity to either VMs. Once complete the original VM is powered down.      

Requires a separate 1GB VMotion network to facilitate the synchronisation in a timely manor.  

Storage VMotion

Much like VMotion only the actual VM files are moved from one storage system to another. This enables VMs that have become disk I/O bound to be migrated without downtime.

ESX v4 Introduces the concept of HA VMs

A VM is run on two separate physical ESX servers utilising specific Hardware capability of the new Intel chipset the VMs are CPU synced all CPU cycles are executed one primary host and secondary host so that in the event of a failure a reverse ARP is issued and the secondary VM is instantly available.       

Cons:

This requires specific hardware to implement

Pros:

If this works it will be the single most effective HA solution available to the Windows Systems Administrator. At present MSCS Clusters are the best option for true HA applications.

HV

Utilises Microsoft Cluster services for HA. Currently there is no zero downtime migration capability.

MSCS Microsoft Cluster Services

This is a area were the future is bright for HV but right now is a poor relation to ESX. In order to create HA solutions for HV a MSCS cluster is created and VMs are stored on the shared volume.

This results in its own problems

1) In order to facilitate a failure between hosts MSCS need to gain LUN level ownership for a host, this means that in order to provided a failover solution that only effects the specific VM that needs migration (thing network card failure) then a VM must have its own LUN

2) As each VM needs its own LUN SAN configuration becomes complex very quickly. Invariably this will add extra load to the SAN storage processors and a much higher administration overhead on fabric management. Furthermore a host can rapidly run out of drive letters to assign to LUNs

HV R2 – Clustered Shared Volumes 

The reason that ESX has VMotion and Storage VMotion is because of the VMFS File system. Put simply VMFS allows multiple hosts to see the same LUN at the same time. Access is at the file level and therefore multiple hosts can write to a LUN simultaneously without effecting the integrity of the LUN. This is not the case with NTFS. NTFS has its hands tied here however there is light at the end of the tunnel…..

CSV allows multiple hosts to connect to a LUN at the same time. All writes are effectively proxied  through a master host (one per CSV) and any host (in the cluster) can read at anytime from the LUN.

This combined with geographically diverse clusters is a very useful technology and has uses beyond virtualisation. Think geographically diverse SQL databases… and while I think about it possibly load balanced SQL DBs for web applications e.g. predominantly read orientated DB access. (I am looking at a mySQL load balanced configuration with ZXTMs at the moment V. Clever. )

Comments:

At Tech.Ed this technology was demonstrated and looks the business. Is it as easy to setup and manage as ESX alternatives? I would have to say no.

HA / Resilience – ESX 1  HV 0

Useful Links

Hyper-V
Hyper-V Step-by-Step Guide: Hyper-V and Failover Clustering

VMware Infrastructure 3 Online Library (HTML)

In the next part I will look at Networking….