Earlier this week I saw a tweet from the MIX 2011 conference that highlighted how Microsoft’s Office 365 software as a service platform runs entirely on their Hyper-V hypervisor.
There are those (generally those who have a big investment in VMware technologies) who say Microsoft’s hypervisor lacks the features to make it suitable for use in the enterprise. I don’t know how much bigger you have to get than Office 365, but the choice of hypervisor is becoming less and less relevant as we move away from infrastructure and concentrate more on the platform.
Even so, now that Hyper-V has reached the magical version 3 milestone (at which people generally start to accept Microsoft products) I thought it was worth a post to look back at where Hyper-V has come from, and where it’s at now:
Windows Server 2008 R2 Service Pack 1 includes the third Hyper-V release, bringing new dynamic memory and RemoteFX capabilities. (At around the same time, SCVMM 2008 R2 was updated to support Hyper-V v3Â and the SCVMM 2012 beta was announced).
Performance is as close as no difference for differentiator between hypervisors.
Hyper-V uses Windows clustering for high availability – the same technology as is usedÂ for live migration.
In terms of storage scalabilityÂ – it’s up to the customer to choose how to slice/dice storageÂ – with partner support for multipathing, hardware snapshotting, etc.Â Hyper-V users can have 1 LUN for each VM, or for 1000 VMs (of course, no-one would actually do this).
In terms of data protection, the volume shadow copy service on the host is used an there are a number of choices to make around agent placement.Â A single agent can be deployed to the host, with all guests protected (allowing whole machine recovery) or guests can have their own agents to allow backups at the application level (for Exchange, SQL Server, etc.).
I’m sure that competitor products may have a longer list of features but in terms of capability, Hyper-V is “good enough” for most scenarios I can think of – I’d be interested to hear what barriers to enterprise adoption people see for Hyper-V?
Congratulations to the Windows client and server teams on shipping two great operating system releases. They have their own blog posts on the subject (Windows client and server). I’ll be writing more Windows 7 (and Server 2008 R2) content over the coming days and months so stay tuned!
FCCP solutions are recommended by Microsoft but are not strictly required for support – as long as all the components (i.e. server and SAN) are certified for Windows Server 2008 – a failover clustering validation report will still be required though – FCCP provides another level of confidence.
With a failover cluster, Hyper-V has always been able to offer high availability for unplanned outages – just as VMware do with their HA product (although Windows Server 2008 Enterprise or Datacenter Editions were required – Standard Edition does not include failover clustering).
For planned outages, quick migration offered the ability to pause a virtual machine and move it to another Hyper-V host but there was one significant downside of this. Because Microsoft didn’t have a clustered file system, each storage LUN could only be owned by one cluster node at a time (a “shared nothing” model). If several VMs were on the same LUN, all of them needed to be managed as a group so that they could be paused, the connectivity failed over, and then restarted, which slowed down transfer times and limited flexibility. The recommendation was for 1 LUN per VM and this doesn’t scale well with tens, hundreds, or thousands of virtual machines although it does offer one advantage as there is no contention for disk access. Third party clustered file system solutions are available for Windows (e.g. Sanbolic Melio FS) but, as Rakesh Malhotra explains on his blog, these products have their limitations too.
Windows Server 2008 R2 Hyper-V can now provide Live Migration for planned failovers – so Microsoft finally has an alternative to VMware VMotion (at no additional cost). This is made possible because of the new clustered shared volume (CSV) feature with IO fault tolerance (dynamic IO) overcomes the limitations with the shared nothing model and allows up to 256TB per LUN, running on NTFS with no need for third party products. The VM is still stored on a shared storage volume and at the time of failover, memory is scanned for dirty pages whilst still running on the source cluster node. Using an iterative process of scanning memory for dirty pages and transferring them to the target node, the memory contents are transferred (over a dedicated network link) until there are so few that the last few pages may be sent and control passed to the target node in fraction of a second with no discernible downtime (including ARP table updates to maintain network connectivity).
Allowing multiple cluster nodes to access a shared LUN is as simple as marking the LUN as a CSV in the Failover Clustering MMC snap-in. Each node has a consistent namespace for LUNS so as many VMs as required my be stored on a CSV as need (although all nodes must use the same letter for the system drive – e.g. C:). Each CSV appears as an NTFS mount point, e.g. C:\ClusterStorage\Volume1
and even though the volume is only mounted on one node, distributed file access is co-ordinated through another node so that the VM can perform direct IO. Dynamic IO ensures that, if the SAN (or Ethernet) connection fails then IO is re-routed accordingly and if the owning node fails then volume ownership is redirected accordingly. CSV is based on two assumptions (that data read/write requests far outnumber metadata access/modification requests; and that concurrent multi-node cached access to files is not needed for files such as VHDs) and is optimised for Hyper-V.
At a technical level, CSVs:
Are implemented as a file system mini-filter driver, pinning files to prevent block allocation movement and tracking the logical-to-physical mapping information on a per-file basis, using this to perform direct reads/writes.
Enable all nodes to perform high performance direct reads/writes to all clustered storage and read/write IO performance to a volume is the same from any node.
Use SMB v2 connections for all namespace and file metadata operations (e.g. to create, open, delete or extend a file).
No special hardware requirements.
No special application requirements.
No file type restrictions.
No directory structure or depth limitations.
No special agents or additional installations.
No proprietary file system (using the well established NTFS).
Live migration and clustered storage are major improvements but other new features for Hyper-V R2 include:
32 logical processor (core) support, up from 16 at RTM and 24 with a hotfix (to support 6-core CPUs) so that Hyper-V will now support up to 4 8-core CPUs (and I would expect this to be increased as multi-core CPUs continue to develop).
Core parking to allow more intelligent use of processor cores – putting them into a low power suspend state if the workload allows (configurable via group policy).
The ability to hot add/remove storage so that additional VHDs or pass through disks may be assigned to to running VMs if the guest OS supports supports the Hyper-V SCSI controller (which should cover most recent operating systems but not Windows XP 32-bit or 2000).
Boot from VHD – allowing virtual hard disks to be deployed to virtual or or physical machines.
Network improvements (jumbo frames to allow larger Ethernet frames and TCP offload for on-NIC TCP/IP processing).
So that’s covered the Hyper-V role in Windows Server 2008 R2 but what about its baby brother – Hyper-V Server 2008 R2? The good news is that Hyper-V Server 2008 R2 will have the same capabilities as Hyper-V in Windows Server 2008 R2 Enterprise Edition (previously it was based on Standard Edition) to allow access to up to 1TB of memory, 32 logical cores, hot addition/removal of storage, and failover clustering (with clustered shared volumes and live migration). It’s also free, and requires no dedicated management product although it does need to be managed using the RSAT tools for Windows Server 2008 R2 of Windows 7 (Microsoft’s advice is never to manage an uplevel operating system from a downlevel client).
With all that for free, why would you buy Windows Server 2008 R2 as a virtualisation host? The answer is that Hyper-V Server does not include licenses for guest operating systems as Windows Server 2008 Standard, Enterprise and Datacenter Editions do; it is intended for running non-Windows workloads in a heterogeneous datacentre standardised on Microsoft virtualisation technologies.
System Center Virtual Machine Manager (SCVMM) 2008 R2 is currently in beta and is expected to follow Windows Server 2008 R2 release within 60 days. Complementing the other System Center products to orchestrate VMs, SCVMM 2008 R2 will feature support for live migration and multiple VMs per LUN (using CSVs or 3rd party tools), SAN enhancements (for SAN migration in/out of a cluster – e.g. migrating VMs between two environments on the same SAN – such as staging to production), network optimisations and maintenance mode for simple VM evacuation (no more manual selection and migration of VMs in order to patch the host)!
There are a couple of caveats to note: the SCVMM 2008 R2 features mentioned are in the beta – more can be expected at final release; and, based on previous experience when Hyper-V RTMed, there may be some incompatibilities between the beta of SCVMM and the release candidate of Windows Server Hyper-V R2 (expected to ship soon).
That’s about it for the R2 wave of Microsoft Virtualization – for the datacentre at least – but there’s a lot of improvements in the upcoming release. Sure, there are things that are missing (memory ballooning may not a good idea for server consolidation but it will be needed for any kind of scalability with VDI – and using RDP as a workaround for USB device support doesn’t always cut it) and I’m sure there will be a lot of noise about how VMware can do more with vSphere but, as I’ve said previously, VMware costs more too – and I’d rather have most of the functionality at a much lower price point (unless one or more of those extra features will make a significant difference to the business case). Of course there are other factors too – like maturity in the market – but Hyper-V is not far off its first anniversary and, other than a couple of networking issues on guests (which were fixed) I’ve not heard anyone complaining about it.
Details are still a little sketchy but some of the new features and functionality included in SCVMM 2008 R2 are:
Support for the new features in Windows Server 2008 R2, including:
Live Migration: Enables the movement of running virtual machines from one virtual host to another with no downtime.
Hot addition/removal of VHDs: Allows the addition and removal of new virtual hard disks (VHDs) on a running virtual machine.
New networking protocols: Virtual Machine Queue (VMQ) and TCP Chimney.
Streamlined process for managing host upgrades: Controlling the application of updates or performance of maintenance on a host server by safely evacuating all virtual machines to other hosts on a cluster using live migration or by putting those workloads into a saved state to be safely reactivated when maintenance or upgrades are complete. Maintenance mode is enabled for all supported platforms Hyper-V on Windows Server 2008 R2.
Support for Virtual Desktop Infrastructure (VDI): Enables administrators to deploy and manage virtual desktop infrastructure (VDI) in their data center environment.
Support of disjoint domains: Reduces the complexity of reconciling host servers with differing domain names in Active Directory and DNS. In these situations, SCVMM 2008 R2 automatically creates a custom service principal name (SPN) configured in Active Directory and DNS to allow for successful authentication.
Use of defined port groups with VMware Virtual Center: On installation, SCVMM 2008 R2 will present available port groups for SCVMMâ€™s use – allowing administrators to maintain control over which port groups are used.
Availability of System Center Virtual Machine Manager 2008 R2 is yet to be confirmed; however I would expect this to follow shortly after Hyper-V v2 (which should be in Windows Server 2008 R2). Also missing from the current feature list is the ability to manage Citrix XenServer hosts – something else which we can expect to see following Citrix and Microsoft’s joint announcements last month.
For those interested in taking a look at the beta of SCVMM 2008 R2, it is available via Microsoft Connect.
Every now and again, Microsoft Press makes free e-books available. I just missed out on the PDF version of the Windows Vista Resource Kit as part of the Microsoft Press 25th anniversary (the offer was only valid for a few days and it expired yesterday… that’s what happens when I don’t keep on top of my e-mail newsletters) but Mitch Tulloch’s book on Understanding Microsoft Virtualization Solutions is also available for free download (I don’t know how long for though… based on previous experience, that link won’t be valid for long).
This book covers Windows Server 2008 Hyper-V, System Center Virtual Machine Manager 2008, Microsoft Application Virtualization 4.5 (App-V), Microsoft Enterprise Desktop Virtualization (MED-V), and Microsoft Virtual Desktop Infrastructure. If you’re looking to learn about any of these technologies, it would be a good place to start.
Microsoft’s hypervisor-based virtualisation platform (Hyper-V) has been around for a few months now and, even though there is a whole host of information out there on the web, it’s still a source of confusion for many.
This post is a list of questions and answers for those trying to get started with the Microsoft hypervisor. It is based, in part, on information provided during the Hyper-V technology adoption programme and has been used with the kind permission of Microsoft Windows Virtualization product team, supplemented with additional information where appropriate.
Q. What are my options for installing Hyper-V?
A. Hyper-V is available as a role for x64 Editions of Windows Server 2008 Standard, Enterprise or Datacentre editions (i.e. not for 32-bit x86 or Itanium architectures, nor for web edition). The Hyper-V role is supported on either a server core or a full installation; however server core is recommended, due to its increased security. In addition, there is a standalone version of Hyper-V – Microsoft Hyper-V Server 2008 – designed for organisations who would like the benefits of Hyper-V but who do not run Windows (a comparison of features in the various Hyper-V products is available).
Q. How does Hyper-V’s disk input/output (IO) compare with a non-virtualised solution?
A. In order to ensure that IO will never be reported complete until it has been written to the physical disk, Hyper-V does not employ any additional disk caching other than that provided by the guest operating system. In certain circumstances, a Hyper-V VM can appear to provide faster disk access than a physical computer because Hyper-V batches up multiple requests and coalesces interrupts for greater efficiency and performance. In Microsft’s internal testing they also found that:
Pass-through disks can sustain physical device throughput.
Fixed VHDs can also sustain physical device throughput at the cost of slightly higher CPU usage.
Dynamically expanding and differencing VHDs do not usually hit physical throughput numbers due to the overhead of expansion and greater likelihood of disk fragmentation.
Q. How can I measure performance in Hyper-V?
A. The MSDN website features a section on measuring performance on Hyper-V (specifically relating to running BizTalk Server in a VM but equally applicable to many other workloads).
Q. Sometimes, my virtual machines are paused automatically – why does this happen?
A. Rather than let a virtual machine run out of disk space, Hyper-V will pause the VM if the server is running critically low on space. In addition, an event (ID 16050) is written to the Hyper-V VMMS log.
Q. Is there a method to incorporate the Hyper-V synthetic devices with Windows Preinstallation Environment (WinPE) for servicing?
A. Performing maintenance on a Hyper-V host from within WinPE represents a challenge for systems administrators in that, without the integration components, virtual hard disks (.VHDs) must be connected to the IDE controller (limiting the number of VHD’s that can be used at any given point in time) and legacy network adapters might be required in order to provide network access. Mike Sterling has a great blog post on using the Hyper-V integration components with WinPE (using the Windows Automated Installation Kit to create a custom WinPE image including the appropriate files extracted from the Hyper-V integration services setup disk). Attaching the resulting .ISO image to a VM and powering it on should provide full access to all synthetic devices.
Q. What tools does Microsoft provide to manage Hyper-V?
A. Out of the box, Microsoft provides a Microsoft Management Console (MMC) snap-in (Hyper-V Manager). This snap-in is also available for x86 (32-bit) versions of Windows Server 2008, as well as for Windows Vista SP1 (x86 or x64 – see also Microsoft knowledge base article 952627). If you have the management tools installed on a Windows Vista machine then you might also find Tore Lervik’s Hyper-V Monitor Gadget for the Windows Sidebar useful.
Hyper-V can also be managed using Windows Management Instrumentation (WMI), for example in a Windows PowerShell script and there is an open source PowerShell Management Library for Hyper-V available on CodePlex.
Q. My version of Windows Server 2008 does not seem to have the Hyper-V Management tools available.
A. Windows Server 2008 SKUs without Hyper-V or for other architectures (i.e. 32-bit x86 and Itanium) do not include the Hyper-V management tools.
Physical to virtual (P2V) and limited virtual to virtual (V2V) conversion (V2V is from VMware to Hyper-V – for Virtual Server to Hyper-V there is a free tool available (Matthijs ten Seldamâ€™s VMC to Hyper-V Import Tool) and, for conversions from other products or back to physical hardware, various third party tools are available).
Orchestration of migration activities (i.e. quick migration for Hyper-V, VMotion for ESX).
Intelligent placement of virtual machines.
Management of virtual machine templates, virtual hard disks, CD/DVD (.ISO) images, etc.
Full integration with Windows PowerShell (with supported PowerShell cmdlets) as well as other System Center products such as System Center Operations Manager and PRO packs.
Virtual machine self-service for users to provision their own VMs, based on a quota system.
Q. I’m using the Hyper-V Virtual Machine Connection to access the console of one of my Hyper-V virtual machines but every time I press Ctrl+Alt+left to release the mouse (I do not have integration components installed) my screen turns 90°. Have i been infected with a virus?
A. Probably not! Some Intel chipsets use that key combination to rotate the display. Either turn off that functionality in the display driver settings or press Alt+Tab to break out of the VM and change the hotkey in the Hyper-V settings.
Q. Virtualisation is often cited as an enabler for green IT – how can that be? Surely I’m just moving the same heat and power requirements into one place?
A. An underutilised server still uses a significant proportion of its maximum power and consolidation of many low-utilisation servers onto a shared infrastructure will normally result in power supplies running more efficiently and a net reduction in power consumption.
By consolidating many servers onto onto a smaller number of servers using virtualisation then many servers may be retired. These older servers are likely to be less efficient than a modern server and will all require cooling, resulting in further power cooling savings.
Whilst disposal of old servers is not very “green”, some servers may be redeployed in scenarios where a physical infrastructure is still required.
Q. Does Hyper-V work in conjunction with the Processor Power Management (PPM) power savings in Windows Server 2008?
A. When the Hyper-V server role is enabled system sleep states (standby and hibernate) are disabled. The major savings in power and cooling requirements are gained by switching servers off and, by viewing overall demand for the entire virtualised infrastructure rather than working at an individual sever level, it is possible to use management technologies to bring servers on and offline in order to meet demand.
Virtual machine settings
Q. With Microsoft Virtual Server, itâ€™s really difficult to access the virtual machine BIOS. Is there still a virtual machine BIOS?
A. Hyper-V VMs do still have a virtual machine BIOS; however, all of the BIOS features (e.g. numlock setting, boot device order, etc.) may be set in the virtual machine configuration or using a script. As a conseqence of this, Microsoft has removed the ability to access the BIOS at boot time.
Q. Can a virtual machine boot from SAN (FC or iSCSI), NAS, USB disk or Firewire disks (the boot order in the BIOS settings only shows floppy, CD, IDE and network)?
A. Virtual hard disks (VHDs) can be used to boot or run a VM from:
Local storage (IDE or SCSI).
USB storage (USB key or disk).
SAN Storage Area Network (iSCSI or fibre channel).
NAS Network Attached Storage (file share, NAS device).
It’s also possible to assign a non-removable volume (direct attached storage or a SAN LUN) to an IDE channel in the VM settings and to boot from that device.
Q. I’m confused by the various network interfaces on my Hyper-V host – what’s going on?
A. It’s not as confusing as it first looks! The parent partition is also virtualised and all communications run via a virtual switch (vswitch). In effect the physical network adapters (pNICs) are unbound from all clients, services and protocols, except the Microsoft Virtual Network Switch Protocol. The virtual network adapters (vNICs) in the parent and child partitions connect to the vswitch. Further vswitches may be created for internal communications, or bound to additional pNICs; however only one vswitch can be bound to a particular pNIC at any one time. Virtual machines can have multiple vNICs connected to multiple vswitches. Ben Armstrong has a good explanation of Hyper-V networking (with pictures) on his blog and I described more in an earlier post on Hyper-V and networking.
Q. The virtual machine settings include a processor option which limits processor functionality to run an older operating system such as Windows NT on the virtual machine. What does this feature actually do?
A. This feature is designed to allow backwards compatibility for older operating systems such as Windows NT 4.0 (which performs a CPUID check and, if CPUID returns more than three leaves, it will fail). By selecting the processor functionality check box Hyper-V will limit CPUID to only return three leaves and therefore allow Windows NT 4.0 to successfully install. It is possible that other legacy operating systems could have a similar issue.
Q. Does this mean that Windows NT 4.0 is supported on Hyper-V?
A. Absolutely not. Windows NT 4.0 is outside its mainstream and extended support lifecycle and is not supported on Hyper-V and no integration components will be released for Windows NT 4.0.
Q. But one of the stated advantages for virtualisation is running legacy operating systems where hardware support is becoming problematic. Does this mean I can’t virtualise my remaining Windows NT computers?
A. The difference here is between “possible” and “supported”. Many legacy (and current) operating systems will run on Hyper-V (with emulated drivers) but are not supported. Windows NT is no longer supported, whether it is running on physical or virtual hardware. Microsoft do highlight that Windows NT 4.0 has been tested and qualified on Virtual Server 2005 and that Virtual Server may be managed (along with Hyper-V and VMware ESX) using System Center Virtual Machine Manager 2008.
Copying files between virtual machines
Q. I want to copy files between Hyper-V virtual machines. Is there a way to do this?
A. Microsoft make a distinction between client-side and server-side virtualisation usage scenarios and note that virtualisation servers are typically managed by a group of administrators who want to deploy a secure, locked down server by default (and do not want additional attack vectors created through virtualisation). This is the reason that Hyper-V does not include shared folder or drag and drop functionality (nor are there any plans to do so at a later date). The options for transferring data from one virtual machine to another are:
Setup a virtual network just as you would for physical systems.
Use a virtual CD/DVD creation tool and insert a virtual CD/DVD; this can be done while the virtual machine is running.
Microsoft’s stated position is that, in the case of client-side virtualisation, a single user is running a virtualisation product (e.g. Virtual PC) locally ands expects the capability to move files from one virtual machine to another. For this reason, Virtual PC includes shared folder support (but are not set by default).
Q. How does this work if I move a Virtual PC VM with shared folders to a Virtual Server or Hyper-V system?
A. In this case the shared folders guest components wonâ€™t load because the required server-side components are not available in Virtual Server or Hyper-V.
As a result of a query I had about the supportability (or otherwise) of running System Center Virtual Machine Manager (SCVMM) 2008 in a Hyper-V virtual machine, Clive Watson pointed me in the direction of Microsoft knowledge base article 957006, which discusses the support policy for running Microsoft server software in a virtual environment.
For anyone working with Microsoft software on a virtual infrastructure (even a non-Microsoft environment via the SVVP) it looks like a useful article to be aware of.
(and yes, SCVMM 2008 is supported in a VM – both server and agents.)
Clinic 5935: Introducing Hyper-V in Windows Server 2008
Clinic 6334: Exploring Microsoft System Center Virtual Machine Manager 2008
Clinic 6335: Exploring Microsoft Application Virtualization
Clinic 6336: Exploring Terminal Services in Windows Server 2008
Microsoft’s virtualisation portfolio is not complete (storage and network virtualisation are not included but these are not exactly Microsoft’s core competencies either); however it is strong, growing fast, and not to be dismissed.
System Center is at the heart of the Microsoft Virtualization portfolio and this is where Microsoft’s strength lies as management is absolutely critical to successful implementation of virtualisation technologies. Arguably, no other virtualisation vendor has such a complete management portfolio for all the different forms of virtualisation (although competitors may have additional products in certain niche areas) – and no-one else that I’m aware of is able to manage physical and virtual systems in the same tools and in the same view:
System Center Virtual Machine Manager (SCVMM) provides virtual machine management and server consolidation and resource utilisation optimisation, as well as providing the ability for physical to virtual (P2V) and limited virtual to virtual (V2V) conversion (predictably, from VMware to Microsoft, but not back again).
System Center Operations Manager (SCOM) 2007 (due for a second release in the first quarter of 2009) provides the end-to-end service management; server and application health monitoring and management (regardless of whether the server is physical or virtual); and performance monitoring and analysis.
System Center Data Protection Manager (SCDPM) completes the picture, providing live host virtual machine backup with in-guest consistency and rapid recovery (basically, quiescing VMs, before taking a snapshot and restarting the VM whilst backup continues – in a similar manner to VMware Consolidated Backup but also with the ability to act as a traditional backup solution).
But hang on – isn’t that four products to license? Yes, but there are ways to do this in a very cost-effective manner – albeit requiring some knowledge of Microsoft’s licensing policies which can be very confusing at times, so I’ll have a go at explaining things…
Unlike SCVMM 2007 (which was only available as part of the SMSE), SCVMM 2008 is available as a standalone product but it should be noted that, based on Microsoft’s example pricing, SCVMM 2008 (at $1304) is only marginally less expensive than the cost of the SMSE (at $1497) – both quoted prices include two years of software assurance and, for reference, the lowest price for VMware Virtual Center Management Server (VCMS) on the VMware website this morning is $6044. Whilst it should be noted that the VCMS price is not a direct comparison as it includes 1 year of Gold 12×5 support, it is considerably more expensive and has lower functionality.
It should be noted that the SMSE is virtualisation-technology-agnostic and grants unlimited virtualisation rights. By assigning an SMSE to the physical server, it can be:
Guest Operating System patching/configuration management
End-to-end operating system monitoring
Integrated physical and virtual management
This table is based on one from Microsoft and, in fairness, there are a few features that VMware would cite that Microsoft doesn’t yet have (memory management and live migration are the usual ones). It’s true to say that VMware is also making acquisitions and developing products for additional virtualisation scenarios (and has a new version of Virtual Infrastructure on the way – VI4) but the features and functionality in this table are the ones that the majority of organisations will look for today. VMware has some great products (read my post from the recent VMware Virtualization Forum) – but if I was an IT Manager looking to virtualise my infrastructure, then I’d be thinking hard about whether I really should be spending all that money on the VMware solution, when I could use the same hardware with less expensive software from Microsoft – and manage my virtual estate using the same tools (and processes) that I use for the physical infrastructure (reducing the overall management cost). VMware may have maturity on their side but, when push comes to shove, the total cost of ownership is going to be a major consideration in any technology selection.