Microsoft virtualisation futures

Last week, John Howard presented a Microsoft TechNet UK event about virtualisation. I’ve blogged about Virtual Server before and there is a good overview white paper on the Microsoft website. What I found particularly interesting was the insight which John gave into where virtualisation technology is heading (at least in the Microsoft space).

Microsoft are keeping quiet about where Virtual PC is heading (the official answer to various questions that I raised about the future of Virtual PC was “Virtual PC is not going to be quietly killed off – however we have not made any announcements about the Virtual PC roadmap at this stage”) but there is lots happening with Virtual Server.

First of all, we have Virtual Server 2005 release 2 (R2), formerly service pack 1 (SP1), which is expected to provide a number of improvements over the current release, including:

  • A 64-bit port of the Virtual Machine Manager to support Windows Server 2003 x64 Edition and Windows XP Professional x64 Editions as hosts.
  • Support for Windows Server 2003 service pack 1 as a guest operating system (currently SP1 is newer than the Virtual Server 2005 virtual machine additions and so does not perform well as a guest – as detailed in Microsoft knowledge base article 900076, although Microsoft product support services will also supply the latest additions on request).
  • Support for non-Microsoft guest operating systems (with rumours of virtual machine additions for SUSE and Red Hat Linux, and a revival of the OS/2 support inherited with the purchase of the original Connectix product which Microsoft developed to become Virtual Server).
  • Improved manageability through pre-boot execution environment (PXE) boot support within the virtual machine BIOS.
  • Bug fixes.
  • Performance improvements (up to a 60% improvement for memory intensive applications and 50% better host CPU utilisation).
  • A SCSI shunt driver (i.e. better support for SCSI mass storage device driver installation at bootup).
  • iSCSI clustering support.
  • Virtual hard disk (.VHD) pre-compaction tool.
  • Saved state (.VSV) hard disk space reservation.

Also, when R2 is released there will be a free download (and supporting white paper) called Virtual Server host clustering. This is Microsoft’s answer to VMware Vmotion, allowing virtual machines to fail over between hosts using shared disk clustering (direct attached storage, SAN, or iSCSI). Virtual Server host clustering will be agnostic of guest operating system and Microsoft see two scenarios where it will add significant value:

  1. Planned downtime – allowing live migration of a virtual machine whilst running (effectively a save and restore state). Initial figures indicate that on an iSCSI disk, a virtual machine using 128MB of RAM would fail over in 10 seconds.
  2. Unplanned downtime – migrating virtual machines to another cluster node in the event of failure, albeit with a restart as the virtual server state would be lost when the original node fails.

Post-Virtual Server R2, a service pack is expected (although the cynic in me asks whether it will be repositioned as a new release…) which will offer support for Intel virtualization technology (formerly codenamed Vanderpool) and equivalent technology from AMD (codenamed Pacifica). These technologies will provide hardware assistance for virtualisation, enabling improved performance for non-Windows operating systems (Windows performance is already improved through the use of virtual machine additions, which will no longer be required). Current milestones (obviously likely to change) are for a public beta in the first quarter of 2006 and release to manufacture (RTM) in the third quarter.

VMM arrangements

Further out, in the Longhorn Server wave will include technology called Windows virtualisation for servers (codenamed Viridian), avoiding the requirement for a host operating system.

Windows Virtualisation Architecture

Windows virtualisation for servers is based on a thin, trusted software layer, sometimes referred to as a hypervisor (although this term actually belongs to IBM, hence the long-winded Microsoft product marketing name) and a separate, small, management partition, designed as a foundation role to reduce the attack surface (sometimes known as “MinWin”). Windows virtualisation for servers does require hardware support (there are no device drivers as such as these would impinge on the trusted status of the hypervisor, although an API for independent software vendors is provided at a higher level) and is supplemented by a virtualisation stack with a WMI provider and virtual machine worker processes as well as a system of virtualisation service providers (VSPs) that provide a hardware sharing architecture (for storage, video, keyboard, mouse, USB devices, etc.) and virtualisation service clients (VSCs) that expose the hardware to the kernel, linked by a high-speed in-memory interconnect called the virtual machine bus. The final element of this technology has been christened “enlightenments” – optimisation technologies possibly best illustrated by way of an example:

  • In today’s virtual environments, guest operating systems are unaware of the fact that they are running on virtual hardware. This means that both the host and the guest operating system(s) perform their own memory management. If the guest were to be enlightened and made aware that virtualisation is in use, then this “doubling-up” could be avoided.

The ring numbers in the diagram refer to the four privilege levels within the Intel x86 processor architecture, with -1 being a new level for the hypervisor layer. Windows currently uses ring 0 (kernel mode) and 3 (user mode), with virtualisation additions running in the rarely-used ring 1 to allow non-trappable instructions to be trapped, thus avoiding negative impacts on the host environment (a technique known as ring compression).

There is no doubt in my mind that virtualisation is becoming ever more important, particularly as an enabler for the dynamic data centre. These enhancements to Microsoft Virtual Server, supported with by the new processor designs from Intel and AMD mean that Microsoft is finally set to become a real player in the enterprise virtualisation market.

Using ADS to deploy Windows XP

One of the main reasons for needing to SysPrep my Windows XP installation was that I wanted to see if it is possible to use Microsoft Automated Deployment Services (ADS) to deploy Windows XP.

Microsoft has a plethora of deployment solutions and the main one for workstation deployment is the solution accelerator for business desktop deployment (BDD); however the enterprise edition of this relies on the use of Microsoft Systems Management Server (SMS) and the standard edition requires third-party imaging tools.

Microsoft Remote Installation Services (RIS) is a perfectly good PXE boot server included within Windows 2000 Server and Windows Server 2003 but what I like about ADS is that it uses PXE to boot a miniature version of Windows Server 2003 (not Windows PE) called the ADS deployment agent (DA), which allows control from the server end. Using this technology, sequences can be built up to powerful jobs that control most aspects of a server build and I wanted to do this with a Windows XP workstation build.

The official line from Microsoft is that ADS is not supported for Windows 2000 Professional or Windows XP. Microsoft states that it is not possible to use ADS to deploy Windows XP or Windows 2000 Professional because:

“In addition to licensing constraints, the design of ADS is limited to servers as follows:

  • There is no ability to migrate user state, thus all user information is lost when a new image is applied.
  • ADS is designed to run on server-class hardware and cannot handle the diversity of client hardware.
  • ADS deploys images using a ‘push’ method and does not allow users or staff to initiate a deployment from the client computer.
  • Clients often exist behind slow links and ADS is designed to operate over a well-connected network.”

But ADS works with Windows 2000 Server and Windows Server 2003 (which is very similar to Windows XP in many ways) so I thought it must be possible. In addition, Windows Vista deployment will use Windows Deployment Services (WDS), and although I haven’t looked at WDS, the Windows Automated Installation Kit (WAIK) User’s Guide for Windows Code named “Longhorn” says that:

“WDS enables companies to remotely administer and deploy the latest operating system, using Windows PE and WDS Server. This deployment scenario can be fully unattended, and is customizable and scalable. [WDS] replaces the existing Remote Installation Services (RIS) deployment technology.”

(that sounds like a development of ADS to me!)

One of my ex-colleagues at Conchango pointed me to Paul Edlund’s blog post on using ADS with Windows XP.

This gives advice on SysPrepping the source machine to dump all of the plug and play IDs into the sysprep.inf file (thus avoiding issues with the variety of client hardware).

Quoting from Paul’s article (with minor edits for flow and grammar):

“This allows you to take an image from one machine and use it on a different desktop (assuming the HAL is the same). To perform this step, create a blank sysprep.inf file in the same directory as sysprep.exe. Now open the sysprep.inf file and add the following text to the first line of the file:

[SysprepMassStorage]

Without this tag in the file, SysPrep will run but it won’t put anything in the file (so you can’t forget this). Now save and close sysprep.inf and run sysprep -bmsd. This will dump all of the plug and play IDs from the driver.cab file into the sysprep.inf file. These IDs are used to populate the critical devices database in the registry.

Now copy the contents of the [SysprepMassStorage] section and paste it into the actual sysprep.inf file you want to use from the ADS sysprep.inf templates. The problem is that you will now have populated a huge number of entries in the critical devices database which means that every time your XP machine tries to start, it will try to load each of these drivers, resulting in a very long startup time. So to stop this from happening, add the -clean switch when running SysPrep.”

The SysPrep syntax which Paul gives for the next step didn’t work for me, but I ran sysprep -clean followed by sysprep -reseal -mini -pnp -reboot (although I think the last switch should have been -noreboot as my source computer booted into the mini-setup wizard after SysPrep had completed and I really wanted it to shut down).

There’s some more information in Paul’s article about the various SysPrep switches and the need for a blank administrator password on the source PC (Microsoft knowledge base article 302577 details the usage of SysPrep including the various command line switches).

Screen shot with the ADS deployment in progress

Using Paul’s article, combined with the information in the ADS quick start guide (part of the ADS installation), I was able to successfully capture and deploy a Windows XP image in a Virtual Server environment although there were a couple of gotchas (two of which are related to my use of a virtual environment):

  • Because I’d already SysPrepped the source PC, I couldn’t use the supplied capture-image.xml sequence without editing it to drop the first step (actually I just used the boot-to-da.xml sequence and a one-time job to run the /imaging/imgbmdeploy.exe command with the imagename \device\harddisk0\partition1 "description" -c -client parameters).
  • Also, my use of dynamically expanding virtual disks in Virtual Server meant that the volume size was recorded by ADS as 17166127104 bytes and so I had to use the ADS sequence editor to edit the parameters in the da-deploy-image-wg.xml sequence to use /C:16371 before the deployment was successful.
  • Finally, as the current version of Virtual Server doesn’t include PXE boot capabilities, I needed to use a virtual floppy disk with the contents of the RIS boot floppy (for details, see my earlier post on trials and tribulations with RIS, although Roudy Bob’s virtual RIS boot disk has moved so the link in my original post seems to be broken).

It’s also worth noting that because I was using Virtual Server, all of my hardware was standard. I’d be interested to hear how anybody gets on with this using a variety of physical workstations, but I didn’t have the time or resources to take the experiment that far.

To summarise, capturing and deploying Windows XP using ADS works, but it is not supported by Microsoft. It’s still something to think about if you’re willing to take that risk (I’m not prepared to risk an unsupported solution on my current project with 16,000 workstations spread across hundreds of sites) but if nothing else it’s a good way to spend some time familiarising yourself with SysPrep and ADS.

SysPrep fails on a Windows XP SP2 installation without file and printer sharing enabled

I’m trying out some workstation deployment scenarios right now and need to use the Microsoft System Preparation Tool (SysPrep) to prepare my Windows XP SP2 build for imaging. The trouble is, that SysPrep was refusing to play ball reporting the following error message:

There is an incompatibility between this tool and the current operating system. Unable to continue.

Although there are other tools available for changing workstation SIDs, like Sysinternals NewSID, SysPrep is the only one supported by Microsoft.

I was using the version from the deploy.cab file on the Windows XP SP2 CD (dated 4 August 2004, 13:00), so I thought that a later version may be available but Microsoft knowledge base article 838080 links to an identical set of deployment tools and even indicates that the version on the SP2 CD is current.

It turns out that because the latest version of SysPrep can be used for either Windows XP or Windows Server 2003, it asks the Server service which operating system it is running on. If the server service is not running (e.g. if the File and Print Sharing for Microsoft Networks service is not installed), this fails.

The workaround is to install the File and Print Sharing for Microsoft Networks service, start SysPrep, and then uninstall the File and Print Sharing for Microsoft Networks service whilst SysPrep is working. I found the answer on the Microsoft Software Forum Network Unattended Windows board, but I’m amazed there is not a Microsoft knowledge base article on this.

Virtual Server guest operating system upgrade requires re-installation of virtual machine additions

Here’s a tip for Virtual Server administrators: remember to remove the Virtual Server virtual machine (VM) additions before upgrading a guest VM’s operating system version!

I just upgraded a Windows Server 2003 VM from standard to enterprise edition and couldn’t get the mouse to connect (keyboard control was present, but not what you might call responsive). Virtual Server reported that the VM additions were not installed, but the guest still showed them as present in the Add/Remove Programs applet. Once they had been removed and then reinstalled, everything was back to normal.

Active Directory operations master management

Moving Active Directory operations masters is not something that you need to do every day, and I can never remember how to do it when I need to (well the RID, PDC, infrastructure and domain naming masters are easy enough, but I always forget how to move the schema master because it requires registration of the Active Directory Schema console).

As part of the final preparations for shutting down a virtual machine which I had running as a temporary domain controller, I needed to transfer all of the operations masters roles to the remaining permanent (physical) machine. Full details (along with details for identification of and seizing roles) can be found in the Active Directory how to… manage operations master roles section of the Microsoft Windows Server 2003 TechCenter.

GPMC modelling after upgrading Active Directory

Earlier today, I came across a interesting hangover from last week’s domain upgrade from Windows 2000 Server to Windows Server 2003.

After installing the group policy management console (GPMC), I was viewing a pre-existing group policy object (GPO) and GPMC notified me that Enterprise Domain Controllers did not have read access to all GPOs in the domain. This was initially worrying, but for once the help link had some useful information at the other end.

It turns out that Windows Server 2003 group policy modelling (simulating the resultant set of policy for a given configuration) is performed by a service that runs on domain controllers and in order to perform the simulation in cross-domain scenarios, the service must have read access to all GPOs in the forest.

In a Windows Server 2003 domain (whether it is upgraded from Windows 2000 or installed as new), the Enterprise Domain Controllers group is automatically given read access to all newly created GPOs. This ensures that the service can read all GPOs in the forest.

However, if the domain was upgraded from Windows 2000, any existing GPOs that were created before the upgrade do not have read access for the Enterprise Domain Controllers group.

GPMC had detected this situation and notified me that Enterprise Domain Controllers do not have read access to all GPOs in this domain and after reading the help text was was directed to use one of the sample scripts provided with GPMC, GrantPermissionOnAllGPOs.wsf to update the permissions for all GPOs in the domain.

Whilst logged on with Domain Admins permissions I simply opened a command prompt, navigated to %programfiles%\gpmc\scripts and issued the command cscript GrantPermissionOnAllGPOs.wsf "Enterprise Domain Controllers" /Permission:Read /Domain:dnsdomainname.

The output was as follows:

C:\Program Files\GPMC\Scripts>Cscript GrantPermissionOnAllGPOs.wsf “Enterprise Domain Controllers” /Permission:Read /Domain:home.local
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.

Warning! By executing this script, all GPOs in the target domain will be updated with the desired security setting.

Both the Active Directory and Sysvol portions of the GPO will be updated. This will result in the Sysvol contents of every GPO being copied to all replica domain controllers, and may cause excessive replication traffic in your domain.

If you have slow network links or restricted bandwidth between your domain controllers, you should check the amount of data on the Sysvol that would be replicated before performing this task.

Do you want to proceed? [Y/N]
Y
Updated GPO ‘Default Domain Policy’ to ‘Read’ for Enterprise Domain Controllers
Updated GPO ‘Windows Software Update Services’ to ‘Read’ for Enterprise Domain Controllers
Updated GPO ‘Default Domain Controllers Policy’ to ‘Read’ for Enterprise Domain Controllers

Once this was completed, GPMC was able to function as normal with the existing GPOs.

Advice for potential eBayers


Click here to buy & sell on eBay!

I have a lot of “stuff” hanging around taking up space, some of which I don’t use (mostly old IT odds and ends, plus some books and videos). Now, some of it turned out to have little or no value (at least commercially), but I have sold some of it on eBay.co.uk.

The trouble is, in my bid to make this stuff attractive (in a market of people selling things for silly money), I didn’t set the postage charges high enough…

I hate it when I buy a £4.99 item and someone charges me £5 to ship it, only to find that they only spent a few pence on postage; so, in a bid to be fair, I weighed the items, looked up the correct prices on the Royal Mail website, and just passed on the Royal Mail costs to my buyers. The trouble is, on a couple of items earlier today, I forgot that the weight of the packaging would push it up into the next bracket (and then there’s the cost of buying a Jiffy bag…). In another (just about to be very expensive) deal which closes in a few minutes time, I only charged £6.95 for up to 2kg by Royal Mail Special Delivery but forgot that whilst that should cover the item, it wouldn’t be enough once I’d added the cables that were also part of the deal and the actual cost to me (as the next price band is up to 10kg) will be £19!

I’ll still make a profit but, when I sell goods for less than I would like (to attract bids), and then don’t add enough to cover my postage and packing costs, I feel a bit stupid. Still, at least it’s better than selling books/videos through Amazon, where they set the postage costs and the seller always seems to lose out.

On top of all this, eBay is not really very user friendly. Sure, it guides you through the process but it takes ages to list a new item and you’re never quite sure what communications the buyer has received from eBay so in a bid to keep my feedback high through excellent customer service, I often find myself contacting the buyer to tell them I’ve shipped it and to ask them to leave feedback if they are happy with the purchase.

I understand that an amazingly high number of people are actually in business selling via eBay. Good luck to them, but for anyone like me who’s just trying to flog their old gear, here’s some advice I’ve worked out over the last few transactions:

  • Use a 10 day auction and time it to cross two weekends to maximise your chances of getting some bids.
  • Make sure your postage and packing charges really will cover your costs.
  • Don’t forget that eBay and PayPal will each take their fees on the transaction.
  • See what other people are selling the same or similar items for (and how much interest they have had) and if you don’t want to let it go that cheaply (or there is no apparent interest), leave it a week or so before advertising at the price you think is fair.
  • Remember that even on second-hand goods, your income from the Internet is taxable (yes, I know, it sucks)!

Good luck!

Migrating physical servers to Microsoft Virtual Server

I’ve spent most of this evening at a Microsoft TechNet UK event where John Howard presented Microsoft’s Virtual Server and Virtual PC products. I’ve blogged about Virtual Server before but something I’ve never seen before is Microsoft’s Virtual Server Migration Toolkit (VSMT).

Available as a free download (although, for some reason, registration is required) and also included within Microsoft automated deployment services (ADS) v1.1, VSMT can be used to migrate from physical to virtual (P2V) hardware, or indeed between virtual servers (V2V) – although that would be easier to achieve by simply copying the configuration files (I guess VSMT could theoretically be used for migrating from a VMware platform to Virtual Server and hence also Virtual PC) – but not from virtual to physical (V2P) hardware.

VSMT moves the entire operating system and installed applications, retaining all identity (SID, MAC address, etc.) intact. Microsoft stress that it is targeted for use by IT Professionals and/or Microsoft consultants as it requires some scripting knowledge as well as DHCP and ADS infrastructures (that caveat seems a little strange to me as I wouldn’t really expect anyone other than IT professionals to be administering virtual servers!).

The various stages of the migration are:

  1. Execute gatherhw.exe on the source computer.
  2. Move the output XML file to the ADS controller.
  3. Execute vmscript.exe against the output XML on the ADS controller to generate custom scripts.
  4. Execute the auto-generated capture.cmd script.
  5. PXE boot the source computer, causing an image to be captured.
  6. Power off the source computer.
  7. Execute the auto-generated createvm.cmd script.
  8. Execute the auto-generated deployvm.cmd script.
  9. Configure virtual machine settings, network storage configuration and virtual machine additions.

VSMT does have some prerequisites in that it requires ADS and Virtual Server 2005 (not Virtual PC 2004). The source machine also has to meet certain requirements:

  • Only Windows NT 4.0 SP6A, Windows 2000 and Windows Server 2003 are officially supported by the tool (although another attendee at the event indicated that Windows XP can also be migrated).
  • A minimum of 96MB physical memory is required (in order for the ADS deployment agent to be loaded). This rises to 160MB if FAT disks are used.
  • Windows management instrumentation (WMI) is also required in order for VSMT to gather information about the hardware. WMI is pre-installed with Windows 2000 and Windows Server 2003 but requires a separate download for Windows NT 4.0.
  • The primary NIC must be pre-boot execution environment (PXE) 0.99c compatible (although PXE boot floppies can be used).

VSMT looks to me to be a fantastic tool for administrators who want to consolidate legacy applications (that perhaps very little is known about and which may be running on aging hardware) onto a single modern virtualised platform, or for moving production servers into a virtualised environment for test and development purposes.

Britains Secret Intelligence Service is not so secret now!

I’m sitting here in my hotel room half-watching Spooks on the BBC, and it reminded me that the British Secret Intelligence Service (MI6) joined the world-wide web community today (Spooks is actually about the British Security Service – MI5, but that’s close enough).

Not bad for an organisation that the very existence of was officially denied until 10 years ago!

Setting up IP forwarding on a Windows network

My network at home has two subnets joined by a wireless link (note that the IP addresses have been changed to protect the innocent):

IP forwarding

You might wonder why it doesn’t all sit under my desk (after all we’re not talking about a multinational corporation here) but the simple fact is that most of my kit has been procured from an eclectic mix of sources over the years (so it is hardly what you might call standard) and the server (on which I do a lot of testing) is a noisy beast, as is the 24-port switch that it’s plugged into – hence the reason they are stored away in the basement.

The trouble with this configuration is that the dual-homed PC which acts as a bridge between the wired and wireless segments in the basement is exactly that – dual-homed – i.e. it needs the 802.3 adapter to be on one subnet and the 802.11b adapter to be on another (otherwise this could all have been on one flat subnet). That means that it also needs to be able to route traffic to and from each subnet, otherwise the server is invisible to the rest of the network (and vice versa).

That’s where IP forwarding comes in (aka IP masquerading in Linux-speak).

Disabled by default in Windows 2000, XP and Server 2003, IP forwarding basically allows a dual-homed host to act as a network bridge. Microsoft knowledge base article 323339 details the registry setting to enable this on Windows Server 2003 – there are other articles for Windows 2000 and XP but they are pretty much identical.

There are, however, a couple of important points to note:

  • Only one interface should have a default gateway. In my case, the default gateway for the bridge’s wired connection is blank.
  • I also had to put a static route to 192.168.2.0/24 on my ADSL router using the IP address of the bridge’s wireless connection as a gateway (so that outbound traffic to the Internet from the 192.168.2.x network has a return path).

For comparison purposes, the routing table on my bridge (192.168.1.50/192.168.2.50) looks like this:

IPv4 Route Table
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x10003 ...00 08 02 xx xx xx ...... Intel(R) PRO/100 VM Network Connection
0x10004 ...00 80 c8 xx xx xx ...... D-Link AirPlus DWL-520+ Wireless PCI Adapter
===========================================================================
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.1.1 192.168.1.50 25
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
192.168.1.0 255.255.255.0 192.168.1.50 192.168.1.50 25
192.168.1.50 255.255.255.255 127.0.0.1 127.0.0.1 25
192.168.1.255 255.255.255.255 192.168.1.50 192.168.1.50 25
192.168.2.0 255.255.255.0 192.168.2.50 192.168.2.50 20
192.168.2.50 255.255.255.255 127.0.0.1 127.0.0.1 20
192.168.2.255 255.255.255.255 192.168.2.50 192.168.2.50 20
224.0.0.0 240.0.0.0 192.168.1.50 192.168.1.50 25
224.0.0.0 240.0.0.0 192.168.2.50 192.168.2.50 20
255.255.255.255 255.255.255.255 192.168.1.50 192.168.1.50 1
255.255.255.255 255.255.255.255 192.168.2.50 192.168.2.50 1
Default Gateway: 192.168.1.1
===========================================================================
Persistent Routes:
None

Whilst on the ADSL router it looks like this:

Network Destination Netmask NextHop IF Type Origin
0.0.0.0 0.0.0.0 isprouter ppp-0 Indirect Dynamic
127.0.0.0 255.0.0.0 127.0.0.1 lo-0 Direct Dynamic
192.168.1.0 255.255.255.0 192.168.1.1 eth-0 Direct Dynamic
192.168.1.1 255.255.255.255 127.0.0.1 lo-0 Direct Dynamic
192.168.2.0 255.255.255.0 192.168.1.50 eth-0 Indirect Local
isprouter 255.255.255.255 mypublicipaddress ppp-0 Direct Dynamic
mypublicipaddress 255.255.255.255 127.0.0.1 lo-0 Direct Dynamic
btrouter1 255.255.255.255 btrouter2 ppp-0 Direct Dynamic

For the other LAN-connected devices, the important details are that for LAN 1 the default gateway is 192.168.1.1 and for LAN 2 the default gateway is 192.168.2.50.