Configuring network connections for ISA Server 2000/2004 (aka when proxy server migrations turn bad)

It was supposed to be so easy. The new server was already built, with the same IP addresses as the old one. All I had to do was disconnect the NT 4.0 Proxy Server from the network and power on the new Windows Server 2003 R2 box with Internet Security and Acceleration (ISA) Server 2004 on it, then configure and test a few filter rules; but I had forgotten the first law of IT consultancy – nothing is ever straightforward – which is why I’m writing this post on the train to work after rolling back the migration and getting just 4 and a half hours sleep last night…

Firstly, I decided that the ISA Server should be joined to the Active Directory. My original plan had been that leaving it in a workgroup would be secure, but as I didn’t want to allow unrestricted anonymous (i.e. unmonitored) Internet access I’d be limited with my authentication options (either set up a RADIUS server to handle authentication or mirror the user accounts on the ISA Server). I wasn’t confident that ISA Server would work well if it was joined a domain after installation so I uninstalled ISA Server, joined the computer to the domain, and reinstalled ISA Server, plus service pack 2 and other updates.

It only took a few seconds to configure the cache and set up a firewall policy rule to allow all ports outbound access (just as a test, I could lock it down again later), add all the internal networks and enable the web proxy client, following which Internet access from the local network was restored. The trouble was that none of the machines on remote sites could access the Internet.

Suspecting a DNS issue, I began to investigate name resolution problems and (here was my mistake) questioning why no forwarders were configured on the internal DNS server (because DNS monitoring showed that the simple queries were fine, but recursive lookups were failing). If I’d been thinking clearly, I would have realised that the internal network doesn’t need to have a recursive DNS path to the ISP’s DNS servers (the proxy server should handle that on behalf of the clients) – although I do think that having a clear path from clients to the internal DNS and onwards to the ISP’s DNS is the most straightforward configuration, supporting both internal (Active Directory) and external (Internet) name resolution (and Microsoft’s advice is to configure only internal or external DNS on the ISA Server – not both).

The problem was the network configuration on the ISA server. Jim Harrison’s excellent article on configuring ISA Server interface settings is my bible when configuring the network cards on an ISA server, but I hadn’t set up the routes from the external network to my internal networks correctly. The local LAN was fine, but ISA Server was rejecting requests from remote internal networks because it didn’t understand the underlying network path (flagging a configuration error alert warning that the address range of an ISA Server network should match the address ranges routable through the associated network adapter as defined in the routing table). When I monitored the traffic flow, I could see incoming requests that were denied with no rule was given as the reason – another clue that there was a problem with the network rules.

Although the configuring ISA Server interface settings article points out that a route will be required to each internal network, I’d set the next hop as the internal interface of the ISA server, rather that the local router (the internal NIC doesn’t have a default gateway if configured correctly). Adding persistent routes for each of the internal networks (route -p add remoteinternalnetwork mask subnetmask routeripaddress) fixed the issue, after which nslookup (and web access) began to work from all sites.

Unfortunately, by the time I’d worked this out, it was too late to set up and test the various filter rules that are needed to ensure correct (authenticated) HTTP(S) and FTP browsing, SMTP e-mail, access to OWA, etc., so I decided to back out and reconnect the legacy proxy server. At least now I know that the connectivity problems are resolved, I can attempt the migration again another evening.

High definition TV appears pixelated

A few months back, I wrote about how I was receiving free-to-air digital TV from Sky but yesterday, I noticed that the picture was clearer on analog than on digital, which appeared pixelated. Well, it seems that Sky is getting ready to launch it’s high definition service next week in glorious 1080i or 768p; however for those of us who are still hoping to get a few more years of life from our widescreen CRT TVs (I bought a 32″ Sony Trinitron in 1998 and it will probably outlast many flat panel TVs on sale today) it seems that our picture quality will deteriorate.

If HD’s a mystery to you then check out the HD TV article on Wikipedia . According to the BBC, Sky’s new service starts on 22 May but digital terrestrial (Freeview) viewers may have to wait until after the UK has completely switched to digital TV in 2012 to receive HD broadcasts.

Finding out how Windows product activation works

Most of the work I do with Microsoft software is carried out for clients who have a volume license agreement, so working with OEM copies of Windows Server 2003 R2 over the last week or so has been my first exposure to Windows product activation. After having built and activated a server, then wondering whether blowing it away and starting again would affect the activation status I found Alex Nichol’s description of Windows product activation on Windows XP (it’s basically he same for Windows Server 2003 R2). I decided not to rebuild the server in the end but Alex’s article was a certainly a useful description of how the activation process works and the hardware changes that can affect the validity of the software.

Using unprivileged accounts in Windows

A few weeks back, Microsoft UK’s Steve Lamb presented a session on using the principle of least privileged access to reduce exposure to security threats under Windows (basically, running as much as possible as a standard, non-administrative user). Unfortunately I missed the event but I was chatting with Steve last week and he filled me in on the basic principles (which I’ve padded out with a few notes from his slidedeck).

The runas command can be used to start a program as a different user (as programs inherit their permissions from the parent process, starting a cmd shell as an Administrator and then launching an application will launch that application as an Administrator. Within the Windows GUI, there is often a right click option for runas, although for control panel applets shift and right click is used to expose the runas option. Shortcuts can be modified to run with different credentials for applications that always require a higher level of access.

There are occasions when runas just doesn’t work – for example applications that reuse existing instances (Windows Explorer, Microsoft Word) or those that are started through the shell using the ShellExecute() API call or dynamic data exchange (DDE). Unfortunately Microsoft Update is one of those applications for which runas won’t work. Aaron Margosis has some advice on his blog to help work around issues with runas and Windows Explorer.

Privileged command shell windows can be set apart using a different colour scheme, for example:

cmd.exe /t:cf /k title Administration Shell

For the GUI, the TweakUI power toy can be used to set an alternative bitmap for Internet Explorer and Windows Explorer, or Aaron Margosis’ PrivBar displays the current privilege level.

Whilst it’s true that using a local account will prevent domain-wide issues, there are side effects in that there is no access to domain resources, different profile settings (and per-user policy settings) are in effect and some applications assume that the installer is the end user. One possible resolution is Aaron Margosis‘ MakeMeAdmin tool which allows for temporary elevation of the current account’s privileges (and any applications which inherit the user context. MakeMeAdmin can be downloaded from Aaron’s blog and he has a later follow-up post with more information.

Some applications are written to run as Administrator and there’s not a lot that an end user can do about poor coding (other than replacing the application with something else). Adding the user to the local Administrators group to resolve such issues is not good practice, although it may be possible to loosen the ACLs on application-specific resources (i.e. %ProgramFiles%\applicationname\ and HKEY_LOCAL_MACHINE\SOFTWARE\applicationname\Settings) but this should not be carried on operating system resources (e.g. %windir%, %windir%\System32 and
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows). The important thing to remember is to do this in a granular fashion, applying additional permissions only to those resources to which access is required.

If an application writes to HKEY_CLASSES_ROOT, then it’s usually a bug. HKEY_CLASSES_ROOT is a merged view of HKEY_LOCAL_MACHINE\SOFTWARE\Classes and HKEY_CURRENT_USER\SOFTWARE\Classes so writing to HKEY_CLASSES_ROOT effectively goes to HKEY_CURRENT_USER if the key already exists. Consequently, problems with HKEY_CLASSES_ROOT can often be overcome by pre-creating keys under HKEY_CURRENT_USER.

If all else fails, utilities such as MakeMeAdmin can be used to allow an application to run with elevated privileges but they require the user to know the Administrator password – alternatives include Valery Pryamikov‘s RunAsAdmin and DesktopStandard PolicyMaker Application Security.

In Windows Vista, everything changes again with new functionality known as user access control (also known by other names including user access protection and flexible account control technologies):

  • All users run as an unprivileged user by default, even when logged on as an Administrator.
  • Once running, the privilege of an application cannot be changed.
  • Administrators only use full privilege for administrative tasks or applications.
  • Users are prompted to provide explicit consent before using elevated privilege, which then lasts for the life of the process.
  • A high level of application compatibility is achieved using redirection (which allows legacy applications to run as a normal user with HKEY_LOCAL_MACHINE\Software access being emulated by a virtual location under HKEY_CURRENT_USER and attempted writes to the %SystemRoot% and %ProgramFiles% folders being redirected to a per-user store); however this is a temporary mitigation for 32-bit product versions only (i.e. not implemented in 64-bit versions of Windows Vista).

Although Windows has come a long way to making least privileged access usable, it’s important to remember that there are some things that least privileged access can’t guard against:

  • Anything you can do to yourself.
  • Weak passwords.
  • Attacks on services.
  • Phishing.
  • Stupidity.

Unfortunately I’m writing this post on the notebook PC supplied by my employer with a standard corporate build and my domain account is also a local administrator. I think that probably falls into the last category listed above… doh!

Configuring database size limits with Exchange Server 2003 SP2

One of the enhancements provided in Exchange Server 2003 service pack 2 (SP2) is the increase of the maximum database size for Exchange Server 2003 standard edition from 16GB to 75GB. I originally thought that it was just a case of installing the service pack after which the database would be allowed to grow past the 16GB barrier but as Microsoft knowledge base article 912375 and the database size limit configuration and management article in the Microsoft Exchange Server TechCenter describe, the limit actually needs to be configured manually.

Scott Lowe’s article at TechRepublic details the necessary configuration as follows:

  1. Start the registry editor and browse to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\servername\storeguid.
  2. Create a new DWORD value named Database Size Limit in GB and set it to a size between 1 and 75.
  3. Restart the Microsoft Exchange Information Store service.

This configuration method can be used with Exchange Server 2003 standard edition to configure (on a per-store basis) any value between 1GB and 75GB and for enterprise edition (which has no limit by default) to enforce a limit of up to 8000GB if required; however if the Exchange server is recovered using the /disasterrecovery switch, the registry keys will need to be recreated.

Outlook Web Access tips and tricks

After I installed Exchange Server 2003 last weekend, I found that accessing Outlook Web Access (OWA) didn’t prompt for any authentication – it simply displayed the contents of the currently logged on user’s mailbox (if it was homed on Exchange Server 2003) or redirecting to the old Exchange Server 5.5 OWA server (where the mailbox had not yet been migrated). In order to allow users to authenticate with a username and password, it was necessary to enable forms-based authentication (Daniel Petri’s description also includes the process of configuring SSL).

Once this had been enabled, HTTP requests were greeted a normal browser authentication dialog; however HTTPS access was be greeted with a much more pleasant OWA logon screen.

OWA authentication via HTTP

OWA authentication via HTTPS

It is also possible to change the username format for OWA forms-based authentication from domainname\username to a simple username.

Finally, Microsoft knowledge base article 319878 describes the process to make OWA the default website on an Exchange Server, avoiding the need to access the server as http://servername/exchange/ or https://servername/exchange/.

Some more about what to expect in Exchange Server 2007

A few months back, I wrote a bit about what to expect in the next version of Microsoft Exchange Server. Since then, I’ve learned a lot more about Exchange Server 2007 (formerly codenamed E12) but couldn’t repeat much of it. The following highlights are some of the additional information that was made public in Eileen Brown‘s presentation at last week’s Microsoft Technical Roadshow, starting with Exchange Server product progress (since the launch of Exchange Server 2003):

Key new features with Exchange Server 2007 (some of which I’ve written about previously) include:

  • The use of 64-bit server technology facilitates a reduction in input/output operations and hence allows more databases (with larger mailboxes) to be placed on each server.
  • High availability enhancements, allowing increased data and service availability: database continuous replication (either local or clustered) allows daily full backups to be replaced with weekly full backups and daily incrementals the second copy of the database plays the transaction logs from the first as they are written, meaning that it is never more than once transaction log behind the first; and there is now support for geoclustering.
  • Improved mobile e-mail, including push e-mail, policy-based provisioning and security.
  • Improved system management tools: a new MMC 3.0-based Exchange System Manager and the Exchange Management Shell (running on Windows PowerShell).
  • Automatic client configuration.
  • Encrypted and signed intra-organisation e-mail (by default) and encryption at the gateway (where supported) for business-to-business e-mail with no additional client requirements.
  • New message hygiene functionality.
  • Message journalling at the transport level with new data retention rules and a new content indexer, allowing a multiple-mailbox search with low CPU impact and fast re-indexing.
  • Improvements in Outlook cached mode operation.
  • Integration with SharePoint folders.
  • New AD schema extensions for resource mailboxes (room or equipment) as well as an all rooms address list.
  • Meeting requests can now be accepted or declined within a preview (no need to open the message).
  • A to-do bar allowing tasks to roll over to the next day if they have not been completed.
  • Ability to view multiple calendars overlaid on one another.
  • An improved scheduling assistant that provides hints to the best times for all meeting attendees as well as tentatively accepting new requests, automatically handling updates to meeting information and removing out of date meeting requests/updates.
  • Calendar sharing is now more granular, with the ability to deny all access, show time and free/busy status only, show time, subject, location and free/busy status, or show full details, even down to a per-user level.
  • Schedules can be set for out of office replies.

From a client perspective, Outlook 2007 provides the richest user experience (when connected to Exchange Server 2007, although it will also work with previous versions); however Outlook Web Access (OWA) is now almost as good. There’s also Outlook Mobile for Windows Mobile 5.0 and the ability to interact with Exchange’s unified messaging functionality from a phone.

Exchange Server 2007 offers significant improvements over 2003 and earlier versions; however it will be interesting to see if these improvements are enough to entice the many companies that are still running basic e-mail services using Exchange Server 5.5 to upgrade their systems.

Exchange Server 5.5-2003 migration gotchas

I’ve not been doing as much work with Exchange Server as I’d like in recent years; so when a friend asked me to help out with carry out an Exchange Server 5.5 to 2003 migration for one of his contacts I was happy to get involved (although I was slightly nervous as this was effectively a refresher course for me being carried out on his production system).

I’m not going to make this a “how to do it” post as I posted an article about migrating from Exchange Server 5.5 to 2003 a couple of years back and one of the areas where the Exchange Server team have really excelled is in the creation of the Exchange Server Deployment Tools which guide an administrator through each step of the process, running diagnostic and setup utilities as they go. For further information, the Exchange Server Deployment Guide is also worth a read.

This article highlights simply some of the issues I came across (on what was a fairly simple migration – Outlook Web Access and Exchange Server 5.5 on two separate servers to a new Exchange Server 2003 server in the same organisation and site) and how to resolve them:

  • The first problem came when Exchange Server setup detected that the installation was being performed on a Windows Server 2003 service pack 1 (SP1) computer (Windows Server 2003 R2 is effectively the same as Windows Server 2003 SP1) and advised that this has known compatibility issues with Exchange Server 2003. After reviewing the Exchange Server system requirements it turned out that it’s not a problem on a non-clustered server if Exchange Server is also running SP1 or later so Exchange Server service pack 2 (SP2) was installed immediately after Exchange Server setup had completed.
  • The Active Directory Connector (ADC) is probably the most difficult part of an Exchange Server 5.5 upgrade but the latest version of ADC includes tools to guide an administrator through the process of creating connection agreements between the Active Directory and Exchange Server directory services and verifying replication. In this case, the ADC Tools highlighted an issue which meant it was necessary to grant Full Control NTFS permissions to the Exchange Server 5.5 service account on the C:\Documents and Settings\username\Local Settings\Application Data\Microsoft\Active Directory Connector folder (as described in Microsoft knowledge base article 820268). Further problems with objects reporting as not replicated (as described in Microsoft knowledge base article 842142) were resolved by reinstalling the ADC, using the version supplied with Exchange Server 2003 SP2.
  • After installing Exchange Server 2003, one shutdown took an extended period of time; however this is a known issue, described in Microsoft knowledge base article 555025.
  • One of the great features of Exchange Server has always been the referral mechanism that allows MAPI clients to update their profiles when a mailbox is moved between servers; however, on this occasion, some Outlook 2003 clients failed to update their MAPI profiles. This is a known issue and is resolved by installing Office 2003 SP2, as described in Microsoft knowledge base article 914855. No such problems were experienced with Outlook 2002 (XP) clients, although the site replication service (SRS) did hang on one occasion and needed to be started before clients could successfully remap their profiles.
  • When accessing Outlook Web Access (OWA), requests to http://exchange2003servername/exchange/ appeared to be diverting to http://exchange55servername/exchange/; however it was later discovered that the referral was only taking place where the currently logged on user (domainname\Administrator in my case) had a mailbox that had not yet been migrated. Once all mailboxes had been moved across, OWA stopped redirecting access.
  • Many Exchange Server 5.5 administrators are used to being able to access all objects (including the contents of other user’s mailboxes) using the Exchange Server service account; however with Exchange 2003, even when an account is delegated Exchange Full Administrator rights over the Exchange organisation it is unable to access other mailboxes as inherited permissions apply an explicit deny over certain rights. This is by design but can be overridden as described in Microsoft knowledge base article 821897 to give an account full access to all objects in a particular store. In this case I delegated Exchange Full Administrator rights to a global security group called Exchange Admins (and added that group into the local Administrators group on the Exchange server), then granted another account full control over all objects in the mailbox store. This mean that I had a group over which the membership could be edited as required to grant rights to administer the Exchange organisation, plus another account (I should really have made this a group too) that could view the contents of other user’s mailboxes.

In all, the migration was reasonably successful, although I do still need to decommission Exchange Server 5.5 (it was left in place to allow the Outlook profiles to update as users log in to the system) and some HTTPS publishing issues with Proxy Server 2.0 need to be resolved before I can call the job complete. In fact, those HTTPS publishing issues turned out to be the cause of much panic on Monday morning when Exchange seemed to be falling down around us. One of the methods we had tried to proxy inbound SSL was using the Winsock proxy client on the Exchange Server as described in Microsoft knowledge base article 184030. Although the SSL proxying hadn’t worked, the Winsock Proxy Client had been left installed on the Exchange Server – it didn’t seem to be causing any issues on Sunday night but by Monday morning the Exchange System Manager and Active Directory Users and Computers administration tools were inaccessible, which Microsoft knowledge base article 325322 suggests is related to a DNS problem. It was purely by chance that I managed to trace this back to the Winsock proxy client (as described in Microsoft knowledge base article 280833) and once this was uninstalled, all services became available.

One final issue left to resolve was to restore access to mailboxes for BlackBerry users, caused by the problems publishing OWA via HTTPS (although any change to the URL used to access OWA externally would have caused this). The resolution was to remove existing account details from users’ Vodafone Mobile E-mail profiles and recreate them using the new address as described in BlackBerry knowledge base article KB-03133.

Finally, for all Exchange Server admins, whether migrating to a new version of Exchange or administering an existing system, there are many tools for Exchange Server 2003 available for download from the Microsoft website.

Configuring and troubleshooting services and applications with Proxy Server 2.0

I’ve spent a lot of time over the last few days struggling to configure a Microsoft Proxy Server 2.0 running on Windows NT 4.0 and Internet Information Server (IIS) 3.0 to reverse proxy (i.e. publish) an HTTPS website. Eventually I had to admit defeat (I’m trying to convince my client to upgrade to ISA Server 2004); however I did find a useful resource for Proxy Server 2.0 information that should be worth a look next time I’m trying to administer/troubleshoot a Microsoft Proxy Server configuration.

Tracking Windows server product licenses

I just had a call from a client who was concerned that he couldn’t add client access licences (CALs ) for his new Exchange Server in the Licensing administrative tool. I’ve never really used this tool so I had to do some research before I could answer his question. Microsoft knowledge base article 824196 describes the license logging service (LLS) but the key points to note are all in the article summary:

  • LLS was originally designed to help administrators manage licenses for Microsoft server products that are licensed on a per-server basis (the server CAL model).
  • LLS was introduced with Windows NT Server 3.51 but it is disabled by default in Windows Server 2003.
  • Because of design constraints and evolving licencing terms, LLS cannot provide an accurate view of the total number of per-user CALs purchased, compared with the total number of CALs that are used on a single server or across the enterprise.
  • LLS will not be included in future versions of the Windows operating system.

Basically, it seems that LLS is a hangover from Windows NT and nowadays there is no real reason to use it in Windows Server 2003.