Warming up SharePoint

Posted on By Mark Wilson

I’m spending a few days this week learning about how to implement an enterprise search solution using Microsoft Office SharePoint Server (MOSS) 2007 (so expect a few SharePoint-related posts to follow over the next few days). I was intrigued to note that the Virtual PC image used for the training course included a number of “warm up” scripts for SharePoint. Not having come across this before, I was intrigued as to their purpose but my instructor (Martin Harwar from English Tiger) explained that the principle is equally applicable to all ASP.NET applications and is about improving the end user experience.

Normally, ASP.NET applications are compiled on a just in time (JIT) basis and this means that the first page load after making changes such as resetting IIS, refreshing application pools or editing web.config can be very slow. By “warming up” the application (accessing key pages deliberately), JIT compilation is triggered, meaning that when an end user accesses the page then it is already compiled (and hence fast).

Joel Oleson’s SharePoint Land has a much better explaination as well as the actual MOSS warmup scripts for download. There are also some more comments on warming up SharePoint at Andrew Connell’s SharePoint developer tips and tricks site.

Incidentally, the scripts on Joel’s site are intended for MOSS but can be modified for WSS (and I understand that the principle may be equally applicable to other ASP.NET web applications).

Outsourcing syndicated content from WordPress to Feedburner without losing readers

Posted on By Mark Wilson

Earlier, I wrote about some of the measures I’ve taken to reduce the bandwidth usage of this site, one of which is outsourcing the RSS feeds to FeedBurner (i.e. put them on Google’s bandwidth bill!).

The new feed location for syndicated content using either RSS or Atom is http://feeds.markwilson.co.uk/marksweblog/.

Hopefully, I’ve done everything that I need to to make sure that no-one has to make any changes in their feedreader – yesterday’s 176% growth in subscribers (according to the Feedburner stats, which are now picking up the traffic that was previously split across multiple feeds) certainly suggests that it’s all working!

FeedBurner FeedStats showing significant increase after consolidation of feeds

If all you want is the new address then there’s no need to read on; however as this is a technical blog, I thought that some people might be interested in how this all works.

Firstly, the feeds from the old Blogger version of this site (https://www.markwilson.co.uk/blog/atom.xml and https://www.markwilson.co.uk/blog/rss.xml) have permanent redirects (HTTP 301) in my .htaccess file to redirect clients to the equivalent WordPress locations. This has been working since the migration to WordPress back in March.

I’ve had a FeedBurner feed at http://www.feedburner.com/marksweblog/ for a few years now and this remains in place. It’s using FeedBurner’s SmartFeed technology to translates the feed on-the-fly into a format (RSS or Atom) compatible with the visiting client. Since FeedBurner have made their MyBrand service free, I’ve set up feeds.markwilson.co.uk as a DNS CNAME record, pointing to feeds.feedburner.com so basically http://www.feedburner.com/marksweblog/ and http://feeds.markwilson.co.uk/marksweblog/ are interchangeable (although there is no guarantee that I will always use FeedBurner, so the http://feeds.markwilson.co.uk/marksweblog/ address is preferable).

Because I needed to make sure that anyone using the standard WordPress feed locations listed below would be redirected to the new feed, I used the FeedBurner FeedSmith WordPress plugin to redirect readers to http://feeds.markwilson.co.uk/marksweblog/ from any of the following:

https://www.markwilson.co.uk/blog/feed/
https://www.markwilson.co.uk/blog/feed/atom/
https://www.markwilson.co.uk/blog/feed/rdf/
https://www.markwilson.co.uk/blog/feed/rss/
https://www.markwilson.co.uk/blog/feed/rss2/

For the time being, the per-post comment feeds are unchanged (very few people use them anyway).

The really smart thing that FeedSmith does is to redirect most clients to FeedBurner except if the user agent indicates that the request is from FeedBurner, in which case access is provided to the syndicated content from WordPress. This is shown in the extracts below from the logs offered by my hosting provider:

HTTP 307 (temporary redirect)

This request (from an Internet Explorer 7 client) receives a temporary redirect (HTTP 307) as can be seen in the results from the SEO Consultants check server headers tool:

SEO Consultants Directory Check Server Headers – Single URI Results
Current Date and Time: 2007-09-13T15:22:18-0700
User IP Address: ipaddress

#1 Server Response: https://www.markwilson.co.uk/blog/feed/
HTTP Status Code: HTTP/1.1 307 Temporary Redirect
Date: Thu, 13 Sep 2007 22:22:03 GMT
Server: Apache/1.3.37 (Unix) mod_fastcgi/2.4.2 mod_auth_passthrough/1.8 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.4.7 FrontPage/5.0.2.2635.SR1.2 mod_ssl/2.8.28 OpenSSL/0.9.7e
X-Powered-By: PHP/4.4.7
Set-Cookie: bb2_screener_=1189722124+216.154.235.143; path=/blog/
X-Pingback: https://www.markwilson.co.uk/blog/xmlrpc.php
Last-Modified: Thu, 13 Sep 2007 21:40:23 GMT
ETag: “d7e58019e9dbb9623c54b0721b0e1f3c”
Location: http://feeds.markwilson.co.uk/marksweblog
Connection: close
Content-Type: text/html

HTTP 200 (OK)

Meanwhile FeedBurner receives an OK (HTTP 200) response and is served the full feed. The advantage to me is that each visitor who receives a redirect is served just 38 bytes from this website whereas the full feed (which varies in length according to the blog content) is considerably heavier (over 17KB based on the example above).

So far the most visible advantage to me is that I’ve consolidated all syndication into a single feed, upon which I have a variety of services running (or available). The as yet unseen advantage is the consequential reduction in the bandwidth taken up with syndicated content – with some feedreaders polling the feed several times a day, that should be a considerable saving.

Attempting to reduce my website’s bandwidth usage

Posted on By Mark Wilson

This website is in a spot of trouble. Over the last few months, I’ve seen the bandwidth usage grow dramatically although it seems to have grown faster than the number of subscribers/readers. We’re not talking vast volumes here and my hosting provider has been very understanding, but even so it’s time to do something about it.

So I had a think, and came up with three options:

  1. Don’t write anything. Tried that for half of June (when I was on holiday). No noticeable change in the webstats!
  2. Write rubbish. It’s debatable as to whether that’s a continuation of the status quo.
  3. Drp ll th vwls s f wrtng txt mssgs.
  4. Optimise the site to reduce bandwidth usage, without a major rewrite. Yeah! That sounds like a challenge.

So, option four sounded like the best course of action. There are two main elements to consider in this:

  1. Site performance (i.e. how fast pages load).
  2. Bandwidth usage.

As far as I can tell, my site performance is not blindingly fast but it’s OK. I could use something like the WP-Cache plugin to cache content but, although that should reduce the load on the server, it won’t actually decrease my bandwidth usage. In fact it might increase it as I’d need to turn off HTTP compression.

That led me to concentrate on the bandwidth issues. This is what I tried (based mostly on Jeff Atwood’s experience of reducing his site’s bandwidth usage):

  • Shut out the spammers. Akismet had blocked over 7000 spam messages in 15 days and each of these would have loaded pages and leeched some bandwidth in the process. Using the Bad Behaviour plugin started to reduce that, blocking IP known spammers based on their IP address. Hopefully it hasn’t blocked legitimate users too. Please let me know if it has blocked you (assuming you can read this!).
  • Compress the content. Check that HTTP compression is enabled for the site (it was). According to Port 80 Software’s real-time compression check, this both reduces my file size by about 77% and decreases download times by 410%. It’s also possible to compress (i.e. remove whitespace and comments) in CSS and JavaScript (as well as tools for HTML compression) but in my opinion, the benefits are slim (as these files are already compressed with HTTP compression) and code readability is more important to me (although at 12.7KB, my main stylesheet is a little on the bloated side of things – and it is one file that gets loaded frequently by clients).
  • Optimise the graphics. I already use Adobe Photoshop/ImageReady to save web optimised graphics but I used a Macintosh utility that Alex pointed me to called Ping to optimise the .PNG files that make up about half the graphics on this site (I still need to do something with the .JPGs and .GIFs) and that shaved just over 10% off their file size – not a huge reduction but it should help.
  • Outsource. Switching the main RSS feed to FeedBurner made me nervous. I’d rather have all my readers come to my domain than to one over which I have no control but then again FeedBurner gives me some great analysis tools. Then I found out about Feedburner’s MyBrand feature (previously a chargable option but free since Feedburner was acquired by Google) which lets me use feeds.markwilson.co.uk (i.e. a domain under my control) instead of feeds.feedburner.com. Combined with the FeedSmith plugin, this has let me keep control over all of my feeds. One more option is to use an external image provider (Jeff Atwood recommends Amazon S3 but I haven’t tried that yet).

At the moment it’s still early days but I do have a feeling that I’m not eating up my bandwidth quite as quickly as I was. I’ll watch the webstats over the coming days and weeks and hope to see a downward trend.

Working around AOL’s short-sighted antispam measures

Posted on By Mark Wilson

For the last year or so, I’ve been running my own mail server. This provides me with a number of advantages:

  • I have no storage limits (other than the physical limits of my hardware).
  • I have complete control over the mail server configuration.
  • I’m not at the mercy of my ISP’s e-mail server issues and delays if the mail queues get full
  • I can use my home system to try out new features and functionality.

It’s been working well. A few newsletters stopped arriving after I installed the Exchange Server intelligent message filter (IMF) and there is no way to whitelist addresses or subjects with the IMF that I’m aware of but, together with the realtime block lists that I use, it generally does a good job of trapping spam with only a few false positives.

I’ve been wondering for a while why there always seemed to be a problem with e-mailing AOL users but it’s such a rare occurrence that I didn’t get too hung up on it. Then I needed to send something over to someone and it bounced back (actually, AOL just refused to accept a connection from my mail server), so I started to look into the problem more closely.

I checked out AOL’s postmaster best practice guidelines and there didn’t seem to be a problem at first. I use a combination of Microsoft Exchange Server 2003 and Entourage 2004 so the mail should be RFC-compliant; I verified the connecting IP address (using AOL’s own tools); I checked that I have a valid reverse DNS entry (again, using AOL’s own tools); my mail server is not operating as an open relay; there were no links (let alone invalid ones) in the bounced messages and I have a static IP address on my business ADSL connection.

Having checked all of AOL’s technical guidelines for whitelisting I applied to join the AOL whitelist using AOL’s spam feedback form. AOL’s postmaster replied with the following:

AOL does not accept email sent directly from dynamic IPs. If you have a static IP, please contact your ISP and have them send us an updated list of dynamic and static IP ranges. If you are on a dynamic IP, please send through your ISP’s mail server.

It seems that the problem is my IP address. AOL’s senders’ FAQ states that:

Customers with residential IP addresses should use the provider’s SMTP servers and should not be sending email directly to another ISP’s SMTP servers.

I do have a static IP address and a business account (i.e. it is non-residential) but that’s not the point – AOL has the audacity to prevent anyone mailing them from an IP address that they have recorded on their database as residential! Admittedly my configuration is not normal but why should AOL dictate how I provide my e-mail service? I do understand that much of the world’s spam will originate from zombie-infected PCs on people’s home networks (i.e. residential) but I have an SPF record implemented in order to verify that e-mail purporting to originate from my mail server really is from my server. In any case, AOL’s reason for blocking me was nothing to do with authenticating my e-mail server but was purely based on their assertion that I have a residential IP address – something that doesn’t seem to bother any other mail hosting provider.

Not really wanting to get into the situation where my ISP says it’s AOL’s problem and AOL says it’s up to my ISP, I decided to work around the problem through reconfiguring Exchange Server:

  1. Firstly, I changed the cost on my existing SMTP connector (set to use DNS to route e-mail and using an address space of *) from 1 to 2.
  2. Next, I created a new SMTP connector for mail to be forwarded to my ISP’s relay, gave this a lower cost (1) and added aol.com to the address space.

Now e-mail for anyone@aol.com will be sent using my ISPs servers and all other external e-mail will go directly based on the MX records that are specified for the recipient’s domain. Amset IT solutions have a page on their website which explains the configuration in detail.

I needed a quick way of testing the message flow, so I sent a test message via my mail server to my e-mail address at work. Checking the headers on receipt showed that it had gone straight from my server to my employer’s e-mail gateway. Next, I added the domain name for my work e-mail address to the address spaces on the connector for e-mail to be routed via my ISP and repeated the test. Again, checking the headers verified that the message had indeed passed through my ISP’s relay. Finally, I took my employer’s domain name out of the address space on the new connector and verified that e-mail was directly routed once more.

It’s not difficult but it is a further complication in my mail server configuration, just to satisfy the requirements of one (admittedly large) ISP …and just one more reason for me to cringe when I hear that someone is an AOL subscriber.

Mounting virtual hard disks in Windows Vista

Posted on By Mark Wilson

Microsoft’s Virtual PC Guy (Ben Armstrong) wrote a blog post last year about using the VHDMount utility from Virtual Server 2005 R2 SP1 with a few registry edits to enable right-click mounting/dismounting of virtual hard disk (.VHD) files.

As .VHD files become ever more prevalent, this is a really useful capability (for example, Windows Vista’s Complete PC Backup functionality writes to a .VHD file).

The trouble is that, as supplied, Ben’s script does not work on Windows Vista as attempting to run vhdmount.exe will return:

Access Denied. Administrator permissions are needed to use the selected options. Use an elevated command prompt to complete these tasks.

An elevated command prompt is fine for entering commands directly (or by running a script) but what about Ben’s example of providing shell-integration to mount .VHDs from Explorer? Thankfully, as Steve Sinchak noted in TweakVista, Michael Murgolo wrote an article about elevating commands within scripts using a free PowerToy called elevate which is available from the Microsoft website. After downloading and extracting the elevate PowerToy scripts, I was able to confirm that they would let me run vhdmount.exe using the command elevate vhdmount.exe

Following that, I edited Ben Armstrong’s registry file to read:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD]

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD\shell]
@="Mount"

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD\shell\Dismount]

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD\shell\Dismount\command]
@="\"C:\\Program Files\\Script Elevation PowerToys\\elevate\" \"C:\\Program Files\\Microsoft Virtual Server\\Vhdmount\\vhdmount.exe\" /u /d \"%1\""

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD\shell\Mount]

[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Virtual.Machine.HD\shell\Mount\command]
@="\"C:\\Program Files\\Script Elevation PowerToys\\elevate\" \"C:\\Program Files\\Microsoft Virtual Server\\Vhdmount\\vhdmount.exe\" /p \"%1\""

[HKEY_CLASSES_ROOT\.vhd]
@="Virtual.Machine.HD"

Note the /d switch in the dismount command. I had to use this (or /c) to allow the disk to be unmounted and avoid the following message:

The specified Virtual Hard Disk (VHD) is plugged in using the default Undo Disk option. Use /c to commit or /d to discard the changes to the mounted disk.

I chose the discard option as most of my .VHDs mounting is simply to extract files but others may prefer to commit.

A few more points to note about VHDMount:

A few Live Meeting tips

Posted on By Mark Wilson

I’ve just spent the last couple of hours listening/watching a Live Meeting webcast. In recent weeks I’ve found that I’m attending more and more of these as part of the various Microsoft beta and technology adoption programmes that I’m participating in and frequently I need to take notes using Microsoft Office OneNote 2007 on the same PC that I’m using to view the slides and listen to the audio. Today I decided to try and connect to the webcast simultaneously from my Mac (i.e. using a second computer to view the slides whilst I write notes on the first) and I’m pleased to say that it worked using Microsoft Office Live Meeting Web Access (unfortunately the full Live Meeting client is required for VOIP audio but all I needed to do in this case was view the slides).

Although Live Meeting supports a pretty wide selection of browser and Java VM combinations, Firefox 2.0 on Mac OS X is not a supported browser/platform combination – the workaround is to use Safari and Apple Java (at least v1.4.1).

Here’s some of the advice and guidance that I’ve accumulated as I’ve been working on local (via Microsoft Office Communications Server 2007) and hosted Live Meeting calls over recent weeks (this is just what I’ve found and is not a comprehensive list):

  • The Microsoft Office Live Meeting 2007 Client can be downloaded from Microsoft Office Online.
    If the link in the meeting invitation doesn’t work, try launching the client and entering the details manually.
  • If colleagues can’t hear you on the meeting, check that your microphone is unmuted (the default is muted), that (if you are using a webcam) the microphone is close enough to pick up your voice and don’t assume that your notebook computer has a built-in microphone (this one stumped me for a while until I plugged in a microphone and everything jumped into life)!
  • (Microsoft Connect users may find the Live Meeting audio issues FAQ useful.)
  • The Live Meeting support website features a knowledge base for troubleshooting issues with Live Meeting.

Installing Microsoft Dynamics CRM without domain administrator rights

Posted on By Mark Wilson

I recently inherited the task of designing the infrastructure for a Microsoft Dynamics CRM 3.0 implementation. After being briefed by the consultancy partner that we are using for the application customisation and reading Microsoft’s implementation guide I was fairly comfortable with the basic principles but I was also alarmed that the product seems to require installation to be carried out using an account with Domain Admins permissions. There’s no way that I will be granted those rights on our corporate Active Directory (and nor should I be) – too many applications seem to require elevated permissions for service accounts and it makes life very difficult when trying to define a delegation policy for Active Directory administration.

Regardless of the assurances I was given that Domain Admins rights are only required to carry out the installation (and subsequent updates) and that the account can be relegated to a standard domain user afterwards, I felt that there must be a way around this – surely the groups that the CRM installation creates can be pre-staged somehow, or that a organizational unit can be created with delegated rights to create and manage objects?

It seems the answer to my question is yes – I’ve now been pointed in the direction of Microsoft knowledge base article 908984 which describes how to install Microsoft Dynamics CRM 3.0 as a user who is not a domain administrator by using the minimum required permissions.

Unconditionally contactable – no thanks.

Posted on By Mark Wilson

This last week has been manic – hence the lack of blog posts… taking a day’s annual leave on Monday and then spending half of it catching up on my administration didn’t bode well, then there were two nights when I was up until 1am trying to write an infrastructure design document and the usual mix of travel, conflicting meeting requirements and trying to get some “real work” done.

“But Mark”, I hear you ask,”surely you use some of the technologies that I see you write about to improve productivity?”

The answer is that I do – I’m using Microsoft Office Communication Server 2007, the Office Communicator client 2007 and Live Meeting 2007 a fair bit – as well as our corporate conferencing service. Soon I’ll be linking all of that in to my voice mail to make use of Exchange Server 2007’s unified messaging capabilities. It’s a really good solution (especially when Communicator reads my calendar or Live Meeting status and sets my presence accordingly). But the technology is no panacea: sometimes something doesn’t work – I spent quite a bit of time this week waiting on a Live Meeting call as Microsoft struggled to get the audio working (they later postponed that particular meeting as even they couldn’t fix it); and other times there is no substitute for getting together in a room – like my main meeting on Friday which necessitated 4 hours travel (which could have been better spent doing something else) but resulted in the production of a migration strategy for a key customer’s messaging infrastructure – something which we had failed to do several times over the phone (and which I doubt even advances in video conferencing would have helped with).

As someone who struggles at times with information overload, and who was described by a friend and ex-colleague as “[sometimes] exhibiting workaholic tendencies”, I need to help myself to become more productive. As I already have a pile of books by the bedside, it’s probably time for an audiobook or two on Getting Things Done (or at least to check out 43 Folders from time to time).

As for unified communications (UC), Dave Bailey wrote an interesting comment for IT Week on the difficulties of getting away from it all – it was only a few days previously that, as I was busily IMing one contact, another team member started e-mailing me on the same subject and I had Outlook “toast” popping up as fast as I could type. Then I spent half of Friday afternoon this week reducing the size of my mailbox so that I could get below the system limits and send mail again (there is one simple answer – the delete key… but that’s not exactly productive either). As my colleague pointed out, it seems that UC really stands for “unconditionally contactable”. No thanks.

iPod Touch… or unlocked iPhone?

Posted on By Mark Wilson

Yesterday afternoon, Garry walked into the office and asked, “so, have you seen the new iPod then?”. I hadn’t and, after waiting and waiting, then finally getting a 5.5G iPod with video just a few months back I promptly stuck my fingers in my ears and said something like “na-na-na-na I can’t hear you” until he finished telling me about it. Later, when I was feeling a bit more rational, I checked it out and, apart from only having 16GB of storage (I’m presently struggling with 30GB) it’s looking pretty sweet – almost everything I’d asked for and more.

In his interview alongside Bill Gates at D5, Steve Jobs said that Apple was working on something special and he was not kidding. There is now an iPod for everyone, from the tiny (and low-cost) Shuffle, to a Nano with Video, the iPod Classic (now with 160GB on board… I had already been considering a hard disk upgrade for my 30GB), the Touch and the iPhone. Then there is Apple’s iTunes Wi-Fi Music Store tie-up with Starbucks (which is not limited to the iPod and iPhone, although it does appear to be limited to United States metropolitan areas for the time being with no mention of a global rollout any time soon).

But the more I think about it, the less I want the iPod Touch. It may be (almost) everything I wanted and I did previously dismiss the iPhone (the people at The Register were happy to hand theirs back) but now it appears to have been hacked and the price has dropped to $399 (which should translate to around £200 here in the UK, except it won’t, because we have extortionate 17.5% sales taxes and US products never sell here at anything like the exchange rate comparison price) – based on the pricing of other Apple products, I’d expect to see the iPhone sell in Europe at around €399 or £275. I won’t buy an iPhone with an O2 contract when it finally reaches these shores but if I was to unlock one and use my Vodafone SIM… (or just buy one in France, where Apple will be legally obliged to unlock it for me) now that would be good.

Not sure that it would go down too well in the office though…

Nikon finally releases a full-frame DSLR

Posted on By Mark Wilson

NikonWhen I switched my photography to a digital format I had too much invested in Nikon lenses to change manufacturers; however I do know that many professionals using DSLRs today have Canon equipment to make the most of a full-frame image sensor (and I bought my best Nikkor lens from a pro who made the switch to Canon). For years, Nikon has told us that their DX format is all we need (and the focal lengths of our lenses have grown by around 150% as a result) but this has a cost in the higher signal to noise ratio. It seems that, finally, Nikon has recognised this with their new “FX” format D3 (which can also use DX lenses, although it shuts down part of the image sensor to do so). At the wrong side of £3000 it’s too much for me but I like the sound of features like a self-cleaning sensor, dual CF-card slots, LiveView and virtual horizon adjustment. I also like the sound of a 12 megapixel sensor, 51 point autofocus, 3″ LCD and larger viewfinder (something I can only experience by shooting some rolls of film in my old F90x) although I’d like to see low ISO settings too (when shooting landscapes I find ISO 200 too restrictive and would like the option for slow-speeds in the ISO 25-100 range) and the Canon EOS 1Ds Mark III already meets (or even exceeds) most of the D3 specifications (admittedly with a £6000 price tag). I had hoped that a prosumer FX format model would follow the D3 but it seems not – the new D300 is DX only. Regardless, I’d better get saving to replace my D70, which is still a fantastic camera but not quite in the same league – maybe by the time my piggy bank is full there will be a prosumer model with an FX sensor.

Rob Galbraith has a full review of the D3 at digital photography insights.