Inside the Microsoft datacentres

A datacentre is just a datacentre isn’t it? After all, isn’t it just a bigger version of the server room in the basement? But what about the huge datacentres that run cloud services? What’s it like inside the Microsoft datacentres that host Azure, Office 365, etc.?

Last week, Microsoft’s Modern Workplace webcast titled “An Inside Look at Your Secure Cloud” gave a sneak peek inside some of the Microsoft datacentres – comparing various generations and showing the improvements along the way.  And, as you might expect, these are the very definition of operating at scale…

As Doug Hauger (General Manager for National Cloud Programs at Microsoft) explained, organisations look to use a cloud datacentre for scale and professionalism.  Anyone can run a datacentre but the Microsoft Cloud is about robustness and security – whether that’s how staff are monitored or the physical and logical security models.

Each time Microsoft moves into a new region (like the two regions that opened in the UK earlier this month) there’s not just one super-scale datacentre but multiple facilities per region, providing redundancy and disaster recovery capability. Each facility has multiple power sources and multiple network ingress and egress points. Then there’s the investment Microsoft is making in physical infrastructure around the world – for example the joint project with Facebook for a new Europe-North America undersea cable (MAREA).

Each time Microsoft considers expanding into a new market they perform a business case analysis on the potential opportunity, considering the scale that they will go in at (tens of thousands of servers). Microsoft now has more than 100 datacentres in 30 regions around the world (with four more under construction). Because of the huge range of locations covered, Microsoft is now the industry leader for compliance and certification – whether that is meeting global or local requirements. Then there is the question of meeting customer needs around data residency, compliance, etc. (for example with the German datacentres that operate under a unique data trustee model in partnership with Deutsche Telekom).

With its cloud datacentres, Microsoft is aiming to meet customer needs around digital transformation, where the question is no longer “why should I go to the cloud” but one of “how to innovate more quickly in the cloud”. That’s what drives the agenda for where to geographically expand, where enhance scalability, etc.

Despite the question I posed in the opening paragraph of this post, a true datacentre is worlds apart from the typical server room in the basement (or wherever). The last time I got to visit a datacentre was when I was working at Fujitsu and I visited the London North facility, an Uptime Institute Tier III datacentre that won awards when it was built in 2008. Seeing the scale at which a modern datacentre operates is impressive. Then ramp it up some more for the big cloud service providers.

In the webcast, Christian Belady (General Manager Cloud Infrastructure Strategy and Architectures at Microsoft) explained that datacentres are the foundation of the Internet – they are where all the cloud services are served from (whether that is Microsoft services, or those provided by other major players).

There are several layers of physical security from the outside fence in, screening people, controlling access to parts of the buildings, even to cabinets themselves with critical customer data in locked cabinets covered with video surveillance. Used disks are destroyed, being wiped and then crushed on site! The physical security surpasses anything provided for on-premises servers and the logical security continues that defence in depth.

Each custom-built server is actually 2 computers with 10s of 1000s of computers per room, 100s of 1000s per datacentre, each datacentre the size of 20-30 football fields. Look at the racks and you can see the attention to detail – keeping things orderly not only adds to operational efficiency but it looks good too! The enterprise servers that most of us run on-premises have plastic bezels to make them look pleasant. Instead, Microsoft’s servers have focused on eliminating anything that has no useful function…

Each iteration of datacentres becomes more industrialised – with improvements to factors such as cooling (which is one of the biggest power usage factors).

A generation 2 datacentre from around 2007 has a Power Usage Effectiveness (PUE) efficiency score of 1.4-1.6 (for comparison, the Fujitsu facility I mentioned earlier has a PUE of 1.4 but a typical enterprise datacentre from the 2000s with a normal raised floor would have a PUE of 2-3). Cool and hot aisles are used with hot air returned to coolers and recirculated. Microsoft then raised the temperature of their servers to a level that is acceptable (working with manufacturers), rather than the lower levels they used to have (reducing the cooling demands).

Moving on to generation 4, efficiency is improved further (a PUE of 1.1-1.2), eliminating chillers by removing roofs, driving down costs and using outside air to chill. Containers use the outside cooling and a system of adiabatic cooling, spraying mist into the air to cool down – which evaporates before it hits the server”. Such datacentres use a lot less water too (compared with older styles of datacentre).

With the latest (generation 5) datacentres, further improvements are made, culminating the features of other generations – learning and adapting. The PUE is now down to 1.1 (and below at certain times of year) with running costs also improved. There are still hot a cold aisles but no raise floor and, instead of outside air, the datacentres use a closed liquid loop system (no chiller – cool the water outside) – and that water doesn’t need to be potable.

The actual datacentre design changes for each facility, based on the geography and the environmental impact. Backup power generation is a key component in the design, with several days of fuel onsite and contracts to keep bringing more fuel in. Power is often sustainably sourced, be that cheap and carbon-free hydro-electric power, wind or solar. Microsoft Research is even working on a tidal-powered under-sea datacentre (Project Natick).

Inside the Microsoft datacentres is very industrial. Whole racks are brought in (pre-tested), rather than single servers and, as previously mentioned, Microsoft design and build the servers for use at scale, stripping out enterprise features and retaining only what’s needed for the Microsoft environment.

Whilst I’ve worked with customers who have visited Microsoft datacentres in Dublin, it seems unlikely that I’ll ever get the chance. Watching the Modern Workplace webcast gave me a fascinating look at how Microsoft operates datacentres at scale though – and it truly is awe-inspiring. To find out more, visit the Microsoft website.

Microsoft’s UK datacentres: what you need to know

This morning, the UK woke up to an announcement from Microsoft that the UK datacentres for Azure and Office 365 are generally available, making Microsoft the first global provider to deliver a complete cloud (Iaas, PaaS and SaaS) from UK data centres.

That means:

  • Two new Azure regions in the UK:
    • UK West (Cardiff)
    • UK South (London)
  • Office 365 services from UK datacentres in Durham and London.

Dynamics CRM online will be offered from the UK in the first half of 2017.

That Azure location information was taken from the Azure regions page on the Microsoft website (although my sources tell me that “Cardiff” is really “Newport” – close enough as to make no difference anyway, and London is probably “near London” too).  The Office location information was taken from the Office 365 Interactive Data Maps.

Now, UK customers already using Azure or Office 365 will be asking “will my data be moved to a UK datacentre?”. There’s no official announcement from Microsoft (not that I’ve seen) but my (unofficial) answer is “no”. At least not automatically.

For Azure, it’s good practice to design across multiple regions. There are also implications around geo-replication (which regions are paired with which for business continuity and disaster recovery purposes). Moving resources from one region to another is possible but is also a project that would need to be undertaken by a customer (possibly working with a partner) as a programme of planned resource moves.

For Office 365, it’s worth reading the TechNet advice on Moving core data to new Office 365 datacenter regions. At the time of writing it hasn’t been updated to reflect UK datacentres (it was last updated 28 July 2016) but it currently says:

“Existing customers that have their core customer data stored in an already existing datacenter region are not impacted by the launch of a new datacenter region”

[…]

“The data residency option, and the availability to move customer data into the new region, is not a default for every new region we launch. As we expand into new regions in the future, we’ll evaluate the availability and the conditions of data moves on a region by region basis.”

“New customers or Office 365 tenants created after the availability of the new datacenter region will have their core customer data stored at rest in the new datacenter region automatically.”

The page goes on to state that, assuming the data residency option is made available for the UK (remember, nothing has been announced yet)

“Customers will need to request to have their data moved within a set enrollment window.”

and that:

“Data moves can take up to 24 months after the request period to complete”

There’s also a footnote on the UK interactive data map to say:

“Customers who signed up and selected the United Kingdom for their Office 365 services before September 2, 2016 will have their customer data located in the EMEA datacenter locations.”

So, in short, Office 365 (SaaS) data stays exactly where it is, unless you sign up for a new tenant, or wait for further announcements from Microsoft. Azure (IaaS and PaaS) workloads can be moved to the new regions whenever you are ready.

 

End user computing – the device doesn’t matter

Following a recent Windows update that “went bad”, I needed to have my work PC rebuilt.  That left me with a period when I had work to do, but only a smartphone to work on or my personal devices. To me, this was also a perfect opportunity to put cloud services to work.

So, armed only with a web browser on another PC, I was perfectly able to access email and send/receive IMs (it’s all in Office 365), pester people on Yammer, catch up on some technical videos, etc. There was absolutely nothing (technically) preventing me from doing my job on another device. That’s how End User Computing should work – providing a flexible computing workstyle that’s accessible regardless of the device and the location.

The real issues are not around technology, but process: questions were asked about why I wasn’t following policy and using my company-supplied device; and I was able to answer with clear reasons and details of what I was doing to ensure no customer information was being processed on a non-corporate device. There are technical approaches to ensuring that only approved devices can be used too – but what’s really needed is a change of mindset…

Retired: Mark’s Office 365 Resource Centre

18 months ago, I created “Mark’s Office 365 Resource Centre” using the public site from my Office 365 subscription. Over the last few months it’s fallen by the wayside as my focus has recently moved towards Azure (and Office 365 public websites are a deprecated feature) so I decided to move it here.  This content is no longer maintained, but may still contain some useful links.

New functionality

See also Office 365 Updates on the Office Blogs.

Planning and Deployment resources

Technical resources

Training and certification

Licensing

To license Office 365, costs are provided on the Microsoft Online Services Customer Portal. These are ordered and paid for directly by customers (although trial tenants may be created by partners though the
FastTrack portal). Customers with Enterprise Agreements have additional options including not just the Office 365 plans but ‘add-on’ and ‘bridge’ licenses for on-premises Office and CAL Suites.

Information for partners

Tips, tricks and more from my blog

There are a few posts missing from this list, because I consider them to be out-of-date (although they are still available):

Useful to know

(Including tips and tricks from elsewhere on the web):

Bits and bytes (downloads)

Short takes: ADFS certificate expiry; Azure Authenticator setup on Windows Phone; checking if a MSOL tenant name exists

Some more snippets of randomness pulled together to make a blog post…

ADFS certificate expiry

One of my colleagues spotted this in a customer’s Office 365 tenant recently:

Office 365 - Renew your certificates

Thankfully, it wasn’t one we were managing… but I did feel the need to flag it to the incumbent service provider. If this happens to you, my colleague Gavin Morrison (@GavinMorrison) flagged a potentially useful blog post from Jack Stromberg about renewing ADFS Certificates.

Azure Authenticator Setup on Windows Phone

Whilst setting up additional authentication for Office 365 (in effect, Azure AD MFA) I found that I couldn’t add an account until the Windows Phone Azure Authentication app had enabled push notifications. Despite repeatedly enabling it in Settings, completing setup of the account needed a phone reboot, at which point it was ready for me to scan a QR code and continue.  Even then the option to allow notifications doesn’t seem to stick!

Checking if a Microsoft Online Services tenant name exists

My colleague Gareth Larter found a neat trick this week for checking if a Microsoft Online Services (MSOL) tenant exists (e.g. for Office 365).

Gareth’s advice is to browse to https://login.windows.net/tenantname.onmicrosoft.com/FederationMetadata/2007-06/FederationMetadata.xml and, if you get an error, it should show “No service namespace named ‘tenantname.onmicrosoft.com’ was found in the data store” at the bottom right meaning that the tenant name is available:

On the other hand, if you get a bunch of XML data returned, then that tenant already exists.

“Unlicensed Product” errors in Microsoft Office

Earlier this evening, I noticed that my copy of Outlook was showing as an “unlicensed product” at the top of the screen. That seemed strange, as I pay for an Office 365 Home subscription, which covers my family’s various copies of Office.

Outlook reports intself as an unlicensed product

So, I took a look at the Office Account settings, and noticed that it wasn’t signed in to Office 365 for some of the connected sources.

Disconnected from Office 365 services

I reconnected to My Office 365 subscription, signing in with my “Work or school account” as that’s what the markwilson.it Office 365 subscription uses, even though the Office 365 Home subscription uses a Personal Account (formerly known as a Microsoft Account):

Which Microsoft account to use?!

After authenticating (and a restart), Outlook was no longer complaining about being unlicenced.

I’m not sure if it’s a complication of having both a Microsoft Account (MSA) and an Organization/Work and School (Azure AD) account with the same email address, but it seems there are various scenarios that can present this issue.

Thankfully this one wasn’t too hard to sort out!

Why Microsoft customers don’t need to worry about EU-US Safe Harbour/Harbor

When European Courts judged the 15-year-old EU-US Safe Harbour/Harbor treaty to be invalid last October, Internet news sites started to report how terrible this was for EU companies placing data into cloud services offered (mostly) by American companies. For some, that may be true, but that assumes Safe Harbour is the only protection in place.

This week, IT news sites are at it again. The Register (the tabloid newspaper of IT news sites) has an article titled Safe Harbor 2.0: US-Europe talks on privacy go down to the wire but the actual URI belies a much more dramatic title of “Safe Harbor countdown to Armageddon”. Sensationalist at best, some might even say irresponsible.

I’m no lawyer but, for my customers, who are implementing Microsoft cloud services, there seems to be nothing to worry about and I’ll explain why in this blog post. Of course, Microsoft is just one of many cloud services providers – and for others there may be valid concerns.

The United States Export.Gov website currently displays the following text regarding Safe Harbor:

“On October 6, 2015, the European Court of Justice issued a judgment declaring as ‘invalid’ the European Commission’s Decision 2000/520/EC of 26 July 2000 ‘on the adequacy of the protection provided by the safe harbour privacy principles and related frequently asked questions issued by the US Department of Commerce.’

In the current rapidly changing environment, the Department of Commerce will continue to administer the Safe Harbor program, including processing submissions for self-certification to the Safe Harbor Framework. If you have questions, please contact the European Commission, the appropriate European national data protection authority, or legal counsel.”

EU Model Clauses trump Safe Harbour

Microsoft President and Chief Legal Officer, Brad Smith, issued a statement on 6 October 2015. Quoting from that article:

“For Microsoft’s enterprise cloud customers, we believe the clear answer is that yes they can continue to transfer data by relying on additional steps and legal safeguards we have put in place. This includes additional and stringent privacy protections and Microsoft’s compliance with the EU Model Clauses, which enable customers to move data between the EU and other places – including the United States – even in the absence of the Safe Harbor. Both the ruling and comments by the European Commission recognized these types of steps earlier today.

Microsoft’s cloud services including Azure Core Services, Office 365, Dynamics CRM Online and Microsoft Intune all comply with the EU Model Clauses and hence are covered in this way.”

There’s also a follow-on post which talks in general terms about the wider issues and privacy beliefs but the key point is that Microsoft offers EU Model Clauses within its contracts, which go beyond Safe Harbour. Microsoft also has an FAQ on the EU Model Clauses that is worth a read.

Quoting again from the 6 October 2015 statement:

“We wanted to make sure all of our enterprise cloud customers receive this benefit so, beginning last year, we included compliance with the EU Model Clauses as a standard part of the contracts for our major enterprise cloud services with every customer. Microsoft cloud customers don’t need to do anything else to be covered in this way.”

That suggests to me that customers who have signed up to Azure Core Services, Office 365, Dynamics CRM Online or Intune since early 2014 already have greater privacy protection than was afforded by Safe Harbour – and that protection meets the EU’s current requirements. In short, Microsoft customers don’t need to worry about Safe Harbor (sic).

Short takes: SharePoint/Delve and shortlinks; CESG guidance on Office 365; removing Sway from the App Launcher

So, it’s Christmas Eve and I’ve run out of annual leave this year so I’m still working… looks like everyone else has gone home though so I’m really just clearing down my mailbox, searching for Inbox Zero nirvana. As I do, there are lots of little snippets that I might like to remember, so here’s a little Christmas compilation…

SharePoint, Delve and short links

We have a URL shortener at work and one of the things it’s really great for is taking reallyreallylongandundigestibleurisfromsharepoint and making them risu.al/short. Unfortunately Alex Eggar, who leads our Business Productivity group, highlighted to me that I’m better off using SharePoint’s sharing functionality… otherwise Delve won’t know what’s going on…

There’s loads of information on Delve for Office 365 administrators and Paul Olenick (SharePoint MVP) has an interesting post the describes more about Delve. What I haven’t managed to get clear in my head yet is why a short URL bypasses the Office Graph… I’m still accessing the content… but I’ll leave that one to the experts!

CESG Guidelines for use of Office 365 at OFFICIAL

I had an interesting meeting with a customer recently, discussing how their Office 365 implementation aligned to UK Government (CESG) guidelines. Whilst they are guidelines, and this customer is only loosely affiliated with the Government, the CESG guidance on Office 365 could be considered as a useful benchmark.

The guidelines are available on the gov.uk website. Currently they include:

Turning off the App Launcher tile for Sway

As I wrote a couple of months ago when describing how to selectively remove tiles from the Office 365 App Launcher, disabling Sway in Office 365 didn’t used to remove the tile from the launcher. Since earlier this month, that behaviour has been changed with more details in Microsoft knowledge base article 3075256.

Encouraging adoption in enterprise social networks

In my job, consulting with many organisations who are adopting Microsoft products and services, including Office 365, I have a lot of discussions about Yammer and other modern communication methods (e.g. Sway).

Many have already had pilots with Yammer and found it didn’t work for them. Some are smart enough to realise that it wasn’t Yammer at fault but a lack of executive sponsorship. Adapting a new medium for communication takes time; it needs a cultural shift. If your boss uses it, you might consider giving it a try (although when I had a team, my experience at getting them to use Yammer was best described as mixed). But if your boss’ boss uses it, or the CEO, and the internal communications team are are using it instead of email, then you might stand some chance of success – because, as well as executive sponsorship, it needs critical mass (which means people need a reason to visit).

Of course, the platform itself has to be usable. In my last place the corporate social platform was Newsgator (which was awful), coupled with an old version of SharePoint and, aside from the teams whose job it was to evangelise its use, it was pretty much ignored. In fact, so much so that other social networks popped up in their own bubbles: the sales community were using Salesforce Chatter; although Yammer actually seemed to gain more traction in some areas (via an external network hosted by Microsoft for partner engagement) because there was something of value there for people.

So, we need executive sponsorship, critical mass, and a usable platform, with content that people value. But there’s something else too – people have to stop using the old methods in parallel.

Recently, I witnessed one organisation where someone posted some infomration on Yammer and it got a flurry of activity/commentary on the original post (so far so good). Then someone else sent an email to a distribution group to highlight the same information. That sender might not have seen the original post but email isn’t a good way to share links about new products. Some (myself included) may consider it as just unsolicited bulk email (spam) but spam that’s sent from inside the organisation. To make matters worse, because Office 365 Clutter doesn’t filter out email from people in your management chain, that email will never be filtered.

No, no, no, no! Post once, on the right medium*. Yammer for information sharing/comments on a topic that might run and run; instant messaging for messages that require a response… instantly (the clue’s in the name) and stop abusing email (which, incidentally is an asynchronous communications mechanism to which you should not require, or even expect, a response). As for voice mail, SMS, etc. Well, who knows… anyway, I’m supposed to be writing about getting people using enterprise social networks here – not a lecture on communication methods (and I know one size doesn’t fit all).

So, that’s my view – which you might agree with, or you may not. But it’s been cathartic to have a little online rant and at least it means I’ll get at least one blog post up this month! For another view, take a look at what the Yammer team at Microsoft shared with me – a 2012 Office blog post on Deploying a Successful Enterprise Social Network: Best Practices From the Field.

 

Mark Wilson is an increasingly busy, grumpy and ranty man, who wants to reduce the volume of email arriving in his Inbox…

* I do have to admit that, on occasion, I have been known to email a group of people and say “please reply to my thread on Yammer”, because I knew a lot of them didn’t use it but I wanted everyone to see the replies withough creating a Reply All email storm. This is not good.

Recovering data after OneDrive for Business “ate” my OneNote notebooks…

Yesterday, I wrote about troubleshooting OneDrive for Business. What I didn’t write about though was the problems that a simple repair to OneDrive for Business (acting on advice to resolve some sync issues on my client) caused for me…

The OneDrive for Business repair operation works as follows:

  • Disconnects all libraries that were experiencing sync problems.
  • Reconnects these libraries. This means Repair downloads and syncs the current server version of each library, as if you were syncing for the first time.
  • Creates an archive copy of any file that had unsynced changes and places these files in a library folder under C:\users\username\OneDrive for Business archives\.

So, if you are using that full 1TB of storage… you’d better have a good network connection to pull the entire contents of the library from the cloud (which is why the next version of the OneDrive client has selective sync).

In my case, I’m only using a few GB but, because I moved my entire Documents folder to OneDrive a few months ago, my OneNote notebooks were part of the data that was pulled down from the cloud.

I rely heavily on OneNote – I stopped using paper notebooks when I left my last job, as my everyday device is a Surface Pro 3 (which I find ideally suited to note-taking) – and here’s the lesson I learned:

OneNote and OneDrive for Business do not (always) play together nicely.

It should work – there’s even Microsoft advice for moving a OneNote notebook to OneDrive (and the same process works for OneDrive for Business) but it seems the mistake I made was to move all of my files in Windows Explorer. Whilst researching this blog post I’ve found Microsoft’s OneNote syncing best practices (KB2819334) and what I should have done is move the OneNote notebooks from within OneNote…

After the OneDrive for Business repair, I was left with a .ms-one-stub file which Explorer reported as being 1KB in size. 6 months of notes had disappeared – and opening OneNote didn’t follow the stub and magically pick up my notes. I felt physically sick. I thought I had two copies – one on the PC and one in OneDrive for Business. But no, OneDrive for Business was my backup – and it had “eaten” my work.

Luckily, there was another backup copy. It wasn’t current, but it was only a couple of days out of date, rather than starting from scratch. I found that OneNote stores a copy of notes in C:\Users\username\AppData\Local\Microsoft\OneNote\15.0\Backup.

That location has a folder for each notebook. Each folder contains a OneDrive recycle bin (OneNote_RecycleBin) and copies of  my .one files for each section, with a date when the backup was taken – for example project.one (On 22-11-2015).one. I’m not sure when the backup is taken (I’ve made changes to sections today that are still not reflected in the OneNote backup, but losing a couple of days is vastly superior to losing 6 months.

Even with the new information about the correct way to sync OneNote to OneDrive for Business, I’m not sure I completely trust it. From now on I’ll be making a third copy to another location…