Update on data residency options for Office 365 customers in the UK

Back in September, Microsoft started offering Azure and Office 365 services from UK datacenters.  At the time, there was no announcement for customers who had existing Office 365 tenants (hosted elsewhere in Europe) about how to move data to the UK but, earlier today, my colleague Brian Cain (@BrianCainUC) tweeted about a Microsoft article titled “moving core data to new Office 365 datacenter regions“. This isn’t a new page but it seems Microsoft has quietly updated it to include reference to a new Data Residency Option for the UK (updated 3 November 2016):

“[Microsoft] offer existing customers that have strict data residency requirements, and that are listed in the table below, an option to have their core customer data moved to the new region.”

Customers with billing address in Previous datacenter region New datacenter region Region available since Announcement
United Kingdom Europe, Middle East, Africa United Kingdom September 2016 Office 365 Blog

[some rows have been removed from the table above]

Previously the UK was covered by the statement that:

“The data residency option, and the availability to move customer data into the new region, is not a default for every new region [Microsoft] launch. As [Microsoft] expand into new regions in the future, [Microsoft will] evaluate the availability and the conditions of data moves on a region by region basis.”

I, and my colleagues at risual, have seen a lot of interest from customers who are UK-based but have Office 365 tenants that were created before 2 September 2016; however my colleague Paul Wooldridge highlighted that the option to move data is time limited.

Microsoft’s “How to request your data move” page is clear that for UK customers the request period begins on 1 December 2016 and ends on 28 February 2017 [update 7 December 2016: my colleague Gavin Morrison spotted that the page has been updated to state 12 December 2016-10 March 2017. Update 12 December 2016: the page has now been updated with date “to be determined”], the actual migration of the data can take up to 2 years, and that:

“[Microsoft is] unable to accept requests to be moved after the deadline in each region”

So, if you’re looking to “Brexit your data”, you have a 3 month window in which to make the request, and potentially up to a 2 year wait. Also, once moved, there is no way back – at least not without performing your own tenant-to-tenant migration.

Further reading

Microsoft’s UK datacenters: what you need to know.

Short takes: what to do when Outlook won’t open HTTP(S) links; how to disable Outlook Clutter; and don’t run externally-facing mail servers in Azure!

Once again, my PC is running out of memory because of the number of open browser tabs, so I’ll convert some into a mini-blog post…

Outlook forgets how to open HTTP(S) links

I recently found that Outlook 2016 had “forgotten” what to do with HTTP(S) links – complaining that:

Something unexpected went wrong with this URL: […] Class not registered.

The fix was to reset my default browser in Windows. Even though I hadn’t changed it away from Edge, a Windows Update (I expect) had changed something and Edge needed to be reset as the default browser, after which Outlook was happy to open links to websites again.

Globally disable Outlook Clutter

I had a customer who moved to Exchange Online and then wanted to turn off the Clutter feature, because “people were complaining some of their email was being moved”.

Unfortunately, Clutter is set with a per-mailbox setting so to globally disable it you’ll need something like this:

get-mailbox | set-clutter -enable $false

That will work for existing mailboxes but what about new ones? Well, if you want to do make sure that Clutter remains “off”, then you’ll need a script to run on a regular basis and turn off Clutter for any new users that have been created – maybe using Azure Automation with Office 365?

Alternatively, you can create a transport rule to bypass Clutter.

Personally, I think this is the wrong choice – the answer isn’t to make software work the way we used to – it’s to lead the cultural change to start using new features and functionality to help us become more productive. Regardless, Clutter will soon be replaced by the Focused Inbox (as in the Outlook mobile app).

Don’t run externally-facing mail servers in Azure

I recently came across a problem when running an Exchange Hybrid server on a VM in Azure. Whilst sending mail directly outbound (i.e. not via Office 365 and hence Exchange Online Protection), consumer ISPs like Talk Talk were refusing our email.  I tried adding PTR records in DNS for the mail server but then I found the real issue – Azure adds it’s IP addresses to public block lists in order to protect against abuse.

It turns out that Microsoft’s documentation on sending e-mail from Azure compute resource to external domains is very clear:

“[…] the Azure compute IP address blocks are added to public block lists (such as the Spamhaus PBL).  There are no exceptions to this policy”

and the recommended approach is to use a mail relay – such as Exchange Online Protection or a third party service like SendGrid. Full details can be found in the Microsoft link above.

Inside the Microsoft datacentres

A datacentre is just a datacentre isn’t it? After all, isn’t it just a bigger version of the server room in the basement? But what about the huge datacentres that run cloud services? What’s it like inside the Microsoft datacentres that host Azure, Office 365, etc.?

Last week, Microsoft’s Modern Workplace webcast titled “An Inside Look at Your Secure Cloud” gave a sneak peek inside some of the Microsoft datacentres – comparing various generations and showing the improvements along the way.  And, as you might expect, these are the very definition of operating at scale…

As Doug Hauger (General Manager for National Cloud Programs at Microsoft) explained, organisations look to use a cloud datacentre for scale and professionalism.  Anyone can run a datacentre but the Microsoft Cloud is about robustness and security – whether that’s how staff are monitored or the physical and logical security models.

Each time Microsoft moves into a new region (like the two regions that opened in the UK earlier this month) there’s not just one super-scale datacentre but multiple facilities per region, providing redundancy and disaster recovery capability. Each facility has multiple power sources and multiple network ingress and egress points. Then there’s the investment Microsoft is making in physical infrastructure around the world – for example the joint project with Facebook for a new Europe-North America undersea cable (MAREA).

Each time Microsoft considers expanding into a new market they perform a business case analysis on the potential opportunity, considering the scale that they will go in at (tens of thousands of servers). Microsoft now has more than 100 datacentres in 30 regions around the world (with four more under construction). Because of the huge range of locations covered, Microsoft is now the industry leader for compliance and certification – whether that is meeting global or local requirements. Then there is the question of meeting customer needs around data residency, compliance, etc. (for example with the German datacentres that operate under a unique data trustee model in partnership with Deutsche Telekom).

With its cloud datacentres, Microsoft is aiming to meet customer needs around digital transformation, where the question is no longer “why should I go to the cloud” but one of “how to innovate more quickly in the cloud”. That’s what drives the agenda for where to geographically expand, where enhance scalability, etc.

Despite the question I posed in the opening paragraph of this post, a true datacentre is worlds apart from the typical server room in the basement (or wherever). The last time I got to visit a datacentre was when I was working at Fujitsu and I visited the London North facility, an Uptime Institute Tier III datacentre that won awards when it was built in 2008. Seeing the scale at which a modern datacentre operates is impressive. Then ramp it up some more for the big cloud service providers.

In the webcast, Christian Belady (General Manager Cloud Infrastructure Strategy and Architectures at Microsoft) explained that datacentres are the foundation of the Internet – they are where all the cloud services are served from (whether that is Microsoft services, or those provided by other major players).

There are several layers of physical security from the outside fence in, screening people, controlling access to parts of the buildings, even to cabinets themselves with critical customer data in locked cabinets covered with video surveillance. Used disks are destroyed, being wiped and then crushed on site! The physical security surpasses anything provided for on-premises servers and the logical security continues that defence in depth.

Each custom-built server is actually 2 computers with 10s of 1000s of computers per room, 100s of 1000s per datacentre, each datacentre the size of 20-30 football fields. Look at the racks and you can see the attention to detail – keeping things orderly not only adds to operational efficiency but it looks good too! The enterprise servers that most of us run on-premises have plastic bezels to make them look pleasant. Instead, Microsoft’s servers have focused on eliminating anything that has no useful function…

Each iteration of datacentres becomes more industrialised – with improvements to factors such as cooling (which is one of the biggest power usage factors).

A generation 2 datacentre from around 2007 has a Power Usage Effectiveness (PUE) efficiency score of 1.4-1.6 (for comparison, the Fujitsu facility I mentioned earlier has a PUE of 1.4 but a typical enterprise datacentre from the 2000s with a normal raised floor would have a PUE of 2-3). Cool and hot aisles are used with hot air returned to coolers and recirculated. Microsoft then raised the temperature of their servers to a level that is acceptable (working with manufacturers), rather than the lower levels they used to have (reducing the cooling demands).

Moving on to generation 4, efficiency is improved further (a PUE of 1.1-1.2), eliminating chillers by removing roofs, driving down costs and using outside air to chill. Containers use the outside cooling and a system of adiabatic cooling, spraying mist into the air to cool down – which evaporates before it hits the server”. Such datacentres use a lot less water too (compared with older styles of datacentre).

With the latest (generation 5) datacentres, further improvements are made, culminating the features of other generations – learning and adapting. The PUE is now down to 1.1 (and below at certain times of year) with running costs also improved. There are still hot a cold aisles but no raise floor and, instead of outside air, the datacentres use a closed liquid loop system (no chiller – cool the water outside) – and that water doesn’t need to be potable.

The actual datacentre design changes for each facility, based on the geography and the environmental impact. Backup power generation is a key component in the design, with several days of fuel onsite and contracts to keep bringing more fuel in. Power is often sustainably sourced, be that cheap and carbon-free hydro-electric power, wind or solar. Microsoft Research is even working on a tidal-powered under-sea datacentre (Project Natick).

Inside the Microsoft datacentres is very industrial. Whole racks are brought in (pre-tested), rather than single servers and, as previously mentioned, Microsoft design and build the servers for use at scale, stripping out enterprise features and retaining only what’s needed for the Microsoft environment.

Whilst I’ve worked with customers who have visited Microsoft datacentres in Dublin, it seems unlikely that I’ll ever get the chance. Watching the Modern Workplace webcast gave me a fascinating look at how Microsoft operates datacentres at scale though – and it truly is awe-inspiring. To find out more, visit the Microsoft website.

Microsoft’s UK datacentres: what you need to know

This morning, the UK woke up to an announcement from Microsoft that the UK datacentres for Azure and Office 365 are generally available, making Microsoft the first global provider to deliver a complete cloud (Iaas, PaaS and SaaS) from UK data centres.

That means:

  • Two new Azure regions in the UK:
    • UK West (Cardiff)
    • UK South (London)
  • Office 365 services from UK datacentres in Durham and London.

Dynamics CRM online will be offered from the UK in the first half of 2017.

That Azure location information was taken from the Azure regions page on the Microsoft website (although my sources tell me that “Cardiff” is really “Newport” – close enough as to make no difference anyway, and London is probably “near London” too).  The Office location information was taken from the Office 365 Interactive Data Maps.

Now, UK customers already using Azure or Office 365 will be asking “will my data be moved to a UK datacentre?”. There’s no official announcement from Microsoft (not that I’ve seen) but my (unofficial) answer is “no”. At least not automatically.

For Azure, it’s good practice to design across multiple regions. There are also implications around geo-replication (which regions are paired with which for business continuity and disaster recovery purposes). Moving resources from one region to another is possible but is also a project that would need to be undertaken by a customer (possibly working with a partner) as a programme of planned resource moves.

For Office 365, it’s worth reading the TechNet advice on Moving core data to new Office 365 datacenter regions. At the time of writing it hasn’t been updated to reflect UK datacentres (it was last updated 28 July 2016) but it currently says:

“Existing customers that have their core customer data stored in an already existing datacenter region are not impacted by the launch of a new datacenter region”

[…]

“The data residency option, and the availability to move customer data into the new region, is not a default for every new region we launch. As we expand into new regions in the future, we’ll evaluate the availability and the conditions of data moves on a region by region basis.”

“New customers or Office 365 tenants created after the availability of the new datacenter region will have their core customer data stored at rest in the new datacenter region automatically.”

The page goes on to state that, assuming the data residency option is made available for the UK (remember, nothing has been announced yet)

“Customers will need to request to have their data moved within a set enrollment window.”

and that:

“Data moves can take up to 24 months after the request period to complete”

There’s also a footnote on the UK interactive data map to say:

“Customers who signed up and selected the United Kingdom for their Office 365 services before September 2, 2016 will have their customer data located in the EMEA datacenter locations.”

So, in short, Office 365 (SaaS) data stays exactly where it is, unless you sign up for a new tenant, or wait for further announcements from Microsoft. Azure (IaaS and PaaS) workloads can be moved to the new regions whenever you are ready.

 

End user computing – the device doesn’t matter

Following a recent Windows update that “went bad”, I needed to have my work PC rebuilt.  That left me with a period when I had work to do, but only a smartphone to work on or my personal devices. To me, this was also a perfect opportunity to put cloud services to work.

So, armed only with a web browser on another PC, I was perfectly able to access email and send/receive IMs (it’s all in Office 365), pester people on Yammer, catch up on some technical videos, etc. There was absolutely nothing (technically) preventing me from doing my job on another device. That’s how End User Computing should work – providing a flexible computing workstyle that’s accessible regardless of the device and the location.

The real issues are not around technology, but process: questions were asked about why I wasn’t following policy and using my company-supplied device; and I was able to answer with clear reasons and details of what I was doing to ensure no customer information was being processed on a non-corporate device. There are technical approaches to ensuring that only approved devices can be used too – but what’s really needed is a change of mindset…

Retired: Mark’s Office 365 Resource Centre

18 months ago, I created “Mark’s Office 365 Resource Centre” using the public site from my Office 365 subscription. Over the last few months it’s fallen by the wayside as my focus has recently moved towards Azure (and Office 365 public websites are a deprecated feature) so I decided to move it here.  This content is no longer maintained, but may still contain some useful links.

New functionality

See also Office 365 Updates on the Office Blogs.

Planning and Deployment resources

Technical resources

Training and certification

Licensing

To license Office 365, costs are provided on the Microsoft Online Services Customer Portal. These are ordered and paid for directly by customers (although trial tenants may be created by partners though the
FastTrack portal). Customers with Enterprise Agreements have additional options including not just the Office 365 plans but ‘add-on’ and ‘bridge’ licenses for on-premises Office and CAL Suites.

Information for partners

Tips, tricks and more from my blog

There are a few posts missing from this list, because I consider them to be out-of-date (although they are still available):

Useful to know

(Including tips and tricks from elsewhere on the web):

Bits and bytes (downloads)

Short takes: ADFS certificate expiry; Azure Authenticator setup on Windows Phone; checking if a MSOL tenant name exists

Some more snippets of randomness pulled together to make a blog post…

ADFS certificate expiry

One of my colleagues spotted this in a customer’s Office 365 tenant recently:

Office 365 - Renew your certificates

Thankfully, it wasn’t one we were managing… but I did feel the need to flag it to the incumbent service provider. If this happens to you, my colleague Gavin Morrison (@GavinMorrison) flagged a potentially useful blog post from Jack Stromberg about renewing ADFS Certificates.

Azure Authenticator Setup on Windows Phone

Whilst setting up additional authentication for Office 365 (in effect, Azure AD MFA) I found that I couldn’t add an account until the Windows Phone Azure Authentication app had enabled push notifications. Despite repeatedly enabling it in Settings, completing setup of the account needed a phone reboot, at which point it was ready for me to scan a QR code and continue.  Even then the option to allow notifications doesn’t seem to stick!

Checking if a Microsoft Online Services tenant name exists

My colleague Gareth Larter found a neat trick this week for checking if a Microsoft Online Services (MSOL) tenant exists (e.g. for Office 365).

Gareth’s advice is to browse to https://login.windows.net/tenantname.onmicrosoft.com/FederationMetadata/2007-06/FederationMetadata.xml and, if you get an error, it should show “No service namespace named ‘tenantname.onmicrosoft.com’ was found in the data store” at the bottom right meaning that the tenant name is available:

On the other hand, if you get a bunch of XML data returned, then that tenant already exists.

“Unlicensed Product” errors in Microsoft Office

Earlier this evening, I noticed that my copy of Outlook was showing as an “unlicensed product” at the top of the screen. That seemed strange, as I pay for an Office 365 Home subscription, which covers my family’s various copies of Office.

Outlook reports intself as an unlicensed product

So, I took a look at the Office Account settings, and noticed that it wasn’t signed in to Office 365 for some of the connected sources.

Disconnected from Office 365 services

I reconnected to My Office 365 subscription, signing in with my “Work or school account” as that’s what the markwilson.it Office 365 subscription uses, even though the Office 365 Home subscription uses a Personal Account (formerly known as a Microsoft Account):

Which Microsoft account to use?!

After authenticating (and a restart), Outlook was no longer complaining about being unlicenced.

I’m not sure if it’s a complication of having both a Microsoft Account (MSA) and an Organization/Work and School (Azure AD) account with the same email address, but it seems there are various scenarios that can present this issue.

Thankfully this one wasn’t too hard to sort out!

Why Microsoft customers don’t need to worry about EU-US Safe Harbour/Harbor

When European Courts judged the 15-year-old EU-US Safe Harbour/Harbor treaty to be invalid last October, Internet news sites started to report how terrible this was for EU companies placing data into cloud services offered (mostly) by American companies. For some, that may be true, but that assumes Safe Harbour is the only protection in place.

This week, IT news sites are at it again. The Register (the tabloid newspaper of IT news sites) has an article titled Safe Harbor 2.0: US-Europe talks on privacy go down to the wire but the actual URI belies a much more dramatic title of “Safe Harbor countdown to Armageddon”. Sensationalist at best, some might even say irresponsible.

I’m no lawyer but, for my customers, who are implementing Microsoft cloud services, there seems to be nothing to worry about and I’ll explain why in this blog post. Of course, Microsoft is just one of many cloud services providers – and for others there may be valid concerns.

The United States Export.Gov website currently displays the following text regarding Safe Harbor:

“On October 6, 2015, the European Court of Justice issued a judgment declaring as ‘invalid’ the European Commission’s Decision 2000/520/EC of 26 July 2000 ‘on the adequacy of the protection provided by the safe harbour privacy principles and related frequently asked questions issued by the US Department of Commerce.’

In the current rapidly changing environment, the Department of Commerce will continue to administer the Safe Harbor program, including processing submissions for self-certification to the Safe Harbor Framework. If you have questions, please contact the European Commission, the appropriate European national data protection authority, or legal counsel.”

EU Model Clauses trump Safe Harbour

Microsoft President and Chief Legal Officer, Brad Smith, issued a statement on 6 October 2015. Quoting from that article:

“For Microsoft’s enterprise cloud customers, we believe the clear answer is that yes they can continue to transfer data by relying on additional steps and legal safeguards we have put in place. This includes additional and stringent privacy protections and Microsoft’s compliance with the EU Model Clauses, which enable customers to move data between the EU and other places – including the United States – even in the absence of the Safe Harbor. Both the ruling and comments by the European Commission recognized these types of steps earlier today.

Microsoft’s cloud services including Azure Core Services, Office 365, Dynamics CRM Online and Microsoft Intune all comply with the EU Model Clauses and hence are covered in this way.”

There’s also a follow-on post which talks in general terms about the wider issues and privacy beliefs but the key point is that Microsoft offers EU Model Clauses within its contracts, which go beyond Safe Harbour. Microsoft also has an FAQ on the EU Model Clauses that is worth a read.

Quoting again from the 6 October 2015 statement:

“We wanted to make sure all of our enterprise cloud customers receive this benefit so, beginning last year, we included compliance with the EU Model Clauses as a standard part of the contracts for our major enterprise cloud services with every customer. Microsoft cloud customers don’t need to do anything else to be covered in this way.”

That suggests to me that customers who have signed up to Azure Core Services, Office 365, Dynamics CRM Online or Intune since early 2014 already have greater privacy protection than was afforded by Safe Harbour – and that protection meets the EU’s current requirements. In short, Microsoft customers don’t need to worry about Safe Harbor (sic).

Short takes: SharePoint/Delve and shortlinks; CESG guidance on Office 365; removing Sway from the App Launcher

So, it’s Christmas Eve and I’ve run out of annual leave this year so I’m still working… looks like everyone else has gone home though so I’m really just clearing down my mailbox, searching for Inbox Zero nirvana. As I do, there are lots of little snippets that I might like to remember, so here’s a little Christmas compilation…

SharePoint, Delve and short links

We have a URL shortener at work and one of the things it’s really great for is taking reallyreallylongandundigestibleurisfromsharepoint and making them risu.al/short. Unfortunately Alex Eggar, who leads our Business Productivity group, highlighted to me that I’m better off using SharePoint’s sharing functionality… otherwise Delve won’t know what’s going on…

There’s loads of information on Delve for Office 365 administrators and Paul Olenick (SharePoint MVP) has an interesting post the describes more about Delve. What I haven’t managed to get clear in my head yet is why a short URL bypasses the Office Graph… I’m still accessing the content… but I’ll leave that one to the experts!

CESG Guidelines for use of Office 365 at OFFICIAL

I had an interesting meeting with a customer recently, discussing how their Office 365 implementation aligned to UK Government (CESG) guidelines. Whilst they are guidelines, and this customer is only loosely affiliated with the Government, the CESG guidance on Office 365 could be considered as a useful benchmark.

The guidelines are available on the gov.uk website. Currently they include:

Turning off the App Launcher tile for Sway

As I wrote a couple of months ago when describing how to selectively remove tiles from the Office 365 App Launcher, disabling Sway in Office 365 didn’t used to remove the tile from the launcher. Since earlier this month, that behaviour has been changed with more details in Microsoft knowledge base article 3075256.