Earlier this week, Amazon Web Services’ S3 storage service suffered an outage that affected many\u00a0websites (including popular sites to check if a website is down for everyone or just you!).<\/p>\n
\nS3 is experiencing high error rates. We are working hard on recovering.<\/p>\n
\u2014 Amazon Web Services (@awscloud) February 28, 2017<\/a><\/p><\/blockquote>\n
Unsurprisingly, this led to a lot\u00a0of discussion about\u00a0designing for failure – or not, it would seem in many cases, including the architecture behind Amazon’s own status pages:<\/p>\n
\nThe dashboard not changing color is related to S3 issue. See the banner at the top of the dashboard for updates.<\/p>\n
\u2014 Amazon Web Services (@awscloud) February 28, 2017<\/a><\/p><\/blockquote>\n
The Amazon and Azure models are slightly different but in the past we’ve seen outages to the Azure identity system (for example) impact on other Microsoft services (Office 365). When that happened, Microsoft’s Office 365 status page didn’t update because of a caching\/CDN issue. It seems Amazon didn’t learn from Microsoft’s mistakes!<\/p>\n
Randy Bias (@RandyBias<\/a>) is a former Director at OpenStack and a respected expert on many cloud concepts. Randy and I exchanged many tweets<\/a> on the topic of the AWS outage but, after multiple replies, I thought a blog post might be more appropriate. You see, I hold the view that not all systems need to be highly available. Sometimes, failure is OK. It all comes down to requirements:<\/p>\n
\n@randybias<\/a> Depends what the system is. Not everything needs to be highly available. There’s a requirements\/cost\/risk trade-off<\/p>\n
\u2014 Mark Wilson (@markwilsonit) March 1, 2017<\/a><\/p><\/blockquote>\n
And, as my colleague Tim Siddle highlighted:<\/p>\n
\n@markwilsonit<\/a> DR\/multi-region\/multi-cloud is expensive – and it’s always a requirement, until the cost is laid bare…..<\/p>\n
\u2014 Tim Siddle (@tim_siddle) February 28, 2017<\/a><\/p><\/blockquote>\n
I agree. 100%.<\/p>\n
\n@markwilsonit<\/a> and of course, much depends on the application architecture itself<\/p>\n
\u2014 Tim Siddle (@tim_siddle) February 28, 2017<\/a><\/p><\/blockquote>\n
So, what does that architecture look like? Well, it will\u00a0vary according to the provider:<\/p>\n
\n
- For AWS we\u00a0need to think about regions and availability zones. Each region is made up of a number of availability zones (at least two according to the AWS glossary<\/a>).<\/li>\n
- For Azure there are more regions<\/a> and these are paired for availability<\/a> (for example when using geo-redundant storage). In addition, each region will consist of multiple datacentre facilities.<\/li>\n<\/ul>\n
So, if we want to make sure our application can survive a region failure, there are ways to design around this. Just be ready for the solution we sold to the business based on using commodity cloud services to start to look rather expensive. Whereas on-premises we typically have two datacentres\u00a0with resilient connections, then we’ll want to do the same in the cloud. But, just as not all systems are in all datacentres on-premises, that might also be the case in the cloud. If it’s a\u00a0service for which some downtime can be tolerated, then we might not need to worry about\u00a0a multi-region architecture. In\u00a0cases where we’re not at all concerned about downtime we might not even use an availability set<\/a>…<\/p>\n
Other times\u00a0– i.e.\u00a0if the\u00a0application is a web service for which\u00a0an outage would cause reputational or financial damage – we may have a requirement for higher availability.\u00a0 That’s where so many of the services impacted by Tuesday’s AWS outage went wrong:<\/p>\n
\nNo one claims 100% up time, FOR A REASON<\/p>\n
\u2014 Jeorry Balasabas (@jeorryb) February 28, 2017<\/a><\/p><\/blockquote>\n
\nAnd understand it when designing cloud solutions, still your responsibility to deliver resilience, can’t abdicate that to someone else https:\/\/t.co\/VGdunBJqSH<\/a><\/p>\n
\u2014 Paul Stringfellow (@techstringy) February 28, 2017<\/a><\/p><\/blockquote>\n
\nAmazon’s S3 outage is not just a case of getting what you paid for it’s also about getting what you designed for. Availability isn’t cheap.<\/p>\n
\u2014 Mark Twomey (@Storagezilla) February 28, 2017<\/a><\/p><\/blockquote>\n
Of course, we might spread resources around regions for other reasons too – like placing them closer to users – but that comes back to my point about requirements. If there’s a requirement\u00a0for fast, low-latency access then we need to design in the dedicated links (e.g. AWS Direct Connect or Azure ExpressRoute) and we’ll probably have more than one of them too,\u00a0each terminating in a different region, with load balancers and all sorts of other considerations.<\/p>\n
Because a cloud provider could be one of those single points of failure, many people are advocating multi-cloud architectures. But, if you think multi-region is expensive, get\u00a0ready for some seriously complex architecture and associated costs in a multi-cloud environment. Just as in the on-premises world, many enterprises use a single managed services provider (albeit with multiple datacentres), in the cloud many of us will continue to use a single cloud provider. \u00a0Designing for failure does not necessarily mean multi-cloud.<\/p>\n
Of course, a single-cloud solution has its risks. Randy\u00a0is absolutely spot on in his reply below:<\/p>\n
\n@markwilsonit<\/a> Public clouds are walled gardens and create significant points of lock-in. Long term AWS is no different than Oracle software.<\/p>\n
\u2014 Randy Bias (@randybias) March 1, 2017<\/a><\/p><\/blockquote>\n