Weather information provider leaned heavily on Amazon during Hurricane Sandy
Brandon Butler, Network World
During Hurricane Sandy this fall, The Weather Channel experienced its highest traffic ever. Normally the media company — which spans television, desktop and mobile platforms — supports about 90 million Web and mobile users a month. During Sandy, that jumped to 450 million — nearly double the company’s previous high for Web traffic.
Fortunately, The Weather Channel was prepared for the traffic surge. Landon Williams, VP of Platforms and Orchestration, and his IT team had recently architected the company’s real-time radar mapping system to run on Amazon Web Services’ Elastic Compute Cloud (EC2). On normal days, the mapping system runs on about 20 instances, but during Sandy it scaled up to run on 175 nodes in AWS’s cloud.
It’s a classic use-case for the cloud: Variable and unpredictable peak demand for Web services is outsourced to a public cloud provider. It would have required a major investment for Williams to spin up a system across the company’s on-premise and colocation environments to support the traffic load it experienced when Sandy ripped up the Eastern Seaboard, and then it would have likely gone mostly unused during other times. The alternative would have been not to have the compute horsepower to serve all the visitors to The Weather Channel’s platforms, but Williams doesn’t like that option either.
Adequate compute power for the application allows the radar maps to be updated as close to real-time as possible. Without the scaled-up, on-demand cloud resources users would have experienced lagging maps, delayed by seconds or maybe even minutes. “I want to be able to give people that visit our site the most up-to-date information possible,” he says. In extreme circumstances — think hurricanes, tornadoes, floods, lightning — weather information can be a life-or-death matter.
Now that Williams and TWC have had one success with the cloud, they’re looking at it for other applications. “We’re really, really excited about the opportunities the cloud can bring us,” he said at AWS’s recent user conference, which he attended to learn more about the products and services offered from AWS’s cloud and its partner network.
Another potential cloud use for TWC is having access to compute resources as developers need them, allowing an application or system to be developed in-house and then tested almost immediately across the cloud. “It’ll allow us to react to our business better,” Williams says.
TWC has spent the past few years migrating to a largely virtualized environment. The cloud, Williams says, is the next logical step. “There’s a lot of POC [proof of concept],” he says. “Everyone sees the value, but we don’t want to do it because it’s the cool, trendy thing to do; we want to do it to add to the experience of our users.”
There are a lot of considerations to take into account when moving from the multiple data centers the company currently runs most of its programs out of. No. 1 is making sure the system works. Amazon Web Services, which is where Williams and his team have dabbled the most in using cloud resources, hasn’t been immune to weather events itself. This past summer’s severe storms in Northern Virginia caused a power outage that led to AWS’s cloud being unavailable for some customers. TWC, naturally, needs to make sure its network is up during such times.
That’s why fault tolerant and highly available architectures are top-of-mind for Williams. He hopes to deploy Chaos Monkey, Netflix’s randomly generating fault tolerant testing system, to test the cloud architecture for failures. Chaos Monkey is open source software programmed to randomly shut down parts of a system to test if the application is fault tolerant.
Another big part of having a highly available system, Williams says, is not being tied to a single vendor. The goal is to have no single point of failure, he says.
At AWS’s first-ever user conference last month in Las Vegas, Williams was impressed with the breadth of services offered by Amazon, and was particularly excited about the company’s announcement of Redshift, a data warehousing platform. Groups within TWC are pursuing next-generation projects across the company, with the next targeted application going to the cloud being a weather forecasting program, he says.
Which apps make it to the cloud and which don’t is still to be determined. But Williams is optimistic about the possibilities. “This is a game changer to the business world and I’m thrilled that we’re a part of and leader of the change,” he says.
