Big Data Could Mean Big Problems
Data centers consume 2% of the US's total energy use. As demand for centralized data storage increases, what can cloud providers like Amazon do to cut their energy use and costs?
Amazon Web Services (AWS) was launched in 2006 offering cloud computing services that provide customers with on-demand access to computing resources that can be expanded as needed. Cloud computing providers maintain data centers with server space that can be purchased by customers on a “pay as you go” model.
AWS is the uncontested leader in the public cloud market – the company currently has both double the market share and double the revenue of its next three largest competitors combined [1]. With such a remarkable lead in a rapidly expanding market, it is difficult to imagine any of the other major players (i.e. Microsoft, Google, IBM, or Oracle) catching up anytime soon.
To support their cloud infrastructure, AWS operates upwards of 38 data centers around the world [2]. Data centers use an incredible amount of electricity; in 2014, US based data centers consumed over 70 billion kilowatt-hours of electricity. This represents about 2% of the US’s total energy use and constitutes enough energy to power 6.4 million American homes [3]. Looking beyond 2016, demand for data centers and their subsequent energy use will continue to rise as we become more dependent on big data, video streaming, and other data intensive applications.
As climate change is increasingly recognized in the global community, reducing energy demand is a promising avenue for lessening greenhouse gas emissions. Some regions in the US have already implemented regulations that force data centers to drastically reconsider their efficiency policies. For example, in early 2013, California passed a law that made energy more expensive to data center operators and thus incentivized data centers to make energy-efficiency improvements.[4]
Beyond formal regulations, there are significant cost incentives for data centers to improve their efficiency, and many data centers have already taken great strides in this area. AWS operates with a Power Usage Effectiveness (PUE)[5] of less than 1.2, which is notably efficient. The Uptime Institute issues an annual report each year on data center efficiency and found the average PUE for data centers to be around 1.7[6], however many large-scale providers run significantly more efficiently, even at a PUE of 1.07.
There are many ways for data centers to increase their efficiency. Companies can choose more energy efficient processors and can intelligently design their cooling and air flow systems to prevent having to air condition or heat their facilities. Software can be used to increase the efficiency of older servers by pushing through more data and virtualization technology can be used to run multiple virtual servers on a single physical server, resulting in drastic increases in utilization and reducing the need for more hardware. One of the best ways to increase data center efficiency is to use software to identify servers that aren’t operating at full utilization but are still using power (possibly due to the business unit no longer using the server). In one survey, data center operators reported that when investigated they found that between 5% and 25% of servers were not being used. The Green Grid (a non-profit consortium dedicated to improving the efficiencies of data centers) concluded that identifying unused servers could account for the cost of an entire new data center.
In addition to the above improvements, Amazon is also focused on moving away from fossil fuels when it comes to powering their data centers. Amazon has pledged to reach 40% renewable energy by the end of 2016 and 50% renewable energy by the end of 2017.[7] Amazon’s long-term commitment is to hit 100% renewable energy by integrating wind and solar farms across the world. Current projects include building 100 megawatt and 189 megawatt wind farms in Ohio, a 208 megawatt wind farm in North Carolina, a 80 megawatt solar farm in Virginia, and a 150 megawatt wind farm in Indiana.
Data resources will continue to be instrumental in the coming years as we increase our reliance on distributed cloud storage. AWS and other cloud storage companies have many opportunities to simultaneously reduce cost and GHG emissions. An interesting corollary can be drawn between data centers and the light bulb – electric companies worried that as light bulbs became more efficient, it would drive them out of business. Instead the demand for lights quadrupled [8] – as we get more efficient, we often see a greater demand.
Demand for centralized data storage will no doubt increase in the future. We must be cognizant as consumers of the services – if we want greater efficiency we must go to the companies focused on sustainability to signal it is important to us. Without the directive from customers, companies will only become more efficient as required by regulation or to the extent that it saves them money.[9]
(773 words)
[1] Conner Forrest, “Amazon doubles its public cloud lead, can anyone catch up?,” TechRepublic, November 3, 2016, http://www.techrepublic.com/article/amazon-doubles-its-public-cloud-lead-can-anyone-catch-up/, accessed November 2016
[2] Amazon Web Services, “AWS Global Infrastructure,” https://aws.amazon.com/about-aws/global-infrastructure/, accessed November 2016
[3] Yevgeniy Sverdlik, “Here’s How Much Energy All US Data Centers Consume,” Data Center Knowledge, June 27, 2016, http://www.datacenterknowledge.com/archives/2016/06/27/heres-how-much-energy-all-us-data-centers-consume/, accessed November 2016
[4] Robert J. Mullins, “New global warming rules put the heat on data centers,” Network World, August 26, 2013, http://www.networkworld.com/article/2169283/data-center/new-global-warming-rules-put-the-heat-on-data-centers.html, accessed November 2016
[5] PUE is the ratio of the total facility energy to IT equipment energy. A lower number represents a more efficient data center.
[6] Yevgeniy Sverdlik, “Survey: Industry Average Data Center PUE Stays Nearly Flat Over Four Years,” Data Center Knowledge, June 2, 2014, http://www.datacenterknowledge.com/archives/2014/06/02/survey-industry-average-data-center-pue-stays-nearly-flat-four-years/, accessed November 2016
[7] Amazon Web Services, “AWS & Sustainability,” https://aws.amazon.com/about-aws/sustainability/, accessed November 2016
[8] Jason Verge, “Microsoft: Centralization is Driving Energy Efficiency,” Data Center Knowledge, April 30, 2013, http://www.datacenterknowledge.com/archives/2013/04/30/microsoft-centralization-is-driving-the-frontiers-of-energy-efficiency/, accessed November 2016
[9] Jeff Clark, “Who’s Responsible for Data Center Energy Efficiency?,” Data Center Journal, April 16, 2015, http://www.datacenterjournal.com/whos-responsible-data-center-energy-efficiency/, accessed November 2016
Great article! I feel like a lot of businesses further up the stack discount the environmental cost of their own cloud computing based on the fact that it’s highly distributed, and they do not take it into account when calculating their own carbon footprint. That being said, it would be interesting to see if there was a way to quantify the positive environmental effects of businesses customers that have scaled using AWS rather than other storage options and how that would net against the AWS data center electricity usage.
There is actually a really interesting blog post from AWS about this topic where they compute the positive environmental effects of using AWS over a company using its own data centers. I unfortunately did not have the word count to incorporate this info into my blog post, but I liked it below if you are interested.
“On average, AWS customers use 77% fewer servers, 84% less power, and utilize a 28% cleaner power mix, for a total reduction in carbon emissions of 88% from using the AWS Cloud instead of operating their own data centers.”
https://aws.amazon.com/blogs/aws/cloud-computing-server-utilization-the-environment/
This is a more and more pressing issue as demand for cloud computing continues to increase. Recently, there were lots of discussions about facebook’s plan to build data center in Taiwan and it’s impact to the environment. However, working for one of the major server manufacturers in the world, I didn’t see much conversation going on about environmental concerns. Customers care more about processing power and price than environment friendness. With that said, I do see a few new initiatives in the industry. There are start-ups working on next generation GPU with higher processing power and less energy consumption. At advantech, we are also trying to reduce heat generation and energy consumption by redesign our server. It’s important that we recognize this issue and continue to work in it collectively.
Thanks for the post, Jessica. In the spirit of out-of-the-box ideas, I came across a few creative ways that data center operators are trying to reduce their energy consumption. One idea is relocating to the parts of the world with the cheapest electricity, such as Iceland, where they have more geothermal and hydroelectric power than they know what do with and are not able to export (1). In fact, the Icelandic government is even considering changing their tax code to incentivize more foreign companies. Another radical solution is to actually launch the servers into outer space, where they could be powered by solar radiation and cooled by low temperatures. I am not an expert, but I read that zero gravity also has some benefits w/r/t any moving parts (i.e., less friction) (2). Given that Bezos from AMZN also owns Blue Origin, which is expected to provide a variety of spaceflight services, I wouldn’t put the next generation of “cloud” computing passed AMZN…
(1) http://www.ibtimes.com/iceland-lures-data-center-companies-cheap-renewable-energy-2081695
(2) http://fortune.com/2015/01/29/connectx-space-data/
Jessica — Awesome article! Although I was aware that the cloud datacenter market was growing significantly, I did not know that data centers account for ~2% of the US energy consumption each year. I am impressed with Amazon’s targets to reach 100% renewable energy eventually. It seems that the solar and wind farm projects that they have planned will be the next right steps to reach their targets.
One other thing to note is that cloud datacenters may be more energy efficient than the alternative of locating servers on premise. According to their website, Microsoft Azure claims that moving to Azure services can reduce energy usage 30-90% versus using on-premise services.
https://www.microsoft.com/about/csr/environment/solutions/cloud/
Jessica, thanks for starting this discussion! The numbers are already staggering (2% of the total energy consumption?!), and the problem will only grow exponentially with the advent of the 5G networks and the “internet of everything” in the coming years.
An important aspect of the data centers is that they produce an enormous amount of heat and have to be cooled if you don’t want the equipment to fry, which is a substantial part of their energy consumption. An interesting solution that has been proposed is simply moving them to geographically colder regions to take advantage of the natural cooling (to the extent that’s possible). A extreme form of this is currently being developed in Finland, where they plan to use the sea water (from the icy seas) for this – it’s googlable.