Data center power efficiency

There's been quite a flurry of reactions to this front page story on yesterday's Sunday New York Times: Power, Pollution, and the Internet: Industry Wastes Vast Amounts of Electricity, Belying Image.

Here's a few examples of the responses flooding Der Netz:

• Diego Doval: a lot of lead bullets: a response to the new york times article on data center efficiency

  This statement, from an anonymous source no less, matters because it creates the sense in the article that the industry is operating in the shadows, trying to hide a problem that, “if only people knew” would create a huge issue. Nothing could be further from the truth. There are hundreds of conferences and gatherings a year, open to the public, where the people that run services get together to discuss these problems, solutions, and advances. Everyone I know talks about how to make things better, and, without divulging company secrets, exchanges information on how to solve issues that we face. There’s ACM and IEEE publications (to name just two organizations), papers, and conferences. The reason it appears hidden is that it’s just one of the many areas that only interest the people involved, primarily nerds. It’s arcane, perhaps, but not hidden.

• Wired's Robert McMillan: The New York Times Says Data Centers Suck — But Here’s What They Missed

  Facebook engineers no longer forage for fans. They think about how to optimize undersea cable routes. They design brand-new ways of storing data with some of the power turned off. They even design their own servers. And Facebook is now sharing many of its designs with the rest of the world.

• Forbes's Dan Woods: Why The New York Times Story 'Power, Pollution, And The Internet' Is A Sloppy Failure

  Think of it this way: Roads aren’t 100 percent utilized. The telephone system isn’t 100 percent utilized. They are there when they are needed. One might ask how much each paper edition of the New York Times is utilized? What percentage of articles delivered in the paper are read? Are all the articles not read waste?

And on and on and on.

I think it's great that the story was published, and I think it's great that it's generating all this discussion.

It's healthy; it's important; it's helpful.

When I look at the issue myself, I feel at least two primary reactions:

1. On a technical level, I am endlessly fascinated by the technology that is being developed by these systems. I love to read articles like A Guided Tour of Datacenter Networking and The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines and Doing redundant work to speed up distributed queries and I love Solar Power But....

   Some of the most important work in systems software right now is occurring in support of these data center applications: data management, parallel algorithms, virtualization, networking, security. Internet-scale Cloud Computing is driving all of these areas.

2. However, I am concerned about efficiency, and I am concerned about the environment, and I am concerned about waste and pollution.

   I realize that I use services like Google, Amazon, and Facebook a lot, and I know that I often don't stop to think about using those services. I waste my use of those services. I am totally spoiled by their always-on availability, their near-instant response, and their exhaustive and complete coverage.

   So, am I contributing to the problem? I fire off my requests to the Internet without a second thought, and they respond by provisioning massive capacity to serve the demands of millions of people like me.

   I'm not an economist, but it seems like this is one of those cases where the "externalities" are not right: the people who are driving demand for these services (people like me) aren't the ones who are directly paying for the infrastructure, nor are we aware of the bloated consumption that we're triggering by our actions.

So, as I said earlier, I'm glad that the New York Times is raising these issues.

Just as they did last spring with their investigations into working conditions at the manufacturing facilities in China and elsewhere which support our Internet life, their investigations into the computing infrastructure which gives us our photo walls, our music streams, and our news feeds is valuable and important.

I hope the discussion continues.