Some benchmarking principles

Here's a nice slide deck on general benchmarking principles that I came across recently: Tokutek benchmarking principles

For hard-core perf types, there's not much new here, but I thought it was worth passing along in case some found it interesting.

Among my favorite observationss from the slide deck:

• benchmark frequently, to catch performance regressions soon after they occur
• keep a benchmark history, for analysis of data over time
• use graphs and plots to help with data interpretation
• share benchmarking data widely ("developers love data")

At various companies, at various times, I've worked with benchmarking teams, and it's nice to see the slow-but-steady "systematization of knowledge" going on here, since some of these are hard lessons and it's worth your time not to re-learn them yourself.