We'll have a few more of these after #1198 goes in, but plotting and some of the rolling aggregation tests already fall in this category.
For development purposes, it would be nice if these don't run automatically so the whole test suite can finish in handful of seconds. They should still always run on Travis-CI and Appveyor, which take a few minutes to setup.
See
http://doc.pytest.org/en/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option for guidance from pytest.
@shoyer, this seems pretty straight forward from http://doc.pytest.org/en/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option-- all we need is to define the @slow decorator and mark the slow tests, correct? This seems like a fast PR write-up / fulfill following merge of #1198 unless I'm missing something.
I would have a preference for py.test --skip-slow to explicitly skip, rather than default skipping.
And then unit tests _should_ be fast, this is only for tests that really can't be made fast