So, my thoughts on the matter are summed up pretty well in this tweet by @luisobo. But since Quick is built atop of XCTest, it'd be great to provide fun DSL's for all of XCTest's features, including performance tests.
Not sure what the DSL should look like, though.
modocache
All 10 comments
Calling measureBlock on an instance of XCTestCase outside a conventional XCTest method (e.g. -testFoo) results in a thrown assertion鈥攕omething about numberOfTestIterationsForTestWithSelector: with a null argument returning 0 instead of 1. This is the case whether the it spec is inside a call to QuickSpec.spec or QuickConfiguration.configure.
As such, it seems that the best approach for now is to have separate XCTestCase subclasses where you put your performance tests. Would be nice to figure out what dark incantations are required to get this to hook into XCTest's strange machinery.
For kicks, I found that mixing Quick and XCTest functions in the same class doesn't work, probably due to the way QuickSpec is overriding +[XCTestCase testInvocations].
bgerstle
on 13 Apr 2015
@bgerstle I'm not entirely certain it's related to +[QuickSpec testInvocations]--you may be describing #121. Fixing that issue may be the key to getting this working.
modocache
on 13 Apr 2015
On OS X (and in the iOS Simulator as a consequent, and probably on ARM too ), it's possible to use wall-time compensated rdtsc via mach_timebase_info() from xnu to get extremely precise timings. See also:
steakknife
on 17 Jun 2015
What's the consensus on this? Have opinions/feelings changed?
noisypigeon
on 28 Oct 2016
My thoughts _have_ changed, as a matter of fact! I think Nimble should support performance characteristic expectations, like Ruby's MiniTest does. Something like this:
expect { foo() }.to(perform(.linearly))
Basically, I think it'd be cool to do this in Swift, and it'd be neat if Nimble did that. :)
Thoughts?
modocache
on 28 Oct 2016
Oooh! That would be quite rad. I've assigned a bunch of issues to myself so at least they're assigned _to someone_. I'm going to try to resolve as many of them as possible and it would be cool to build something like this. Would you be able to help in the implementation side of things?
noisypigeon
on 28 Oct 2016
I'm all for it. Would love some help building it though. 馃拑
noisypigeon
on 28 Oct 2016
Sure thing! Post any questions you have here.
modocache
on 28 Oct 2016
馃槄 Nothing like a 6 year old issue to remind you how long a library has been around!
Just started exploring Quick/Nimble (unsure why I've never tried it in the past, loving it so far!) and have been exploring how best to do performance tests. It looks like this went nowhere (no worries!) but was wondering if there were any more up-to-date workarounds or extensions people are using?
In my case I'm more looking for something akin to Apple's own measure block with the ability for it to let me know if I've made a function considerably slower. I may just drop back own to a standard XCTestCase for these tests, but if there are random branches/gists etc then I'd love to trial them 馃 Thanks
Sherlouk
on 2 May 2021
Related issues
modocache
路
6Comments
ivangodfather
路
7Comments
ivanruizscm
路
7Comments
lilyball
路
4Comments
sunshinejr
路
4Comments
Most helpful comment
My thoughts _have_ changed, as a matter of fact! I think Nimble should support performance characteristic expectations, like Ruby's MiniTest does. Something like this: