Cocoapods: Xcode 7 Code Coverage

[ZevEisenberg]

I'm developing an open-source library, and I want to run Xcode 7's new code coverage on my unit tests. You can download the library at https://github.com/zeveisenberg/bonmot if you want to follow along at home.

I opened the example project, went to Product → Scheme → Manage Scheme, went to the Test pane, and checked the "Gather Coverage Data" box. When I ran my tests, I got coverage data for all the classes in the sample project (cells, views, view controllers, main.m), but nothing for the classes in my library (BONChainLink, BONTextAlignmentConstraint, BONTextConfiguration, UIImage+BonMotUtilities). Has anyone had any luck with this? I tried enabling the "Generate Test Coverage Files" flag in build settings for the Pods Xcode project, but no dice.

Does anyone know if this is an issue that can be resolved by having CocoaPods change some build setting?

[Ashton-W]

Have you tried with use_frameworks! in the Example podfile?

If that doesn't work, I would not follow the standard pod lib create structure and create a project for your tests that compiles your sources directly. Note that Xcode 7 coverage doesn't work yet work for static libraries (beta 2), so it will need to be an app or framework target you end up testing.

[ZevEisenberg]

I just tried use_frameworks!, and coverage for the dev pod didn't work, even when I went through each scheme and enabled "Gather coverage data". I can create a separate project, and perhaps I will, but it would be great if the default pod lib create project would support this.

[neonichu]

@ZevEisenberg do you have any idea what the difference between the pod-template project and one that works with code coverage is?

[Ashton-W]

Pod lib create has already been updated to create a framework target, hasn't it?

[ZevEisenberg]

@ashton-w I believe so. I created the library with an older version of CocoaPods, so the next thing I’m going to try is to pod lib create it from scratch.

@neonichu I do not.

[segiddins]

Is it possible that Xcode's code coverage only applies to a single target?

[ZevEisenberg]

I suppose, although I doubt it given Apple’s recent introduction of frameworks and their recommendation to use them to break projects into complements. You may find more here: https://developer.apple.com/library/ios/qa/qa1514/_index.html

[ZevEisenberg]

From the Xcode 7b3 release notes "known issues" section:

Testing
• Xcode does not show code coverage information for source files in static libraries. (15605406)
Workaround: Add the source files directly to application or framework targets.

Not clear if this is slated to be fixed before the GM.

Edit: Not sure if this has any implication on frameworks in a CocoaPods context.

[ZevEisenberg]

From the Xcode 7b5 release notes Resolved in Xcode 7 beta 5 — IDE section:

Code Coverage now supports files in static libraries. The source files show up under each binary that links the static library in the Coverage tab of the Test Report. The source editor shows coverage numbers aggregated across all binaries. (21984681)

[segiddins]

@ZevEisenberg can you confirm that the latest beta fixes things for you?

[ZevEisenberg]

@segiddins oops, I forgot to mention the most important part: despite the promising release note, the latest beta did _not_ fix the problem for me, even when I checked the "gather coverage data" box on all my schemes, including the CocoaPods-generated ones.

[ZevEisenberg]

Oh, and I also updated the example project in my pod to the latest release of CocoaPods, just to be sure. Still no go.

[Ashton-W]

Is it fixed for non-cocoapods or otherwise standard Xcode projects with library targets?

[ZevEisenberg]

Not sure. I never tried to get it working with a static library other than with CocoaPods. @ateliercw when you got test coverage working with my library, that was by just wiring it up yourself with a framework, right?

[ateliercw]

That's correct.

On Wed, Aug 12, 2015 at 10:39 PM Zev Eisenberg [email protected]
wrote:

Not sure. I never tried to get it working with a static library other than
with CocoaPods. @ateliercw https://github.com/ateliercw when you got
test coverage working with my library, that was by just wiring it up
yourself with a framework, right?

—
Reply to this email directly or view it on GitHub
https://github.com/CocoaPods/CocoaPods/issues/3706#issuecomment-130510855
.

[Goles]

I have a static library target that's not cocoa-pods generated, and I'm not getting any coverage information either... I opened a radar (22764568) for reference.

[nikita-leonov]

I have the same issues with libraries created with 'pod lib create'. Coverage collected for example project and test bundle itself, but not for testable framework. I didn't find a way yet how to workaround it.

[RaviDesai]

For my framework (called Framework) pod lib create created 2 schemes (in addition to all the pod schemes). One was called Framework and one was called Framework-Example. In addition, it created two targets, one called Framework_Example, and the other called Framework_Tests.

I have the same issue as the other posters, however, when I switched from building and testing using my Framework-Example scheme to building and testing with my Framework scheme, I was able to get coverage data on the Pod files themselves. In order to do this, I had to add the Framework_Tests project to the Tests for that scheme (they were by default empty).

The other issue is that whenever I run pod update, the Framework scheme is rewritten, and the Generate coverage data checkbox is removed, and the Framework_Tests target is removed from the list of tests to run.

[RaviDesai]

So what seems to fix it in a slightly better way is to add your Framework library to the Framework-Example scheme's Build targets. This survives running pod update on your CocoaPod.

[neonichu]

Thanks @RaviDesai - that sounds like something we should be able to incorporate into the Pod template

[nikita-leonov]

I followed @RaviDesai suggestion and it works for me as well. There are two important things to notice:

1. "Gather coverage data" should be enabled not only for Library, but also for Example app.
2. Coverage data will be collected not only for Library and Example app , but also for all Library dependencies.

Second item is something that I really want to suppress, but didn't find a proper way how to deal with it yet.

[ZevEisenberg]

@RaviDesai :+1: :100: that magic formula worked for me. @neonichu I would love to see that incorporated into the default template (or part of the pod lib create setup prompts).

[segiddins]

@ZevEisenberg a PR to CocoaPods/pod-template making that change would be ace

[ZevEisenberg]

@segiddins I will if I can work out exactly how this works. I reproduced @RaviDesai's steps, but I had twiddled a bunch of dials so I wanted to make sure I could reproduce it with the fewest possible changes. I cleaned the build, added use_frameworks!, ran pod install, and tried to build the example project, and it failed:

Ld /Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Products/Debug-iphonesimulator/BonMot.app/BonMot normal x86_64
    cd /Users/zev/Projects/BonMot/Example
    export IPHONEOS_DEPLOYMENT_TARGET=8.0
    export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator9.1.sdk -L/Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Products/Debug-iphonesimulator -F/Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Products/Debug-iphonesimulator -filelist /Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Intermediates/BonMot.build/Debug-iphonesimulator/BonMot.build/Objects-normal/x86_64/BonMot.LinkFileList -Xlinker -rpath -Xlinker @executable_path/Frameworks -Xlinker -rpath -Xlinker @loader_path/Frameworks -mios-simulator-version-min=8.0 -Xlinker -objc_abi_version -Xlinker 2 -framework BonMot -fobjc-arc -fobjc-link-runtime -framework CoreGraphics -framework UIKit -framework Foundation -lPods-BonMot -framework Pods_BonMot -Xlinker -dependency_info -Xlinker /Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Intermediates/BonMot.build/Debug-iphonesimulator/BonMot.build/Objects-normal/x86_64/BonMot_dependency_info.dat -o /Users/zev/Library/Developer/Xcode/DerivedData/BonMot-fdhxdbzgpvcvrfcxzzqajofglwgw/Build/Products/Debug-iphonesimulator/BonMot.app/BonMot

ld: library not found for -lPods-BonMot
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Something must have been cached or something. Searches for the error found unrelated things. Wondering if I should try recreating the sample project with the latest version of CocoaPods? Is Is there a way to do that without just creating the library from scratch and adding the files by hand?

[segiddins]

Not that I can think of

[ZevEisenberg]

Managed to get it working by completely rebuilding the Xcode project from a clean pod lib create, then porting over all the files by hand. Not fun, but it seems to be working. I'll look into adding a PR.

[segiddins]

:+1: awesome, thanks!

[segiddins]

Closing in favor of https://github.com/CocoaPods/pod-template/issues/135.

[jacobvanorder]

@RaviDesai's instructions worked but were a little opaque for me.

In both your example app's scheme and the generated library, enable code coverage. Edit the scheme for the library and go to Test. Under “Info” select the “+” button. Within that dialog, you should see your Test Target for the Example app. Add it!

Clean, build, rinse, repeat.

[bencallis]

@jacobvanorder Thanks for giving a clearer guide.

Is it possible to get code coverage for private files in the development pods?

[jacobvanorder]

@bencallis ¯_(ツ)_/¯