Carthage: Skip schemes

[mstfy]

When I build RxSwift with carthage it builds RxSwift, RxCocoa, RxBlocking and RxTests. Is there a way to tell carthage to build only RxSwift and RxCocoa and skip other schemes?

[mdiep]

No, there's not. :relaxed:

[robertjpayne]

@mstfy I have a similar need and I've simply scripted Carthage via a Rakefile and the subprocess gem. Luckily schemes are located inside *.xcodeproj/xcshareddata/xcschemes/**and are pretty easy to find and remove.

[jeeftor]

so you do a carthage update --no-build -- delete schemes followed by a `carthage build ?

[Antondomashnev]

@robertjpayne can you please elaborate on your solution, would be helpful, because I have similar problem.

[lukescott]

I have a similar problem with https://github.com/aws/aws-sdk-ios

They have one project file, but multiple build targets. It builds all the frameworks:

*** Building scheme "AWSDynamoDB" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSMachineLearning" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSCognitoIdentityProvider" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSAutoScaling" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSCognito" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSAPIGateway" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSSQS" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSSNS" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSKinesis" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSCloudWatch" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSSES" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSElasticLoadBalancing" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSLambda" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSIoT" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSCore" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSSimpleDB" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSEC2" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSS3" in AWSiOSSDKv2.xcodeproj
*** Building scheme "AWSMobileAnalytics" in AWSiOSSDKv2.xcodeproj

Which takes about 30 minutes or more. I just want S3.

[mdiep]

In that case, I'd suggest using something like https://github.com/guidomb/carthage_cache or https://github.com/146BC/RomeBuild.

[Antondomashnev]

@mdiep thanks, that's something at least. I think I'll take a look on Carthage itself and maybe it's not that hard to add scheme(s) parameter to build option. Will follow-up soon 🚀

[guidomb]

@Antondomashnev Hey! I am the author of CarthageCache and I am facing the same issue. AWS takes 30 minutes, and although thanks to carthage cache we can avoid having to wait 45 mins for the entire build, every now and then we need to update AWS. Did you have the chance to take a look at adding a --scheme parameter?

[Antondomashnev]

Hi @guidomb, I've recently implemented the first step in the Commandant and I haven't started yet with the --scheme parameter implementation. I'm also wondering about the solution proposed in #1616, maybe this is a better option but I have to read it carefully.

[Antondomashnev]

Just a follow-up, I'm planning to propose an update to Carthage, but before I require this PR to be merged since the Commandant with my update is in Swift 3.

[fostah]

Just a wanted to post a work-around that I implemented for AWS. It's a makefile target.

carthage-update:
    @cd $(IOS_DIR) && carthage update --no-build 
    @cd $(IOS_DIR)/Carthage/Checkouts/aws-sdk-ios/AWSiOSSDKv2.xcodeproj/xcshareddata/xcschemes && \
        find . ! -name "AWSAPIGateway.xcscheme" ! -name "AWSCore.xcscheme" ! -name "AWSIoT.xcscheme" -delete
    @cd $(IOS_DIR) && carthage build --platform iOS

I hope it helps until a more permanent solution exists.

EDIT

It's actually better to run carthage update --no-build instead of carthage checkout. The former will generate the .resolved file that's needed.

[freak4pc]

Was just looking in Carthage and bumped into this, so practically we would need to install the entire AWS SDK which includes about 30-40 different sub-SDKs, just to use 3 or 4 of them? That seems excessive... Would also make an issue with CI unless the CI itself would also cache the assets (or we would commit the actual built binaries)

Are there any thoughts on how to work around this?

[mdiep]

You can try the new --use-cache option.

Otherwise the Makefile solution shared by @fostah looks fine.

This stinks, but it's really an issue with the AWS SDK—not with Carthage.

[freak4pc]

Well it is definitely an issue of AWS but that kind of granularity doesn't seem like an excessive feature perhaps? I think many of the bigger companies make SDKs this way (Google and Facebook as well). Definitely not saying that is any good but perhaps it's worth discussion of how to achieve that granularity in some way?

[dcaunt]

--cache-builds rather than --use-cache 😃

[BobElDevil]

So I think one of the problems that keeps coming up here and the various dupes is the talk of who is doing the scheme specification:

• The consumer of the dependency: This I agree with the historic answer of "Nope. Never gonna happen." This introduces quite a bit of complexity if you have a common dependency with different requests for schemes in two different downstream projects. There may not be a 'right' answer of what to build in cases like that. In addition, it requires digging into the resolver logic itself to figure out what to build. Since one of the main tenants of carthage is simplicity, this is really a no-go
• The dependency author themselves: This I think makes sense. The author may have schemes that are shared for other developers, but should not be exported to consumers. By limiting it to the author themselves, then there's no resolution time calculations, it's merely an extra filter to apply at build time. Additionally if problems do arise due to 'missing' frameworks, it's a problem to be taken up and addressed by the framework author themselves, since it's a one size fits all solution, as it is today, with just an extra bit of customization.

I've run into situations myself where it would be nice to have a 'workspace that builds all carthage dependencies directly' for development of a framework, but then just having a definitive project scheme for the framework itself. Currently we get around that by getting fancy with symlinks and generated schemes, which is definitely not ideal.

@mdiep If we limited this discussion to the second bullet point (via some sort of cartfile ignore file or something), do you think that would be something we would be interested in supporting in carthage?

[mdiep]

I do think that makes some sense. I definitely agree that this is a framework author issue; not a framework consumer one. And ignoring certain schemes definitely seems like the easiest way to do that.

Where this has been stuck in the past is in how to specify that. In the past, we've talked about a general file that can accept these sorts of things using OGDL. (You can search issues/PRs for mentions of this.) I don't know whether that actually makes sense or not.

We can consider a reasonable proposal for how to specify that. (It might be done best as a PR to the documentation.)

[Jeehut]

@BobElDevil @mdiep I agree with both of your points, that we could improve skipping development schemes for framework projects further. But I don't think that would solve the actual issue here. So I highly disagree with statements like "this is a framework author issue; not a framework consumer one". Let me bring up a very recent discussion as an example why: https://github.com/Moya/Moya/issues/1125.

Here's a summarization of the issue with Moya from my point of view:

• Moya is a framework with the goal of being usable both with and without reactive programming
• Moya supports multiple reactive programming frameworks (ReactiveSwift & RxSwift)
• Moya tries to keep contributing & maintenance as simple as possible and therefore doesn't split their project into separate repositories
• To reach these goals, Moya has three targets ("Moya", "ReactiveMoya" and "RxMoya")
• Each target has a different set of dependencies
• When Moya is used as a dependency within an app project, all of these three schemes are built, including all of their sub dependencies
• A consumer always wants only one of those targets and its dependencies
• A consumer currently always has to wait more than three times longer than needed if he doesn't want to use reactive programming (actually even more than 10 times longer for this case!)

Moya is just a current example here, other frameworks have similar issues, but I find Moya to be a good mixed example since it both has multiple targets itself and different sub dependencies for each target.

I think any solution that would prevent all frameworks from building, which I don't want to be built as a consumer of the framework, would be a great addition to Carthage. That's what I thought this issue (and several others) was about. And I also strongly believe that a package manager should support as many project structures as possible, so long as this does not interfere with the core goals of the project.

This brings me to the point of what the core goals of Carthage actually are – this is what I observed:

• Flexibility
• Unintrusiveness
• Simplicity
• Creates no extra work for framework authors

All of these goals can also be found to be clearly stated in the README.

---

Now, let's see the current state of this framework regarding those goals and the inclusion of Moya:

• [ ] Flexibility: Consumers cannot include just specific schemes/frameworks
• [x] Unintrusiveness: No problems here
• [x] Simplicity: It sure is simple
• [ ] Creates no extra work for framework authors: Not at all, see alone the fact that Moya is discussing to split their project for supporting good Carthage build times ...

---

I don't know about any other suggestion, but see my suggestion on how we could solve this here. Maybe it's not the best solution, but let's see how it serves the goals:

• [x] Flexibility: Consumers can exclude any frameworks they like
• [x] Unintrusiveness: No problems here
• [x] Simplicity: It still is very simple, just adding a new file
• [x] Creates no extra work for framework authors: Moya doesn't need to be changed at all

So, to me there's no reason for Carthage not to add such a feature. Every issue I read so far was about "dependency resolution hell" (e.g. the "The consumer of the dependency" section of @BobElDevil's comment above) – but my suggestion clearly states that the new Cartfile.ignore would only be looked at within the current project (just like Cartfile.private is) and therefore no such issues should arise at all.

---

This is what I think. I'm really interested in your opinions. I'd be willing to tackle this feature myself but of course would only do this if there's a chance to be merged.

[mdiep]

So I highly disagree with statements like "this is a framework author issue; not a framework consumer one".

A better statement to make would probably be: There are 2 orthogonal issues here: (1) framework authors who have schemes that should never be consumed and (2) consumers who wish to avoid extra work during building.

I'm sympathetic to (1), but feel that (2) is adequately addressed by binaries and caching.

[Jeehut]

The _problem with binaries_ is that they only work if the exact same version of Swift was used, so it doesn‘t work in many situations (like right now with Xcode 9 beta). Also this creates extra work for framework authors.

_Caching_ on the other hand has the problems of not being applyable in many cases, too: On first build, on framework updates and on Swift version upgrades.

So, to me those are nice features that will help out in some situations. But they don‘t really solve the problem. Why are you so reluctant against my simple but problem-solving splution?

I‘d be happier to see arguments against my suggestions instead seeing statements over and over again saying that this is _not a real issue_ and was _already solved_: It was not, I know it, I can feel it in my daily work and I can see it in those many issues opened about this and getting many upvotes on both Carthage and many other projects.

[Jeehut]

I've started exploring my suggested option via #1990. It's far away from completion but feel free to give feedback early as I go. Any help is welcome!

[tmspzz]

If you don't want to add another "Cartfile.xxx`, ignoring a scheme could also be specified in the scheme itself as a pre-build action. Carthage would have to parse the scheme and look for something like this:

+      <PreActions>
+         <ExecutionAction
+            ActionType = "Xcode.IDEStandardExecutionActionsCore.ExecutionActionType.ShellScriptAction">
+            <ActionContent
+               title = "Run Script"
+               scriptText = "# CARTHAGE_IGNORE">
+            </ActionContent>
+         </ExecutionAction>
+      </PreActions>

Another solution for caching: Rome (popular name)

[Jeehut]

@blender Does this need to be done within the framework, or can I do that as a consumer of the framework, too?

Also note that I explained above, why caching IMHO does not solve the real problem.

[tmspzz]

I believe it can be done by both sides.

As an alternative why not just pass scheme ignore names on the command line? like [--ignore-schems] [SCHEMES]]

[Jeehut]

Passing them as a command line argument would mean that you have to remember for each project which schemes you want to ignore and type them over and over again. That's simply impractical.

A new file would prevent this – cause the use case I'm talking about (and people are requesting) is when you never ever want to build specific schemes in the context of that project.

How is it done from the side of the framework consumer, I didn't quite get that yet ... don't you edit the frameworks code so that the change would be gone the next time I update the code via Carthage?

[tmspzz]

I agree that the .ignorefile is a better solution. Command line arguments are a nice to have :)

Here is how one could do it consumer side by setting ignores in the scheme

carthage checkout ...
./set_ignores
carthage build ...

[Jeehut]

Sorry, I still don't get it. But never mind, I'd favor the Cartfile.ignore file even if this would work somehow, cause it definitely seems more complex to do than the simple .ignore file.

[lukescott]

For the AWS issue, how is the framework author supposed to address the issue? I only want S3, but Carthage builds evening and takes 30 minutes. Are they supposed to have separate repos? Monorepos are quite common and are much easier to manage. So I don't see them changing that.

Caching is a nice idea, but you still have to build it the first time. The only real solution is the Makefile script that deletes everything else but the schemes you want. That's a hacky workaround though.

Carthage is a tool for the consumer, not the framework author. It is supposed to make managing dependencies easier. To me that means providing reasonable options to the consumer to do that. You can't always dictate one way of doing things to framework authors.

For me the most ideal solution is adding a comma delimited list of Scheme names in the Cartfile. The Cartfile is the package file that lists dependencies. Building schemes are part of that - for AWS my project depends on S3, no CloudFront. That should be formally declared in the package file of the project, not hidden away in a script file, or ignored by caching. It documents what my project actually depends on.

[Jeehut]

@lukescott I think the problem with placing it into the Cartfile was (as stated in many different places by the maintainers) that it adds complexity to a file, which is also part of the dependency graph (meaning the Cartfile is looked for every framework dependency).

Using my suggestion the AWS issue would be solved by creating Cartfile.ignore with this content:

scheme "AWSCore"
scheme "AWSAutoScaling"
scheme "AWSCloudWatch"
scheme "AWSCognito"
scheme "AWSCognitoIdentityProvider"
scheme "AWSDynamoDB"
scheme "AWSEC2"
scheme "AWSElasticLoadBalancing"
scheme "AWSIoT"
scheme "AWSKinesis"
scheme "AWSLambda"
scheme "AWSMachineLearning"
scheme "AWSMobileAnalytics"
# scheme "AWSS3"
scheme "AWSSES"
scheme "AWSSimpleDB"
scheme "AWSSNS"
scheme "AWSSQS"

Only AWSS3 would be built then for the aws-sdk-ios dependency.

Note that you could of course remove the entry # scheme "AWSS3" entirely, the point is just that it should not be part of the ignore file. This solves the problem without introducing any problems with the dependency graph: Cause it is handled by Carthage just like the Cartfile.private is handled – only locally in your project where you consume the framework.

I agree though that specifying the scheme within the Cartfile would solve this issue, too. I just don't see that coming anytime soon (if ever).

[lukescott]

Sure, it would solve the problem, but it's actually much more complex because:

• You have to list every scheme you don't want.
• If a new scheme is added later it would build without adding it to the filter. When a new Amazon product comes out this will happen.
• You are disassociating the schemes with the repos.

An inclusive filter would be better and solve the first two:

scheme AWSS3

The third one is solved simply appending another option to the dependency in the Cartfile. I'm not sure how that is any different than a separate file, other that another file just adds confusion. And a build scheme is a dependency, so it should be treated as such. In a dependency graph AWSEC2 should not be there when I only depend on AWSS3.

[Jeehut]

I think the problem with the AWSS3 scheme is: What if AWSEC2 needed a sub dependency, say Alamofire? How would Carthage know that the scheme "AWSS3" doesn't need that dependency? This would only be possible if the Cartfile in the aws-sdk-ios project looked more like a .podspec file, see the subspecs used in Moya.podspec for an example.

As I stated earlier, "Simplicity" and "Creates no extra work for framework authors" are goals of Carthage and such a complicated Cartfile definition would go against those goals. If you find such a structure better, you probably are better off using CocoaPods anyways.

Additionally, let me also explain why an exclude filter is better than an include filter:
Given the above example, where you just want AWSS3 and there's AWSEC2 in the project, too. Say now that you allow building the scheme "AWSS3" only, what if they add a dependency as a requirement for "AWSS3" in the future? Let it be Alamofire, then it would break the next time you update your dependencies and it would be hard to know what scheme you are missing.

In my opinion it is much better to have a new service that you are not ignoring yet (only downside: a little longer build time) than to have a non-building project (a major issue). And updating the Cartfile.ignore should be straightforward, too. Since Carthage is printing everything it's building, the person who updates the dependencies should recognizes fairly quickly that a new unexpected framework is being built.

Also in my experience most of the time it's less than 5 schemes that you want to ignore, but those usually need long compile times (like RxSwift / ReactiveSwift). The AWS case is the exception here, I think.

[lukescott]

I think the problem with the AWSS3 scheme is: What if AWSEC2 needed a sub dependency, say Alamofire? How would Carthage know that the scheme "AWSS3" doesn't need that dependency? This would only be possible if the Cartfile in the aws-sdk-ios project looked more like a .podspec file, see the subspecs used in Moya.podspec for an example

I think we're talking about two different things here: Resolving (downloading) dependencies vs building. Downloading dependencies takes < 5 minutes. Building everything takes 30. I don't particular care if Alamofire is downloaded, even if it's not used. What I care about is reducing build time of the most expensive operation. Perhaps resolving dependencies is a later optimization, but not one I feel is particular important right now.

Additionally, let me also explain why an exclude filter is better than an include filter:
Given the above example, where you just want AWSS3 and there's AWSEC2 in the project, too. Say now that you allow building the scheme "AWSS3" only, what if they add a dependency as a requirement for "AWSS3" in the future? Let it be Alamofire, then it would break the next time you update your dependencies and it would be hard to know what scheme you are missing.

And if one of the schemes I added to the ignore file now becomes a dependency of AWSS3? I would much rather include AWSCore in my list than try to figure out what I can exclude. It is much easier to figure out. I would start by adding AWSS3 and get build errors complaining about a particular dependency(s) being missing, then add them to the list to include. Or the author can document AWSCore, etc. is used by all api frameworks.

Inclusive also makes it clear what the project actually depends on.

[Jeehut]

There must be a misunderstanding here, maybe because I used the term "dependency". We were talking about the same thing, I don't care as much about downloading the dependency either. I was talking about building, too.

If AWSS3 sometime in the future relies on Alamofire as a dependency, then Alamofire would need to be both downloaded and built alongside AWSS3. Otherwise AWSS3 won't build – and I got the feeling that figuring that issue out with several dependencies that are being updated can be way more error-prone and exhausting than it sounds from you right now. Xcode does not always show errors with a useful description when sub dependencies are missing. Keep in mind that some projects have several dozens of dependencies and are only first updated again after one or two years.

Regarding the other point: Yes, that could happen. If AWSEC2 depends on Alamofire and I exclude it within the Cartfile.ignore then sometime later AWSS3 could depend on Alamofire, too. In that case you're right, I would have the same problem as described above, with two key differences though:

1. Since there are far more frameworks out there that could be potentially included as new frameworks than there are frameworks that I might be already ignoring, this problem will occur by far not as many times as it would with an inclusive list solution.
2. The frameworks which are being ignored explicitly listed in the Cartfile.ignore. This means if I have reason to believe that a framework might be missing, I can simply check the dependencies that I'm ignoring rather than read through all the release notes of all my dependencies that I'm specifying. Since, again, I usually will have less dependencies that I'm ignoring, the issue is much quicker to solve in this case.

[Jeehut]

Inclusive also makes it clear what the project actually depends on.

Again, I'm not against this, but then it should be made within the Cartfile (not a different place) since that's already the place where we define "what we depend on". And that has already been discussed and rejected in #437. My approach is to find an alternative solution that's not going to be rejected due to its complexity.

[lukescott]

Again, I'm not against this, but then it should be made within the Cartfile (not a different place) since that's already the place where we define "what we depend on".

I would like to see that. IMO, it's the most intuitive option.

And that has already been discussed and rejected in #437. My approach is to find an alternative solution that's not going to be rejected due to its complexity.

It's not clear to me how/why that's complex, or why such complexity is not a worthy problem to solve. I mean, clearly it is a problem that needs to be solved, and people are already doing it. If a blunt method of deleting schemes before Carthage builds works, why can't Carthage simply (for now) filter build schemes?

I understand that there can be a ton of edge cases related to this. What to build and in what order, etc. Instead of starting with a more complex solution, can't we start with something simple that solves the most obvious use-case, and then slowly build on it?

[Jeehut]

I just searched the issues and couldn't find one that was specifically related to adding a scheme option to the Cartfile (for both framework consumers and authors). I found #1616 but I got the feeling that it's not about that in the end.

@lukescott Would you like to open a new issue then with the more specific suggestion of specifying schemes in Cartfiles? It would not help prevent building sub-dependencies like in the case of Moya (where RxSwift and ReactiveSwift take long build times). But as you said: Having that option would be the first step and solve cases like aws-sdk-ios and then we can build on that and see if we want such a structure in the Cartfile like the Podfile already offers via subspecs.

[lukescott]

I opened this two years ago:

https://github.com/Carthage/Carthage/issues/728

Although the cited example was SQLite at the time.

[Jeehut]

And your issue is yet another proof that the project maintainers simply don't want this feature as they think it's enough to work around it by using binaries and caching. I've already answered why those two are not a solution to this problem above but didn't get an answer on that.

So, I'm back to my alternative solution with the Cartfile.ignore as a pragmatic approach that doesn't add "complexity" to the Cartfile, which @mdiep clearly refuses to accept. I already started implementing it in #1990. I'm gonna complete that one over time and if that's not going to be merged, then only because there will be work started on scheme support on the Cartfile already, I hope.

If the feature gets rejected entirely, I might consider forking Carthage, rename it to CarthagePro and maintain that one on my own, rebasing it onto the original Carthage project from time to time. I'm pretty sure I'll get more than 500+ stars for that project within a year or so ...

I've not seen real issues from your side on my solution either, you're simply saying "let's fix the problem in the Cartfile" – which I'm totally okay with, but again, let's be pragmatic... if you have improvement suggestions on #1990 please post them there.

Thanks for the discussion anyway! Good to hear that others see this problem, too ... for me this is now the number one drawback of Carthage (previously it was the missing --cache-builds option).

[lukescott]

I haven't had the chance to fully look at this yet, but I'm looking at https://github.com/apple/swift-package-manager as a possible alternative.

[Jeehut]

Sure SPM is gonna be great once it's production ready for iOS projects. There's no official support for iOS projects yet though, they are concentrating on the server-side Swift (as there's no package manager for server-side Swift at all). My best guess is that it's going to be the go-to-solution in October 2019. Until then we're stuck with Carthage and CocoaPods.

Note though that it's technically possible to use it for your iOS project already – it just isn't that much fun. See this gist for more details on how to get it to work (I've not tried it though).

[mdiep]

Caching on the other hand has the problems of not being applyable in many cases, too: On first build, on framework updates and on Swift version upgrades.

Those are relatively rare events. They're definitely not daily or even weekly occurrences.

So, to me those are nice features that will help out in some situations. But they don‘t really solve the problem. Why are you so reluctant against my simple but problem-solving splution?

Because it adds overall complexity to the project for a small subset of users.

I maintain this project mostly in my free time (my employer generously allows me to spend a small portion of my day responding to issues and PRs). We have a small number of contributors and maintainers. I don't think any of them are paid to work on Carthage. There's a limit to our ability to maintain the project.

Every feature like this, no matter how small, contributes to the overall complexity of the project. That means it's harder for reviewers to review and test. It's harder to verify that nothing is broken between releases. It's slower to add important bug fixes. Carthage is already a difficult project to test. If the source was better factored for testability, then I'd be more open to adding additional functionality. But we have to be conscious of our constraints.

Ultimately this means balancing 3 factors:

• Reach: How many people are affected?
• Impact: How deeply are those people affected?
• Maintenance: How much harder does it become to maintain Carthage?

In this case, I'd say:

• Reach: Small. A small number of people are affected
• Impact: Low. This doesn't affect correctness or access to projects. It only affects build times in a small number of cases.
• Maintenance: This would be hard to test due to the current code structure.

That's why I'm not inclined to add support for framework consumers to ignore certain schemes.

[Jeehut]

Well, I understand all that (I’m doing all open source contributions in my free time) and yet again I have to completely disagree:

The reach is high: Every single developer I know of who has been using Carthage for more than 3 projects has already told me they dislike the fact Carthage builds stuff that is not really required. Think alone about how many users are including Moya or Realm into their projects. Every single one of them needs to wait much longer than needed.

Impact: Since for many users the build times have a factor between 2-5 times more than needed, I think the impact is not to ignore. Having to wait on CIs that do not support caching for example would be much less cumbersome. But true, it doesn‘t stop building or something, but that only tells me it‘s not a bug fix - it‘s an improvement.

Maintenance: Why would it be so hard to maintain this pretty independent feature. We only have to hook into the logic where the list of schemes are filtered, everything else is independent since it‘s a new file that I‘m suggesting. Also it sounds to me like „our code structure is not good, that‘s why we don‘t want any more features“. What‘s that for a reasoning? That could make sense if there was a plan to streamline the structure. But then you couln‘t say „we don‘t want this“ instead you could say „we don‘t want this now“. That‘s a huge difference.

[mdiep]

Also it sounds to me like „our code structure is not good, that‘s why we don‘t want any more features“. What‘s that for a reasoning? That could make sense if there was a plan to streamline the structure.

Unfortunately, I don't get to work on the Carthage code very much because my time is taken up by issues and PRs. (But if you look at my recent PRs, they're all aimed at improving the existing code and making it testable.) Contributions that improve the code are very welcome.

Since this project relies solely on volunteer effort, a plan isn't very valuable. People mostly contribute to scratch their own itches.

But then you couln‘t say „we don‘t want this“ instead you could say „we don‘t want this now“. That‘s a huge difference.

Okay: We don't want this _now_.

[tmspzz]

@Dschee won't running a script to delete the schemes you don't want to build work for you? Like suggested here: https://github.com/Carthage/Carthage/issues/1227#issuecomment-285800169

[lukescott]

@mdiep

I maintain this project mostly in my free time (my employer generously allows me to spend a small portion of my day responding to issues and PRs). We have a small number of contributors and maintainers. I don't think any of them are paid to work on Carthage. There's a limit to our ability to maintain the project.

I think many of us can understand this. You don't get to dedicate time you want to the projects you care about.

Unfortunately, I don't get to work on the Carthage code very much because my time is taken up by issues and PRs. (But if you look at my recent PRs, they're all aimed at improving the existing code and making it testable.)

Can you have some of the contributors respond and approve issues and PRs? If you don't have time to do it yourself, delegate.

Every feature like this, no matter how small, contributes to the overall complexity of the project. That means it's harder for reviewers to review and test. It's harder to verify that nothing is broken between releases. It's slower to add important bug fixes. Carthage is already a difficult project to test. If the source was better factored for testability, then I'd be more open to adding additional functionality. But we have to be conscious of our constraints.

Most projects are like this. In a perfect world TDD is applied in the beginning. Most projects don't do this. As a result, it's very difficult to reach near 100% test coverage for any project. But some projects get there, it just takes more than the efforts of a single person.

My biggest source of inspiration for this is the author of Babel. He started out in very much the same way, but realized that the project became bigger than him and he needed to delegate. Babel now has many major and minor contributors. It's an amazing project.

Since this project relies solely on volunteer effort, a plan isn't very valuable. People mostly contribute to scratch their own itches.

And this is because of two reasons:

1. People actively use Carthage and feel the pain of its limitations. Those who feel it enough try to contribute back in a way that helps them.

2. There is no clear direction of what your vision is for the project, so people gravitate towards solving their own problems.

I think that instead of trying to do everything yourself, you need to:

• Create a roadmap of where you want this project to go.
• List goals that you want to achieve for the project, and how you want to accomplish them.
• Ask for help and trust people to fill the needed responsibilities.

Contributions that improve the code are very welcome.

If you do the things I mentioned above, people will start to believe that this is true. When I opened my issue 2 years ago and was immediately shut down, I did not feel like this was the case, and I moved on as a result.

[mdiep]

Can you have some of the contributors respond and approve issues and PRs?

Absolutely. I try to do that as much as possible. Contributors are encouraged to respond to issues and approve PRs. The @carthage/carthage team is meant to have shared ownership over this project. It's not a dictatorship—I'm just the most active member of the team.

• @ikesyo responds to a lot issues, fixes a lot of bugs, reviews PRs, and does a lot of general maintenance.
• @BobElDevil has started contributing more PRs and started reviewing as well
• @jdhealy fixes a fair amount of issues and jumps in with some reviews
• @erichoracek has been fixing some performance issues and commenting on some things he's knowledgable about

Most projects are like this. In a perfect world TDD is applied in the beginning. Most projects don't do this. As a result, it's very difficult to reach near 100% test coverage for any project. But some projects get there, it just takes more than the efforts of a single person.

Absolutely. There are 37 open issues labeled _help wanted_. I've tweeted about wanting more contributors and maintainers especially, PRs that add tests and improve code quality. I try to encourage contributions as much as possible.

If you do the things I mentioned above, people will start to believe that this is true. When I opened my issue 2 years ago and was immediately shut down, I did not feel like this was the case, and I moved on as a result.

I'm sorry to hear that. 😞 We get a lot of feature requests from users, which we often have to say no to. (This is probably the worst part of maintaining a successful open source project. Saying no sucks.) My impression is that most users requesting features are not interested in contributing to the project. But I'm sorry if my saying no gave you the impression that your contribution wasn't wanted.

[Jeehut]

@mdiep You'd "_love to see PRs that add tests and improve code quality?_"

Well, tests I will add to the new feature I'm implementing in #1990.
And here you go regarding code quality, just for you: #2001.

I hope that it brings this project forward.

[ikzjfr0]

any update here?
i have the same issue with aws, i have --cache-builds, but i still have to build all schemes even though most of them are not required in my project.

[Jeehut]

Hi @ikzjfr0, sorry for not updating this previously, I've been on vacation the past few weeks and am now back at work. I will try to finish my PR #1990 off until sometime next week. Stay tuned! :)

[ikzjfr0]

@Dschee you are the star, let us know when it's ready :)

[DerekK19]

My solution (and it's a horrible one, but it works for my case) is to fork the repo and then strip the targets and schemes that I don't want from my copy of the repo. In my case I'm only using AWS for SNS, so I have cut down the project to only have AWSCore and AWSSNS. I also removed the AWSAuthSDK project. Of course I will have problems when I want to get AWS updates, but I can easily strip the project again after I pull changes from the upstream repo.

I'm happy to do this, because I'm only using this hacked AWS for my own private work

[raid5]

I've been dealing with a timeout issue on BuddyBuild due to the huge AWS SDK. Similar to @fostah's approach, I created a shell script for Carthage that BuddyBuild executes which strips out schemes I do not need to build.

Here is the script: buddybuild_carthage_command.sh

[Jeehut]

If anybody is interested in this feature, please help me out with my PR #1990 – I'm stuck with the reactiveness of this repo ...

[Jeehut]

Hey guys, I just finally finished my PR #1990 with help from the amazing @petester42.

I've tested it out and it's working great, so you can taste faster build times too, once it get's merged!
Just wanted to keep you updated! 😉

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Jeehut]

Issue is still apparent with many people waiting for #1990 to be merged but @mdiep being reluctant to merge as repeated again here lately.

[jdhealy]

Not just @mdiep — idk if ’reluctant to merge’ is the term to use, but might as well throw me in there too.

(Haven't posted in that PR as-of-yet, because @mdiep has summed up my thoughts so far, but I guess it bears repeating…)

[lukescott]

I'm not sure I like the solution presented in #1990. I would rather list the dependencies I use rather than the ones I don't want to use. Still, I would like a solution, even if it is put behind an experimental flag.

[JetForMe]

I'd love to be able to specify the schemes I want in my Cartfile. Right now, I have:

github "segmentio/analytics-ios" ~> 3.6
github "Alamofire/Alamofire" ~> 4.7.3
github "agilebits/onepassword-app-extension" "add-framework-support"

And I get:

$ carthage update
*** Fetching Alamofire
*** Fetching onepassword-app-extension
*** Fetching analytics-ios
*** Checking out onepassword-app-extension at "bcc4cc97fed9a6e73fa204f2e61138e353cb3ef7"
*** Checking out analytics-ios at "3.6.9"
*** Checking out Alamofire at "4.7.3"
*** xcodebuild output can be found in /var/folders/16/jfb809_s2fz01ql7k494f6pc0000gn/T/carthage-xcodebuild.Nl71yf.log
*** Downloading analytics-ios.framework binary at "Version 3.6.9"
*** Building scheme "Alamofire tvOS" in Alamofire.xcworkspace
*** Building scheme "Alamofire watchOS" in Alamofire.xcworkspace
*** Building scheme "Alamofire iOS" in Alamofire.xcworkspace
*** Building scheme "Alamofire macOS" in Alamofire.xcworkspace
*** Building scheme "OnePasswordExtension" in 1Password Extension Demos.xcworkspace

But I wish I could write something like this. This is akin to me specifying the version number: I need some a priori knowledge of the versions available, it's acceptable to rely on knowledge of the schemes available:

github "segmentio/analytics-ios" ~> 3.6
github "Alamofire/Alamofire" ["Alamofire IOS", "Alamofire watchOS"] ~> 4.7.3
github "agilebits/onepassword-app-extension" "add-framework-support"

and get

$ carthage update
...
*** Checking out Alamofire at "4.7.3"
*** xcodebuild output can be found in /var/folders/16/jfb809_s2fz01ql7k494f6pc0000gn/T/carthage-xcodebuild.Nl71yf.log
*** Downloading analytics-ios.framework binary at "Version 3.6.9"
*** Building scheme "Alamofire watchOS" in Alamofire.xcworkspace
*** Building scheme "Alamofire iOS" in Alamofire.xcworkspace
*** Building scheme "OnePasswordExtension" in 1Password Extension Demos.xcworkspace

You could get more complicated and abstract the notion of desired build platforms, but that requires the individual authors to do more to support it, whereas this should be transparent to anyone who doesn't care.

[yvbeek]

I'll add my use case here.

I'm working on a Swift library and it provides examples for iOS, macOS and tvOS. The examples require a bit of configuration (for example that building the example also builds any changes made to the library code) and I'd like to make the schemes shared so that I can commit them to GitHub.

A directive to skip building specific schemes in the Cartfile (of the library) would be great.

[sirghi]

From what I've skimmed through, this is a painful, long, and heated issue, so forgive me if I'm not fully aware of current state of things and the associated PR. I'll just add my use case and my 2 cents.

I'm developing framework A that relies on framework B, some schemes of which depend on other frameworks (C, D etc). Framework A only uses the scheme from B without dependencies. But every time I archive I have to wait for the other recursive dependencies to be built, which you have to agree, is painful (and when you're archiving caching doesn't help you there).

In my opinion, the solution to all of this boils down to adding subspec behaviour.

[KyleLeneau]

@JetForMe What you are asking for can be done today by using the --platform parameter. You use case would look something like this:

// This will skip the build of tvOS and MacOS like you described, absence of the --platform property will build every public scheme.
carthage update --platform ios,watchos

@sirghi I agree with you and I would like to further limit publics chems from building with some opt-in behavior like --platform provides. The use case I have is that GRDB.swift has 7 targets/public schemes (GRDBiOS, GRDBWatchOS, GRDBMacOS, GRDBCipheriOS, GRDBCipherMacOS, GRDBCustomiOS and GRDBCustomMacOS). Carthage is only compatible with the first 5, the last 2 (custom) are for custom SQLite builds and can only be done with manual installation (submodule) or Cocoapods. I would like a way to either tell carthage what schemes to build like a whitelist or to specify an exclusion list, personally I think the whitelist is better for the project overall).

Here is what I would propose for syntax:

Currently (Base Line)

As is, build all public schemes (no breaking change)

github "Groue/GRDB.swift" ~> 3.4.0

// Build all targets and variants
carthage update

// Build all targets and only iOS variants
carthage update --platform ios

Option 1: Specify whitelist in Cartfile's

Add whitelist support to Cartfile syntax

github "Groue/GRDB.swift" ["GRDB", "GRDBCipher"] ~> 3.4.0

// Build all targets based on whitelist rules for entries and variants
// Would build: `GRDBiOS`, `GRDBWatchOS`, `GRDBMacOS`, `GRDBCipheriOS`,  and `GRDBCipherMacOS`
carthage update

// Build all targets and only iOS variants
// Would build: `GRDBiOS`, `GRDBCipheriOS`
carthage update --platform iOS

Option 2: Specify the dependences whitelist outside of the Cartfile's

Add whitelist support to the CLI to that it could be passed in via another file or input...

github "Groue/GRDB.swift" ~> 3.4.0

// Build all targets based on whitelist rules for entries and variants
// Would build: `GRDBiOS`, `GRDBWatchOS`, `GRDBMacOS`, `GRDBCipheriOS`,  and `GRDBCipherMacOS`
carthage update --only GRDB,GRDBCipher
// OR
carthage update --only target_whitelist_cartfile

// Build all targets and only iOS variants
// Would build: `GRDBiOS`, `GRDBCipheriOS`
carthage update --platform iOS --only GRDB,GRDBCipher
// OR
carthage update --platform iOS --only target_whitelist_cartfile

My Vote is on Option 1. I would actually like to see the Cartfile syntax be versioned in someway so this can be supported and maybe a yaml based syntax could be introduced for future extensions, but we can save that for another discussion.

[JetForMe]

For me, the key is to be able to specify it all in the Cartfile, so that a simple carthage update is all that's needed to build the minimum necessary for that specific project.

[mdiep]

I think we would be okay with a whitelist approach _if_ we could decide on an addition to our file format that would support this in a consistent matter. If anyone is interested in furthering this issue, I would suggest starting by researching the current Cartfile format and making a proposal about how this information could be added. OGDL has been considered in the past, but I don't think anyone has a good sense of what that would look like.

[sirghi]

For instance:
<schemeName(optional)> : <dependency(what we have now)> : <dependencySchemes(optional)>

And as, an example, this would be an app Cartfile:

github "apollographql/apollo-ios": "Apollo, ApolloSocket"

And the _apollo-ios_ Cartfile would look like:

git "https://domain.org/framework.git" ~> 1.3 // common, would be used in any case
"Apollo" : github "gordon/chewbacca" == 1.2.1
"Apollo" : github "xverb/normalize" >= 3.5 : "Core"
"ApolloSQL" : github "stephencelis/SQLite.swift" ~> 0.11.5
"ApolloSocket": github "daltoniam/Starscream" ~> 3.0.5 : "Futures, Sockets"

In which case Carthage would pull in the first 3 and the last dependency from the _apollo-ios_ Cartfile, resolving the transitive schema dependencies further down the stream. There's a bit of duplication, but still. And since the schema prefixes and suffixes are optional, this would seemingly be backwards-compatible with the current version of the Cartfile.

If you like this approach, we can move this to it's own ticket, or maybe #1876 ?

[ifndefgt]

I know this is not what you are looking for. But if your case is just filtering for platform targets, a simple script like this would help.

Steps:

1. 
   
   • Usebootstrap/update with --no-build. This is required for resolving dependencies all together.
   
   
2. 
   
   • Define separate lists for targets, like iOS-watchOS, iOS, watchOS
   
   
3. 
   
   • Build each list separately, but build order is important for getting benefits of cache. Thus, build inclusive lists first. For this example build iOS-watchOS first, then build iOS and watchOS (Otherwise your cache will just contain entries for iOS/watchOS, so you have to build the same things again and again)
   
   

Sample:

#!/bin/bash

############ DEFINITIONS ############

iOS_watchOS_Targets=('Alamofire')
iOS_Targets=(
     'lottie-ios'
     'KeychainAccess'
     'MBProgressHUD'
     'CocoaAsyncSocket'
     'AlamofireImage'
   )

function carthage_build_ios_watchOS {
     for i in ${iOS_watchOS_Targets[@]}; do
          carthage build ${i} --platform iOS,watchOS --cache-builds
     done
}

function carthage_build_ios {
     for i in ${iOS_Targets[@]}; do
          carthage build ${i} --platform iOS --cache-builds
     done
}

############ RUN SCRIPT ############

carthage bootstrap --no-build
carthage_build_ios_watchOS
carthage_build_ios

[mdiep]

Please, let's discuss any format changes to the Cartfile in a new issue. If you have a proposal, open an issue or PR for it.

Any format change must be generally extensible. It cannot be limited to adding support for this one specific thing. If we make a change, we want to have the flexibility to support other things as well.

[yvbeek]

Not a fan of putting the scheme name in front of the dependency. General approach in software is to put optional parameters at the end. I would prefer the Cartfile syntax of @KyleLeneau over that of @sirghi

It also seems a bit inefficient to me that every user of a library is going to have to specify the relevant schemes in their Cartfile. I would suggest that the library maker should (also) be able to specify which schemes are relevant. That way we can tell Carthage to e.g. always ignore example projects.

Maybe we could add a simple feature that shared schemes that start with an asterisk (or other symbol) are ignored by Carthage. This could be added to Xcode.swift - schemesInProjects, by adding a guard on the scheme name in schemes.filter.

https://github.com/Carthage/Carthage/blob/0f561eeeab04fe9ef646ea0c4cc8a167967292b7/Source/CarthageKit/Xcode.swift#L180-L188

[tmspzz]

https://github.com/Carthage/Carthage/issues/2294 has other ideas too.

I'm just leaving this here for an example of a yaml cartfile.

[sirghi]

Any format change must be generally extensible. It cannot be limited to adding support for this one specific thing. If we make a change, we want to have the flexibility to support other things as well.

One can only propose a change to overcome the problem they're aware of. Then, if the owners are aware of other problems, they might compile the suggestions into a discussion that would lead to a solution. I took the liberty of initiating it in #2627, if you don't mind

@Zyphrax Having schemes only at the end wouldn't help in filtering transitive dependencies of the library you're pointing to. Hence the pointed-to library specifies prefixes to show which scheme depends on which dependency (omitting the prefix for common deps). That being said, the need for a library-vendor to specify relevant schemes to be built in general seems legit too.

[Jeehut]

Anybody coming across this issue nowadays, please checkout my comment here.

[hasayakey]

LOL...4 yerars later it's still can't support building specified scheme...