Rust: Tracking issue for RFC 2282 - Cargo profile dependencies

[Manishearth]

This is a tracking issue for RFC 2282 (rust-lang/rfcs#2282).

Cargo issue: https://github.com/rust-lang/cargo/issues/5298

Steps:

(cc @rust-lang/cargo for mentorship instructions)

• [x] Implement the ability to override profiles via .cargo/config (https://github.com/rust-lang/cargo/pull/5384)
• [x] Implement the ability to override profile keys for specific dependencies (https://github.com/rust-lang/cargo/pull/5506)
• [x] Adjust documentation (see instructions on forge)
• [ ] Stabilization PR (see instructions on forge)

Unresolved questions:

• [ ] We can bikeshed the names of the new keys that were introduces

[Boscop]

Any update on this? :)

[Manishearth]

Someone needs to implement it.

[ehuss]

If you're looking for someone to work on this I'd be happy to help!

[Manishearth]

Sure! I don't have the bandwidth to mentor this but feel free to try it out. Might want to file an issue on the cargo side to coordinate the implementation work.

[ehuss]

Some issues came up during implementation.

• @alexcrichton mentioned that the package name could support full-spec syntax (crates.io/foo, etc.). It looks like it should be easy to support, just let me know if you'd like me to add it! (Currently it just matches the name.)
• I briefly discussed with @matklad the possibility of having a dedicated profile used for scripts. It would be easy to add, and would remove "build-override" which is essentially a pseudo-profile (and seems a little confusing to me). It would also decouple building build scripts from the --release flag. Let me know if you want to consider this route!

[Manishearth]

Full spec sounds great.

The problem with having a dedicated profile for scripts is that you may wish them to be compiled differently in debug vs release. build-override lets you do both.

The original proposal (not the one we RFCd) allowed nesting of profiles which would have made this work well (you declare a new profile, and link it in as the build profile for an existing one)

[Manishearth]

(Also to me it seems like having a build profile just makes the whole profile vs workflow deal even messier -- we already are in a situation where the test/bench profiles are a bit weird)

[ehuss]

Yep! I was just meaning that it can probably be improved on without too much effort. I think there was some concern during the RFC process that shared dependencies would be a problem, but they shouldn't be. I think @matklad had more thoughts on the profile/workflow stuff, so I'll let him say more.

[matklad]

may wish them [build scripts] to be compiled differently in debug vs release.

@Manishearth could you give an example for this? An important practical reason for compiling build scripts exactly the same in dev and release is to avoid compiling syn and friends twice, when you do , say, cargo test followed by cargo bench. You can achieve this with current design of build_overrides, but you'll have to explicitly override most of options for build profiles, because, by default, dev and release are quite different.

More broadly, I think the following properties make a dedicated build profile an interesting option to consider:

• It makes sense to use a single profile for build scripts, so as to compile them once for all cargo invocations.
• Semantically, it makes little sense to use the same profile for build script and library code. This is easily understood in the context of cross compilation: you compile library for the target arch, and the build script for host arch, and they are executed on completely different machines. The motivations for picking particular compiler flags must be different in these two separate worlds.
• Similarly to the previous bullet, the build scripts profile settings do not affect the generated artifacts in any way.

• The best defaults for the build scripts are likely to be different. It seems to me that we want something like the following:

  [profile.build]
  # low hanging optimization fruits which shouldn't affect build time much
  opt-level = 1 
  # save build time and disk space by not emitting huge amount of debug-info
  debug = false 
  # speed up both compile time and runtime
  debug-assertions = false
  overflow-checks = false
  # build scripts are small and rarely modified
  incremental = false
  

  this differs significantly from both dev and release profiles.

The interaction with workflows is also interesting here: in a nutshell, a workflow would allow one to compile and run some custom rust code on the host to do execute custom tasks, like cargo subcommands, but locally to the project. And this code needs to be compiled with some optimization settings and such :)

I envision the build profile as the profile for all of build scripts, proc macros and workflow tasks. All three are compiled on the host, could be shared across cargo build and cargo build --release and have similar runtime/compile-time trade offs.

It is true that test, bench and especially doc profile (which exists and does nothing) are messy today. But it seems that, unlike those profiles, the suggested build one actually has compelling reasons to exist, as discussed above. Also, if we compare build_overrides with the build profile, I would argue that the latter has less overall complexity, because it reuses the existing concept, while the former is one is a special case with a special syntax and such.

[Manishearth]

could you give an example for this?

My dev workflow may be one where I'm recompiling often (but I don't edit the build scripts) so I need the build scripts to be fast, but my release workflow may be one where I only run it occasionally (incremental builds are less common), so build scripts should be compiled in dev.

This isn't everyone, but this could be someone.

An important practical reason for compiling build scripts exactly the same in dev and release is to avoid compiling

This is an argument for why the defaults should be such that build profiles are the same , not an argument for why we shouldn't expose this functionality.

Firefox, for example, would probably want an optimized build script with an unoptimized libsyntax/syn/whatever.

you compile library for the target arch, and the build script for host arch, and they are executed on completely different machines. The motivations for picking particular compiler flags must be different in these two separate worlds.

IMO it's not relevant to bring cross compilation into the picture here because cross compilation works differently -- dependencies are _not_ shared. If cargo was rearchitected so that shared deps between build scripts and the main build may be compiled into separate artifacts, this makes more sense, but as it stands, these are shared, and IMO it's easier to reason about these shared dependencies when you have a single profile instead of two.

[ehuss]

these are shared

Not anymore. Shared deps are now compiled separately. (The graph for features/versions is still unified, though.)

[ehuss]

I had some questions on how to feature-gate profiles in config files. I can imagine a variety of ways to implement it:

• Command-line option (-Z profile-config)?
• Feature flag in Cargo.toml (cargo-features = ["profile-config"])?
• Some kind of flag in the config file itself?
• Should it ignore entries in config if you don't specify the flag? Or warn? Or error?

I can imagine if you put a profile in a global config it could be really fussy. Let me know if you have any ideas.

[Manishearth]

I'd have some kind of flag in the config itself.

[Manishearth]

But that's not a strong opinion

[ehuss]

I have some more questions (cc @alexcrichton @matklad).

• I had some questions/doubts about warnings for config files. In general, it looks like config files never warn about misspelled/unused keys. Is that intentional for some sort of forward-compatibility?
  
  
  
  • My implementation currently does not warn about unused override package names in config files ([profile.dev.overrides.foo]). I was thinking someone might place an entry in a global config, which would cause warnings in all projects not using that package. Sound reasonable?
  
  
  • Let me know if I should add warnings or errors for the following:
    
    
    
    
    
    • Unknown profile names ([profile.misspelled]).
    
    
    
    • Unsupported profile names (test, bench, doc).
    
    
    
    • Unknown keys in a profile.
    
    
    
  
  
• Config files do not support merging values with mixed data types. For example, if you have lto = true in one config and lto = 'thin' in another, it fails. How much of a problem is this? (This affects lto, opt-level, and debug.)
• If we go the route of adding unstable feature flags to config files, there are some design questions I have. To start with:
  
  
  
  • Should it be an error to specify a feature flag when using a stable release? Or should it warn and ignore the new settings? Or maybe just silently ignore? My concern is that it would be very cumbersome to use unstable features in a global config file (since running on stable would always fail or print annoying warnings).
  
  
  • Should unknown feature names be ignored, warn, or error?
  
  
• (EDIT) Another idea for handling warnings would be to silently ignore unless you run with verbose.

[alexcrichton]

Aha some excellent questions @ehuss! Lemme see if I can dig in...

• It's true yeah that we're quite lenient with config files. I don't think we can warn about unused (aka misspelled) keys here though in all situations. Not all builds may use all configuration, so we'd probably be going a bit too far out of our way to get this working. Now that being said I think profiles specifically could probably helpfully reject any configuration other than release/dev or any other obvious missteps, I think it's only possible to reject configuration that couldn't possibly be used though rather than "this isn't used for this particular build"

• Hm... We may just want to tweak how configs are merged and say that a string can override a string or something like that. I don't think there's necessarily a downside to that approach I think, but do you forsee any problems with that?

• I think it's probably safe to just require features to be enabled in the local cargo build (via the CLI maybe?) rather than the configuration. If the local features aren't enabled but configuration is detected then we could print a warning or something like that saying that the features aren't adequately enabled.

[ehuss]

profiles specifically could probably helpfully reject any configuration other than release/dev or any other obvious missteps,

My concern with rejecting with an error is that if in the future cargo supports additional profile names, it would make it impossible to use the new names or keys in a global config and continue using older versions of cargo. I would feel more comfortable with just warnings (profile `foo` is not supported or profile key `bar` is not supported).

string can override a string

Do you mean string can override an int? If so, that sounds fine to me.

via the CLI maybe?

That seems fine with me, too. Would this be a command-line flag that is only necessary during the unstable period?

My concern with unconditionally warning if you have a profile in a global config and you don't specify the command-line option is that you will end up with seeing that warning everywhere you don't specify the flag, which would be rather noisy. I kinda like the idea of only warning with -v since that would avoid the noise. That may not be very discoverable, though. ☹️ How common do we think profiles in global configs will be? If it's extremely rare, maybe the noisiness doesn't matter.

I had another question: What should the precedence be? This comment suggests it should be manifest over config. However, incremental uses config over manifest. The RFC doesn't specify. I was actually thinking it would be config over manifest to give the user control over what happens. With manifest over config their only alternative would be to edit the manifest. If config takes precedence, I wouldn't worry too much about a user globally specifying an important flag like panic (that just seems weird to me).

[alexcrichton]

I would feel more comfortable with just warnings

An excellent point and sounds good to me!

Do you mean string can override an int? If so, that sounds fine to me.

Oops yes, indeed!

Would this be a command-line flag that is only necessary during the unstable period?

Correct!

I'd also be ok not warning for now. It may lead to some confusion but you're right in that it's likely less than otherwise

What should the precedence be?

I definitely agree it's config over manifest because you modify your local config as opposed to the manifest which is shared amongst all users.

[ehuss]

Part 2 that includes config profiles has landed on nightly. The (minimal) documentation is at: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#config-profiles

[Centril]

@Manishearth uhm -- why is this T-lang? was that a mistake?

[Manishearth]

uh.....

yes :smile:

oops

[jrmuizel]

We're suffering bad codegen issues in webrender with bincode that this would help to solve. Any chance someone can estimate a timeline for stabilization? Or should we try to look for alternative ways to solve this?

[ehuss]

@jrmuizel Unfortunately there currently is no timeline. I have a concern about the design, and I'll follow up here to try to spur discussion.

[ehuss]

The current implementation has some drawbacks regarding build script profiles and shared dependencies. Some of this is likely due to misunderstanding/miscommunication on my part. It centers around this statement in the RFC:

It is not possible to have the same crate compiled in different modes as a
build dependency and a regular dependency within the same profile when not
cross compiling. (This is a current limitation in Cargo, but it would be
nice if we could fix this)

This limitations was lifted, but has some possibly unintended consequences, and highlights some drawbacks. There might be some weird interactions with precedence, too. The current implementation merges overrides in the following order:

1. Default hard-coded profile
2. Base profile (like [profile.dev])
3. build-override
4. "*" override
5. Package-specific override

An example of where this might be an issue: Build-override settings can cause shared dependencies to be built multiple times. This may be desirable in some cases, and in some it causes unwanted longer initial build times. Example:

[profile.dev.build-override]
debug = false

This will cause a shared dependency to be built once with debug=true (as a normal dependency) and once with debug=false (as a build dependency). You can override the shared dependency to make it compile once, but that is awkward.

How much of a problem is that?

There are some other hypothetical concerns about the lack of control:

• If you have a package-specific override, there's no way to say if it is for a build-dependency or a normal dependency (it overrides in both cases). Probably ok?
• Configuring proc-macros can be tedious. You need to target each individual one, and any non-shared dependency. You can't separately control shared dependencies of proc-macros with normal dependencies. This may be particularly weird when cross-compiling.

Implementation-wise I think these are easy to change, but UI/UX-wise it's not clear what's the best design. I would appreciate any feedback from anyone following here. Is there a better way to control normal dependencies shared with build scripts/proc-macro? Should it be easier to configure proc-macros? Are there any other concerns before stabilizing?

[alexcrichton]

I think I'm not personally too terribly concerned about multiple builds. Over time I think minimizing the number of builds for non-cross builds has ended up being a mistake as it just ends up causing a lot of problems here and there that amount to a large papercut. In practice these duplicated crates are always very small and at the bottom of the dependency graph where there's a lot of parallelism, so even practically I don't think it has a terrible amount of impact.

I think having package-specific overrides apply for both build/normal use cases is fine for now, we can always add more tweaks later! It's a pretty straightforward way to interpret it in the near term..

For procedural macros, I think I'd probably expect uniform handling of both procedural macros and build script dependencies (e.g. both should probably have debuginfo turned off by default). It's possible to pretty easily fold in proc-macro deps into the category of build-override I think?

[matklad]

Over time I think minimizing the number of builds for non-cross builds has ended up being a mistake as it just ends up causing a lot of problems here and there that amount to a large papercut.

If we go a route of not deduping between dev/non-dev dependencies, then I feel a dedicated build profile (which, unlike build-override, is the same for --relase and --debug) is a better solution. More about build profile in this comment. I am out of the loop wrt current state of the art though, so might miss something obvious :)

[alexcrichton]

Yeah that seems reasonable to me, and such a profile we could define as "used for artifacts that are only transiently used during the build process itself", ranging from build scripts to procedural macros.

[ehuss]

fold in proc-macro deps into the category of build-override

I like the idea of including proc-macro/plugins in build-override. @Manishearth do you have any thoughts for or against doing that?

a dedicated build profile is a better solution.

I was originally in this camp, but I'm not convinced that it is necessarily better.

Pros for build-override:

• Allows separate control of dev vs release settings.
• Allows opportunistic reuse of shared dependencies (when using default settings).

Pros for dedicated build profile:

• Avoid recompile of build scripts when switching between dev and release.
• Only one place to specify settings (simpler to understand?).

The benefits to a dedicated profile do not seem very strong to me. @Manishearth gave explicit use cases where the current design is more powerful. Does anyone have any thoughts about that?

I was curious how common shared build dependencies are. I scanned all the crates on crates.io (all features enabled). Of the 8288 crates that have a build dependency anywhere in their graph, 2426 have at least one shared build dependency. The top 20 most shared crates are:

1949  libc
1872  winapi
1462  winapi-x86_64-pc-windows-gnu
1462  winapi-i686-pc-windows-gnu
1461  bitflags
1151  rand_core
955   fuchsia-zircon-sys
955   fuchsia-zircon
846   rand
846   cfg-if
738   phf_shared
737   siphasher
591   unicase
581   cloudabi
528   log
398   lazy_static
358   unicode-xid
264   quote
259   serde
239   syn

The rustc repo has an unusually large number of shared build dependencies (30).

[Mark-Simulacrum]

Given the inclusion of proc-macros and plugins, the build-override having a separate profile could make it feasible to have build scripts, proc macros, etc. compiled in release mode by-default (even on cargo build giving us overall faster compilation speeds. I don't think we have concrete numbers, but it does seem feasible that compiling essentially parts of the compiler (proc macros) in release mode would be advantageous.

[matklad]

@Manishearth gave explicit use cases where the current design is more powerful. Does anyone have any thoughts about that?

Agree that having dev/release gives more power, but I'd expect that, in common case, one does need such power. If one indeed runs release/dev workflows on different machines (so that sharing compiled build scripts is not benefitial), then, on release machine, one can override build profile to match release via env-vars/cargo config. If you do dev and release on the same machine (overwhelmingly common case I suppose), then don't recompiling build scripts/proc macros seems to be a plain win?

Avoid recompile of build scripts when switching between dev and release.

A slightly different spin here is avoiding recompiling builds/proc macros when tweaking profiles. For example, my software is slow, so I want to build a flame graph, so I add debug = true to my release profile, and here I'd like to avoid recompiling syn.

Only one place to specify settings (simpler to understand?).

To me, a separate profile for "everything compiled for host" seems significantly simpler then a layer of overrides. So, I'd personally give this benefit a relatively heavy weight.

TL;DR: I think we need to weigh "power" against "common case": overrides win on power, dedicated profile wins on "common case"

[alexcrichton]

I think I'd personally be more of a fan of a dedicated profile, and we could figure out later how to have cascading overrides perhaps or something like that. Having a dedicated profile seems best for configuration and simplicity from the get-go

[Manishearth]

Here's an example of Firefox's use case that may make some needs clear:

Firefox has a build time dependency on bindgen. Bindgen reads a crapload of c++ headers, does a bunch of analysis, and spits out rust code.

In debug mode, bindgen should be compiled in release mode because otherwise it's _super_ slow.

However, the rust generation step is not slow. Compiling bindgen in release mode means compiling syn in release mode, which takes forever and is not really necessary.

I think a separate profile works with this? You can still support per-dep overrides with this.

[alercah]

Why not both?

[alercah]

Oops, I'd like to apologize for the double-post. But I accidentally submitted before I wrote the rest of my post...

Maybe just add an option controlling profile deduplication? For all non-main profiles (that is, profiles other than dev or release), you could have a [profile.*.dedup] field that controls the selection of deduplication behaviour between preferring the main profile, preferring the non-main profile, or double compilation. For test and bench profiles, it could match whatever the current behaviour is (which isn't documented, but I assume is to match the main profile?). For build, it could default to whatever was agreed upon as the best behaviour.

[ehuss]

Sorry for the delay in response. I was going to suggest both, too. I was just hoping a better idea would pop in my mind. It doesn't seem too bad. build-overrides would just be a small, niche feature for more advanced usage. It solves the common case of wanting a straightforward way to control build dependencies ([profile.build]), have different dev/release behavior ([profile.release.build-override]), and allows you to deduplicate if you want, and is fairly simple (at least implementation-wise).

The new profiles have largely not supported test/bench, since it's not clear if those will even be kept long-term. So I'm not sure I completely follow the dedup field. If Cargo supported both methods, and deduplication is important to a project, then one could specify [profile.dev.build-override] and [profile.release.build-override] to match the corresponding dev/release settings.

[alercah]

Using a build override forces you to change the settings for all crates being built, not just duplicated ones. You can name some specifically for overrides, but that won't cover indirect dependencies. So I was thinking that a general deduplication field could tell Cargo to prefer one profile or another automatically. For instance, if I want to minimize compile times and filesystem size (say, I'm on a relatively resource-constrained system) while still getting full debug info for my builds, I want debug info generated for all runtime dependencies, but not on build-only dependencies. build-override doesn't provide a mechanism to do that, but a deduplication field does.

That said, this is a relatively obscure case and there is no reason it can't be added later.

[Manishearth]

Profile overrides were implemented in https://github.com/rust-lang/cargo/pull/5384

Config profiles were implemented in https://github.com/rust-lang/cargo/pull/5506

We should figure out what the path to stabilization is.

[ehuss]

We should figure out what the path to stabilization is.

Are there any issues or details you think need to be addressed?

I feel like it is ready for stabilization. The only concern is that there are two changes being looked at that may affect how profiles work:

• Dedicated build profile: https://github.com/rust-lang/cargo/pull/6577
• Custom named profiles: https://internals.rust-lang.org/t/pre-rfc-cargo-custom-named-profiles/9538

These will probably not have much impact on profile overrides. The only issue is that the dedicated Build Profile changes build-override to also encompass proc-macros. I see a few options for that:

• Isolate that change now, and make build-override handle proc-macros.
• Stabilize profile overrides, and maybe change the behavior later.
• Delay stabilization until build profiles are done. This may take a while, as there are a number of blocking issues (https://github.com/rust-lang/cargo/pull/6668).

I don't have a sense what people think about adding proc-macros to build-override. It makes sense to me (because they are build-time, host-only dependencies), but I don't know if it would be confusing or problematic. Does anyone have a preferred path here?

[Manishearth]

I don't have a strong opinion. Weak preference towards shipping, however with custom profiles this feature can be designed better, perhaps (the original RFC used custom profiles to make this work, and it was pretty neat). Such designs can still be tacked on after the fact giving you multiple equivalent ways to do things.

As for proc macros I agree, we can add them to the build profile.

[Jasper-Bekkers]

Are there any issues or details you think need to be addressed?

This might be nit-picky or just a bug, but from a usability perspective right now if I specify for example:

[profile.dev.overrides.gltf]
opt-level = 3

I would expect gltf and all of it's dependencies to be compiled with -O3, however as it stands right now only the gltf crate would be compiled with -O3 and it's dependencies would be compiled as normal. To me this seems like a bit of a usability problem because either I have to specify '*' and opt out for all other dependencies that my crate has, or hunt down which specific part of gltf is slow in debug and specify exactly those crates in [profile.dev.overrides...].

[Manishearth]

No, this is by design, you are overriding a specific crate here. This kind of thing mostly should be done only after investigating the perf of your deptree, so it's largely fine. The use case in Firefox at least needs the ability to override single deps (not entire deptrees)

At some point the RFC had a proposal to allow .gltf.* or something but I think it got removed.

[Jasper-Bekkers]

No, this is by design, you are overriding a specific crate here. This kind of thing mostly should be done only after investigating the perf of your deptree, so it's largely fine. The use case in Firefox at least needs the ability to override single deps (not entire deptrees)

I think both use cases are valuable & valid. For example, in my case the bottleneck ended up (mostly) in the texture loading, which turned out to be in the image crate, which in turn turned out to be in the png crate, which in turn spent a lot of time in inflate. Setting inflate to -O3 solve most of the issue, but not all of it - since other parts were still slow.

However, in my case I didn't particularly care about the debuggability of gltf as a whole and would've preferred to just mark that as -O3 in it's entirety.

At some point the RFC had a proposal to allow .gltf.* or something but I think it got removed.

This syntax would be very much OK for me, I tried it naturally over the course of experimentation with this feature and that's what I expected it to do.

[Manishearth]

Yeah, so having overrides.foo override an entire deptree means that there isn't much syntax that works the other way, whereas having overrides.foo override just one crate and using .* to do the deptree thing is nicer.

Perhaps we can RFC the deptree thing once this lands and more people ask for it. (or just RFC it anyway, but I'm wary of RFCing an extension to a feature while the feature is still unstable)

[Manishearth]

Then again, that RFC may end up with a new design for this that can't be done backwards compatibly, so perhaps it's better to RFC .* now so people can make use of the wider design space.

(Feel free to ping me with RFC drafts for review if you're interested)

[Jasper-Bekkers]

I'll write something up, what's a good way to reach you?

[Manishearth]

Ping me on the Rust discord

[brson]

In my testing of this feature I've found that [profile.test] and [profile.bench] cannot be overridden. Cargo reports "warning: unused key profile.bench in config file".

I don't see any language in the RFC saying that these should not be overrideable. I am attempting to use config profiles to create a "menu" of profiles, for testing the effect of various configurations on compile and build time.

I can get by without them, but it's not obvious why they can't be overridden.

[brson]

OK, after further testing I find something weirder.

A [profile.dev] override applies to cargo test, and a [profile.release] override applies to cargo bench.

This seems very counterintuitive, and would need to be explained clearly in the docs.

Tested with cargo 1.34.0-nightly (5c6aa46e6 2019-02-22)

[ehuss]

[profile.test] and [profile.bench] cannot be overridden

It wasn't explicitly stated in the RFC, but only dev/release are supported (this is briefly mentioned in the unstable documentation). This was a conservative approach because the future of the test/bench profiles is uncertain. They behave in a strange way. Essentially test/bench only apply to the final artifact, and none of the dependencies (which use dev/release). And the local library is considered a dependency for integration tests. This was likely due to how they were historically implemented, but it is no longer necessary. The current behavior has one benefit where if the dev profile is modified, and you run cargo test, it does not rebuild all dependencies if you previously ran cargo build. (This is confusingly explained here).

@matklad is a proponent of just removing test/bench altogether. I think there are reasonable use cases for them, so I would prefer to keep them, but fix them. I think they should probably remove the "dependencies use another profile", and have the default test/bench profiles inherit the dev/release settings by default.

[matklad]

To elaborate, I am fine with test/bench, if they are fixed as @ehuss proposed, provided that we do have a genuine "profile inheritance" mechanism useful elsewhere.

[kornelski]

Custom profiles #2678 introduce a concept of profile hierarchy (inheritance), so in a sense, it makes all profiles "overrides" of other profiles.

This makes me think that the "override" term here is or will become irrelevant/redundant, and could be removed. They're all just profiles that act as refinements of other profiles.

So I suggest dropping the override word from the terminology:

[profile.dev]
opt-level = 0
debug = true

[profile.dev.dependencies.image]
opt-level = 3

[profile.dev.dependencies."*"]
opt-level = 2

[profile.dev.build-dependencies]
opt-level = 3

[alexcrichton]

I personally agree that "override" may not be the best term here, but it's worth noting that you can name the current crate like:

[profile.dev.override.my-crate]
opt-level = 3

so dependencies may not fit the bill as well?

Perhaps something like crate or pkg?

[profile.dev.crate.image]
opt-level = 3
[profile.dev.crate."*"]
opt-level = 2

# or...

[profile.dev.pkg.image]
opt-level = 3
[profile.dev.pkg."*"]
opt-level = 2

[kornelski]

Yeah. .package.name makes sense, since there's [package] name = "".

[ehuss]

An issue with config-profile has been raised at https://github.com/rust-lang/cargo/issues/7253. There are ambiguities with environment variables that are difficult to resolve.

[alexcrichton]

Note that I wouldn't necessarily consider that a showstopper. I do think though that if we want to put this feature through its paces we would ideally all but delete src/bootstrap/bin/rustc.rs from this repository which largely implements what RUSTFLAGS and this feature should encompass instead.

[ehuss]

cc #63484, a surprising result of using profile overrides.

[est31]

if we want to put this feature through its paces we would ideally all but delete src/bootstrap/bin/rustc.rs

For reference, this is happening in https://github.com/rust-lang/rust/pull/64316 .

also cc https://github.com/rust-lang/rust/pull/64324

[est31]

Update: #64316 has been merged.

I wonder whether this feature can be stabilized. Is it ready for an FCP yet?

[ehuss]

To be clear, there are two or three features here which may be possible to stabilize independently.

Some blockers:

profile-overrides:

• I'd like to wait for named profiles to land, to make sure there aren't any unforeseen interactions. It looks like it should be OK, but that PR needs to land first at a minimum. It looks like it is getting close to finished. That PR also changes some behavior here.
• I plan to rename override to package once 6989 is done.
• I'm not sure whether or not this should wait for build-profile. Probably not, but should think some more about it.

config-profile:

• I'm concerned about the environment variable name issue, and I'd like to figure that out first.

build-profile

• The PR is blocked needing more design work on directory restructuring. Some of that is done here, but there needs to be mitigations and migrations for existing tools' assumptions about the layout.

[est31]

@ehuss now that https://github.com/rust-lang/cargo/pull/6989 and https://github.com/rust-lang/cargo/pull/7504 have been merged, can we maybe think about stabilizing the core profile-overrides feature? I'm mainly interested in the "only optimize dependencies" use case and it seems that feature alone would unlock it.

[elichai]

Almost opened a new issue and then saw this.
I want to change globally the default that all release builds will also be built with -Clto.
this isn't easy to do in a shell function without having a complicated one (i.e. requires checking on cargo install if --debug is set and on cargo build if --release is set)

[ehuss]

The cargo team discussed stabilizing the profile-overrides side, and are generally positive. However, I remembered an issue that I forgot to bring up, and wanted to see what people think.

When defining an override for a package, it can be awkward when the thing you want to override is split across multiple crates. For example, if I wanted to change a couple settings for rand, it may need to list out profiles for 3 or 4 different crates. Another example is specifying the override for the standard library. The user may not even know which crates comprise the standard library, so they won't even know which ones to override.

What do people think about that? Should there be some kind of capability to apply an override to dependencies of a crate? Or maybe a way to specify multiple crates for a single override? Or is this a niche concern, and almost everyone will just use "*"?

I think it might be good to avoid situations like this:

[profile.dev.package.rand]
opt-level = 2
debug-assertions = false
overflow-checks = false

[profile.dev.package.rand_core]
opt-level = 2
debug-assertions = false
overflow-checks = false

[profile.dev.package.rand_chacha]
opt-level = 2
debug-assertions = false
overflow-checks = false

[kornelski]

Could a crate like rand say that its dependencies should inherit its own settings?

Otherwise Cargo and crates-io may need to develop some concept of a published group of crates, and that's probably whole another can of worms.

[est31]

Thanks for pushing this forward, @ehuss !

@kornelski right now, non-root crates can't even override their own settings. The current profile overrides feature only applies to the top level crate's Cargo.toml, to be specific to the root Cargo.toml of the top level crate's workspace. Cargo displays a warning if it finds non-root profile overrides in the same workspace, but it silently ignores profile overrides of dependencies.

I think it's a good idea to provide such a feature (I personally would enable it in lewton to always optimize it, because lewton is very slow in debug mode) and @matklad has also expressed interest in it. Should the current behavior be changed? What if a crate only wants to override something for local development?

@ehuss

What do people think about that? Should there be some kind of capability to apply an override to dependencies of a crate? Or maybe a way to specify multiple crates for a single override? Or is this a niche concern, and almost everyone will just use "*"?

You have a point that you quickly get into the situation of repetitive sections. In the only place where I use profile-overrides, I have copy-pasted the same section for * as for one crate in my workspace.

The same time though, the more features one adds, the harder a Cargo.toml becomes to parse for humans. A Cargo.toml with tons of repetition isn't easy to parse either though. IDK. These features seem they can be added in a backwards compatible fashion as well.

[kornelski]

@est31 I'm not sure if dependencies should be able to set any specific settings for themselves. I was thinking about a flag like:

[package]
name = "rand"
if-my-settings-are-overriden-then-also-override-my-deps = true

or maybe:

[package]
name = "rand"

[dependencies.rand_chacha]
version = "0"
profile = "inherit" # inherit being the only value accepted on crates-io

but this raises a question — what happens if two different crates do this to the same dependency. Whose settings "win"?

[est31]

I'm not sure if dependencies should be able to set any specific settings for themselves.

Only talking about the defaults here. Of course, the leaf crate should always have final say here. @matklad spoke in even weaker terms of suggestions.

I was thinking about a flag like

Oh I see what you want. It isn't even strictly a feature to group crates together like you would group all of the rand crates together. You could also say that e.g. an inflate implementation maintained by a different organization should get the same options as you because your speed depends on inflate.

raises a question — what happens if two different crates do this to the same dependency. Whose settings "win"?

I'd propose that if crate A (transitively) depends on crate B and both depend on C, then A's settings override those of B. If there is no hierarchy between A and B, cargo could create an error message or at least a warning.

---

Anyways, I believe that these discussions shouldn't be blocking stabilization.

[ehuss]

A proposal to stabilize the profile-override part has been posted at https://github.com/rust-lang/cargo/pull/7591.

[ehuss]

The proposal to stabilize the config-profile part has been posted at https://github.com/rust-lang/cargo/pull/7823.

[jonas-schievink]

Is there anything left to do here or can this be closed?

[lnicola]

I want to mention an issue I've just ran into: I tried to use build-overrides to speed up release builds, but some crates ended up being built twice because they were both build- and run-time dependencies. That's not ideal, but it's understandable, I suppose.

[ehuss]

Yes, this can be closed now.

@lnicola That is the intended behavior.

[elichai]

Hi,
Now that we can override profiles in the config file, anyway we can get profile.*.rustflags too?
I'd like to pass specific compiler flags only when compiled with release profile (like target-cpu=native)