Arcade: Provide a better way to add external packages to .NET Core releases.

[StephenBonikowsky]

WCF wants to ship together with the rest of the .NET Core packages but is not part of the build process that produces and publishes packages that the rest of the .NET Core repos are part of.

This has been happening manually and we now want to automate the process for including the WCF packages in each release. Currently the only way we know of to do it is by adding a WCF dependency in core-sdk, this will create "no-op dependency flow PRs". See dotnet/core-sdk#5273

Can Arcade provide a better solution?

[markwilkie]

We're always up for making things better.

Do you have a recommend @StephenBonikowsky ? What about the current process is most obnoxious in your view?

[nguerrera]

I think @dagood had an idea here.

The main thing from the SDK point of view is the current solution means more dependency PRs to our repo taking resources (both in machines and humans having more builds and PRs to handle when they sometimes fail, etc.), yet the builds with these PRs will be identical to the builds without them.

[StephenBonikowsky]

From @dagood …

Multi-root gather-drop is the first thing that comes to mind?

[dagood]

And to clarify what I mean by that, it's that however gather-drop works to recursively grab one product build's artifacts, have a way to input another build to put in the same drop. It'd be the job of whoever/whatever runs gather-drop to determine what the right build of WCF is to include, but that's the case for the product build too, and that is the right place to handle it.

[markwilkie]

Ah - so is the conceptual idea something like adding a new "type" of dependency - namely a "drop" type? That way the product builds only when a "product" type dependency occurs, and "drop" type dependencies are resolved when gather drop is used (or something like that).

Am I thinking about this correctly? (bad implementation aside) cc/ @mmitche

[mmitche]

Ah - so is the conceptual idea something like adding a new "type" of dependency - namely a "drop" type? That way the product builds only when a "product" type dependency occurs, and "drop" type dependencies are resolved when gather drop is used (or something like that).

No...you can't have it be a dependency type. That would involve making a change in the repo, which is still a no-op.

This is just gather-drop with multiple build roots. Technically, we already have this I think (EF6)

[dagood]

+@wtgodbe

Technically, we already have this I think (EF6)

Oh yeah, I suppose it would be necessary for that.

I guess it's true that the docs don't say you can only pass --id once. 🙂 How --repo and --commit work with that (if they do) is less clear--ordering, etc.--but I'd wager the doc just in general doesn't cover this yet. Anyway, --id is the important one since it's unambiguous.

[markwilkie]

This sounds like a path forward was found?? If not, feel free to reopen @dagood

[dagood]

I would say it isn't complete because it isn't documented. (Please close if documented, a link would be nice. This issue may come up in search results for the next person trying to do this.)

[wtgodbe]

darc gather drop now supports passing multiple --ids, which is what we do in 3.1.x to include EF6. Documentation surrounding this whole process is still a bit sparse, though, and is more limited to knowledge that me, @mmitche and @Anipik have - I think there's another issue somewhere tracking documenting the process better (@mmitche do you have a link?)

[mmitche]

The docs got updated for this: https://github.com/dotnet/arcade/pull/5020