Conda-forge.github.io: Rust/Go packages license issues

[isuruf]

A typical rust package use dozens of packages which have different licenses and requirements. A rust package and its dependencies are usually compiled into one library or executable.
For eg: https://github.com/conda-forge/staged-recipes/pull/11315 has a rust package with 91 dependencies with various MIT/BSD-3-Clause/Apache-2.0 licenses and maybe others.

This implies that the licenses and copyrights of the dependencies need to be distributed with the package. There are some tools to help do this like https://github.com/maghoff/cargo-license-hound, https://github.com/onur/cargo-license.

I'm opening this issue so that @conda-forge/staged-recipes and @conda-forge/core know about this when reviewing Rust recipes.

cc @andfoy, @mingwandroid

[andfoy]

What I'm doing in particular is using the JSON output information produced by cargo-license and then grab the repository urls across GitHub, BitBucket and GitLab to call their respective APIs to locate and download all the licenses. However, some libraries need a manual license download still.

[nehaljwani]

Doesn't the same concern apply to go packages?

[dbast]

To not re-invent the wheel here, how are other packaging eco systems solving that e.g. linux distributions like debian or homebrew?

[isuruf]

Yes, the same concern apply to Go packages. See also https://github.com/google/go-licenses

I've no idea how others fix this.

[hadim]

I am not sure how you want to address that but it does not seem straightforward. We could use a script that goes over all the dependencies, parse for the licenses, and list all the licenses per deps in the conda package?

[hadim]

Also at what level this script should be run? conda or conda-forge?

[isuruf]

@hadim, what @andfoy did for rust was to use a script to download licenses and put them in the recipe (and manually add licenses for packages that the script failed). He also added a check in build.sh to check that each dependency had a license file in the recipe. Same can be done for Go.

[hadim]

It makes sense.

That being said I probably don't have the bandwidth at the moment to do that for https://github.com/conda-forge/staged-recipes/pull/11799

[isuruf]

For go, it's simple. See https://github.com/google/go-licenses#complying-with-license-terms

[SylvainCorlay]

Quick thought, this also applies to C++ packages when you link statically with your dependencies.

[nehaljwani]

Should this be extended to header only dependencies as well? For example, if you use pybind11, boost, etc, do you need to package the license file used by them as well? Because that's as good as statically linking parts of them.

[chrisburr]

Perhaps there needs to be a licence_exports field in the conda build metadata.

[isuruf]

Should this be extended to header only dependencies as well?

Depends on the license.

For example, if you use pybind11, boost, etc, do you need to package the license file used by them as well?

pybind11: yes. boost: no.

[bollwyvl]

Thanks for the guidance here on this topic: texlab-feedstock is now using the same approach as pysyntect-feedstock, and "only" required manually hunting down 20 licenses (of 200+). Perhaps we should package cargo-license... seems to cost a couple minutes per build.

[bollwyvl]

As this has come up again for @conda-forge/cryptography:

I wonder if we start curating a community package, e.g. conda-forge-rust-licenses and conda-forge-go-licenses (or just lump them together under conda-forge-license-library) which has some automation to at least allow centralizing the list of known/used <thing>/<version>/(UN)LICEN(S|CE(-.*)(.(txt|md))? (oh and don't forget COPYRIGHT.*). Then packages can demand said package during builds, copying the assets from a well-known location to wherever there license_file points... now that we can use folders, that's _much_ easier. If a new crate/mod shows up, the build would fail, but might suggest...

Some wild crates and mods approach!

- <crate>@<version> <url>
- <mod>@<version> <url>

From inspection, I've found the below licenses. Please visit the upstream repos and verify, then 
make a pull request to https://github.com/conda-forge/conda-forge-license-library adding the lines:

### recipe/licenses/cargo.txt

<repo>@<tag>/LICENSE-MIT
<repo>@<tag>/LICENSE-APACHE

### recipe/licenses/go-mod.txt

<repo>@<tag>/LICENSE-ZLIB-WITH-FREAKY-SPEC

this would in turn update the recipe (once) so we actually have the licenses sha256sums.

[bollwyvl]

So would a conda-incubator/* be the right path? I'm imagining a small (potentially single file) python package with a simple in-build CLI like cargo-licenses | dmv -o $SRC_DIR/third-party-licenses. The JSON/CSV file with, at the very least, the couple hundred licenses URLs/SHAs, would then live in the feedstock... but could contain the actual licenses texts themselves.