Staged-recipes: Add google/or-tools

[pmlandwehr]

Google's OR-Tools are maintained by Google but I'm not sure how easily they play with conda, so we might want to add `em.

[sodre]

@pmlandwehr you can assign it to me.

[jGaboardi]

@sodre Has there been any more progress on this? I was looking this issue earlier when a friend pointed me to this thread.

[sodre]

@jGaboardi, I haven't made any more progress on this since 2017. I was trying to cover all the architectures at that time and failed to complete the macOS and Windows builds.

If you are only interested in Linux, I think it is possible for us to rebase the current branch and only enable the build for Linux. Let me know if you're interested on that.

[jGaboardi]

@sodre thanks for the super quick reply! And yes, I am very interested. The current OS I need a conda build of ortools for is RHEL 6.10, so this would fit my needs perfectly. Much good karma to you if we can get it up on the conda-forge channel.

EDIT
I have just been informed by the main person in charge of approving software install requests that only conda-forge packages from the following two channels mirrored by the US Census Bureau (which is the environment I am working in for this specific project):

• http://mirror1.csvd.census.gov/python/conda-forge/linux-64

• http://mirror1.csvd.census.gov/python/conda-forge/noarch

[sodre]

Okay, I'll continue the work this weekend.

[jGaboardi]

@sodre Also, I am working inpython 2.7 due to a legacy code base.

Although I have never built conda package for or done a conda-forge release, I am happy to help you with this in any way that I am able.

[jGaboardi]

@sodre I wondering if anything more has been done in regards to getting the Linux build up on conda-forge? Please don't think I am trying to be pushy, I am in a time-sensitive situation for a particular project that I need to either be able to ask the person in charge of software to do conda install -c conda-forge ortools (easy option) or go to a committee for approval of pip install ortools (difficult option) . If this is still a work-in-progress, is there any timeline when it would be available? Either way, I want to reiterate that I am not trying to be a demanding jerk and really appreciate the work you are doing. Regardless of whether the conda-forge build will be completed for me to use for this particular project, I do believe the community would greatly benefit from ortools being available through conda.

[lperron]

Just to comment, as the creator of or-tools, thanks and tell me if you need anything.

[astrojuanlu]

Left a comment in https://github.com/google/or-tools/issues/92#issuecomment-455480575 about repackaging the official wheels instead of building from source. Perhaps it's a good first step before trying to compile from source. In any case, ortools supports make builds without Bazel, as far as I understand.

[cdeil]

I'm also very interested in this addition.

@sodre - What's the next step to make this happen?

Is https://github.com/sodre/staged-recipes/commit/5b61e96697465efff39fbcdab7e0d8af3bc179ab a good starting point for a pull request?

If the build fails on some platforms for now, maybe just activate that one platform, and leave the others for later? Something is better than nothing, no?

Thanks for your work on conda packaging!

[wolfv]

I have a working conda build locally. The first step would be to split the existing packages for the coin tools here: https://github.com/conda-forge/coincbc-feedstock (issue is open here: https://github.com/conda-forge/coincbc-feedstock/issues/37)

I can upload the recipes later...

[wolfv]

@lperron the cmake of ortools is quite opinionated -- could you add hooks to also use pkgconfig to find dependencies such as protobuf or the coin libraries?

[lperron]

+Corentin Le Molgat [e-mail]

Who wrote the cmake and is rewriting it.

Le jeu. 9 avr. 2020 à 20:48, Wolf Vollprecht [e-mail] a
écrit :

@lperron https://github.com/lperron the cmake of ortools is quite
opinionated -- could you add hooks to also use pkgconfig to find
dependencies such as protobuf or the coin libraries?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/conda-forge/staged-recipes/issues/2717#issuecomment-611693554,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3OC2WHVTLPFBWSV5HDRLYKAVANCNFSM4DGCFYEA
.

[wolfv]

@cdeil do you want to take over these recipes: https://github.com/wolfv/staged-recipes/tree/add_coin_or/recipes

It has working recipes for or-tools (with tests) at least on linux :)

The remaining issue is that we should probably use pgkconfig / autotools to build the coin dependencies and need to patch or-tools cmake to find the packages using pkgconfig (as we don't want to statically link the dependencies, except for Abseil where we have to as or-tools is using an "unreleased" version (abseil doesn't really seem to have releases)).

The second part of the previous point is that the coincbc package in conda-forge already vendors a bunch of libraries that will collide with the splitted libraries. I talked with @scopatz and he's fine with splitting although we haven't figured out how to do it.

The first part would be to re-open a PR here and then we need to make the existing coincbc a meta-package to the new, correct one.

[cdeil]

@wolfv - I'm just a user that tried out OR-Tools and other packages yesterday for the first time (see python-discrete-optimization-examples). I don't feel qualified to maintain the conda recipes for these large and complex packages. If you think it's useful to add me as a co-maintainer for simple tasks such as e.g. reviewing version update PRs, that's fine.

@tkralphs or anyone from the COIN project - would you be willing to co-maintain conda-forge recipes for the COIN codes or review these changes? (following up on https://github.com/conda-forge/staged-recipes/pull/10935#issuecomment-592594829)

@clemolgat-SBR - Would you or someone from OR-Tools be willing to co-maintain the conda-forge recipe for OR-Tools or review these changes?

@wolfv - I looked a bit at your COIN and OR-tools packaging proposal at: https://github.com/wolfv/staged-recipes/tree/add_coin_or/recipes

Two thoughts:

How did you come up with these version numbers for the COIN packages for OR-Tools
(see here)?
On the other hand within the COIN packages it seems you have no version constraints (see here)?
Is this a good way to maintain versions of dependencies, or will this cause problems in the future?
I see that for Homebrew all the COIN packages are maintained in one git repo by @tkralphs and at https://github.com/Mizux/homebrew-or-tools/blob/master/or-tools.rb @Mizux doesn't have COIN versions pinned for OR tools at all.

Why did you patch to CXX_STANDARD 17 for OR-Tools? Shouldn't it work for CXX_STANDARD 11 and that would be more conservative? (see here

[wolfv]

Hi @cdeil, thanks for pinging some more people.

The version numbers of the coin projects come from their releases on the relevant github projects.

google or-tools does _not_ work with the latest release of coin-clp but works with the version just before (there is an issue filed by @mizux on coin-clp.
The version constraints on the or-tools package are derived from the or-tools cmake (https://github.com/google/or-tools/blob/stable/cmake/dependencies/CMakeLists.txt)
We can assume some level of binary compatibility between releases.

The default CXX standard on conda-forge is C++17 so I had to patch this because of linker and include errors -- the default on conda-forge is C++17 so if you link multiple shared libraries that can lead to inconsistencies for example with std::string_view which exists in C++17 but not in C++11. I faced these problems at least with either protobuf or abseil which has it's own implementation of std::string_view if compiled with C++11 and otherwise uses the C++17 string_view. The issue is that header & library are not having the same interface anymore if compiled with different standards.

On the other hand the coin libraries _do not_ support C++17 and had to be explicitly compiled with C++11 :) but the code is anyways old-school so the aforementioned issues don't play a role.

Sure, I can add you as a maintainer. I'll try to work out a plan with @scopatz to get the split packages into conda-forge.

[Mizux]

Hi, I'm currently the maintainer of the python packaging process.

please notice pep 513 doesn't allow a python package to depends on system libraries, see the manylinux policy...
see: https://www.python.org/dev/peps/pep-0513/#the-manylinux1-policy

thus when building the python package we try to embed all required dependencies (as static library if possible)

We can discuss on this OR-Tools discord: https://discord.gg/ENkQrdf

[wolfv]

the conda package manager is thousand times nicer than PEP 513, it's cross-platform and language agnostic so we can (and should!) link to shared libraries as if it was a deb package :) We're not bound by the Python packaging process here.

[tkralphs]

For the version numbers for the COIN-OR projects, one is a release and the others are stables. If you really want to depend on a tagged version, they should all be releases. In general, there is a Dependencies file that is packaged with each version that says what the dependencies are. This is what coinbrew uses to figure out what versions to fetch if you want to build a particular project. Right now, the Cbc stack is still maintained in SVN (and mirrored to Github), so the URLs are actually for the SVN repo. We'll be moving to git permanently relatively soon. It would probably be good to use Dependencies file somehow here. Anyway, I would be willing to help maintain these conda packages.

Keep in mind that there are about 20 packages in the overall COIN-OR Optimization Suite, all of which I would eventually like to have packages for most/all of these. I'm not sure if the OR-Tools can interface to any of the others, but there is at least the potential to do so in the future, so that should be part of the roadmap.