Or-tools: Option to build without SCIP with Makefile

[ankane]

What language and solver does this apply to?
C++

Describe the problem you are trying to solve.

Hi, it looks like SCIP was added in the latest release, which doesn't have an open source license. It'd be nice if there was a way to build OR-Tools without it.

Describe the solution you'd like

USE_SCIP or similar option for Makefile build, since CMake is still considered experimental.

Describe alternatives you've considered

Use experimental CMake build instead.

[lperron]

Why is that a problem ?

[ankane]

I'd like to add OR-Tools to Homebrew, so Mac users can do brew install or-tools, but Homebrew requires software to have an open source (OSI-approved) license.

More generally, I think it's nice for users to be able to build OR-Tools without having to worry about non-open source software being included. It looks like others have this same concern, fwiw: https://github.com/google/or-tools/issues/2129

[lperron]

OK, We will do that asap.
Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

Le mar. 11 août 2020 à 10:57, Andrew Kane [e-mail] a
écrit :

I'd like to add OR-Tools to Homebrew, so Mac users can do brew install
or-tools, but Homebrew requires software to have an open source
(OSI-approved) license.

More generally, I think it's nice for users to be able to build OR-Tools
without having to worry about non-open source being included. It looks like
others have this same concern, fwiw: #2129
https://github.com/google/or-tools/issues/2129

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2134#issuecomment-671822314,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3OHY47UJIJCCUOAVBLSAEBZBANCNFSM4P2ZJROQ
.

[lperron]

Fixed for make based builds on master.

[lperron]

Side comment, do you need help adding or-tools to homebrew ? This is one of or-tools missing delivery channel. Do you have a description of what you want to include ?

[ankane]

Thanks for adding a flag and offering to help! I have working formula here: https://github.com/ankane/homebrew-core/blob/or-tools/Formula/or-tools.rb

I submitted CBC to Homebrew earlier today, so once that's merged and a new release of OR-Tools is out, we should be good to go. I'll test the new flag in a bit to confirm everything works.

[ankane]

Just tried with master and it's failing with cp: dependencies/sources/scip-7.0.1/src/lpi/lpi_glop.cpp: No such file or directory.

Full logs: https://github.com/ankane/homebrew-core/runs/971549010?check_suite_focus=true

[lperron]

You must add BUILD_SCIP=OFF manually in Makefile.local before make
third_party, or pass it to make

option a:
checkout
make Makefile.local
echo BUILD_SCIP=OFF >> Makefile.local
make third_party
make all

option b:
make BUILD_SCIP=OFF third_party all
Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

Le mar. 11 août 2020 à 16:20, Andrew Kane [e-mail] a
écrit :

Just tried with master and it's failing with cp:
dependencies/sources/scip-7.0.1/src/lpi/lpi_glop.cpp: No such file or
directory.

Full logs:
https://github.com/ankane/homebrew-core/runs/971549010?check_suite_focus=true

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2134#issuecomment-671975709,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3O5SQOH2B363JBSNX3SAFHRTANCNFSM4P2ZJROQ
.

[lperron]

I read your log.

1) I pushed the fix for the makefile.

2) I believe you do not need to run make third_party if every dependency is
compiled before.
You will need to generate Makefile.local (make Makefile.local).
Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

Le mar. 11 août 2020 à 17:12, Laurent Perron [e-mail] a écrit :

You must add BUILD_SCIP=OFF manually in Makefile.local before make
third_party, or pass it to make

option a:
checkout
make Makefile.local
echo BUILD_SCIP=OFF >> Makefile.local
make third_party
make all

option b:
make BUILD_SCIP=OFF third_party all
Laurent Perron | Operations Research | [email protected] | (33) 1 42 68
53 00

Le mar. 11 août 2020 à 16:20, Andrew Kane [e-mail] a
écrit :

Just tried with master and it's failing with cp:
dependencies/sources/scip-7.0.1/src/lpi/lpi_glop.cpp: No such file or
directory.

Full logs:
https://github.com/ankane/homebrew-core/runs/971549010?check_suite_focus=true

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2134#issuecomment-671975709,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3O5SQOH2B363JBSNX3SAFHRTANCNFSM4P2ZJROQ
.

[ankane]

I tried the latest master without make third_party (and USE_SCIP=OFF), but it fails with:

makefiles/Makefile.third_party.unix.mk:46: *** Third party SCIP files was not found! did you run 'make third_party' or set UNIX_SCIP_DIR ?.  Stop.

Full logs: https://github.com/ankane/homebrew-core/runs/972227456?check_suite_focus=true

[lperron]

Oh sh*t

Le mar. 11 août 2020 à 19:04, Andrew Kane [e-mail] a
écrit :

I tried the latest master without make third_party (and USE_SCIP=OFF),
but it fails with:

makefiles/Makefile.third_party.unix.mk:46: * Third party SCIP files was not found! did you run 'make third_party' or set UNIX_SCIP_DIR ?. Stop.

Full logs:
https://github.com/ankane/homebrew-core/runs/972227456?check_suite_focus=true

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2134#issuecomment-672097603,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3JTUNOQROUJYKT2NP3SAF22BANCNFSM4P2ZJROQ
.

[lperron]

Fixed,

thanks
Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

Le mar. 11 août 2020 à 19:12, Laurent Perron [e-mail] a écrit :

Oh sh*t

Le mar. 11 août 2020 à 19:04, Andrew Kane [e-mail] a
écrit :

I tried the latest master without make third_party (and USE_SCIP=OFF),
but it fails with:

makefiles/Makefile.third_party.unix.mk:46: * Third party SCIP files was not found! did you run 'make third_party' or set UNIX_SCIP_DIR ?. Stop.

Full logs:
https://github.com/ankane/homebrew-core/runs/972227456?check_suite_focus=true

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/2134#issuecomment-672097603,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3JTUNOQROUJYKT2NP3SAF22BANCNFSM4P2ZJROQ
.

[Mizux]

FYI i fixed for windows too ;)

[ankane]

Still no luck - same error (with same situation) as https://github.com/google/or-tools/issues/2134#issuecomment-672097603

Full logs: https://github.com/ankane/homebrew-core/runs/973737877?check_suite_focus=true

[Mizux]

Few thinks (in any order):

• Currently the variable is called BUILD_SCIP BUT i'll rename it as USE_SCIP to follow the CMake naming convention
  -> I'll let you know on this thread once done
• FYI I have created a formula long time ago https://github.com/Mizux/homebrew-or-tools/blob/master/or-tools.rb
• Homebrew already provide a "full" coin-or-tools/coinor/cbc (using the monolithic svn archive IIRC) so you don't need your custom cbc, cgl, clp, osi and coinutils custom formulae (unless this formula is broken ?).

[Mizux]

just renamed it, USE_SCIP if you want to test...

[ankane]

Thanks @Mizux, missed that it had a different name. It currently builds but make install_cc USE_SCIP=OFF ... fails (don't mind the 0.0.0 version):

cp -R dependencies/install/include/scip "/usr/local/Cellar/or-tools/0.0.0/include"
cp: dependencies/install/include/scip: No such file or directory
mkdir "/usr/local/Cellar/or-tools/0.0.0/include/ortools"
make: *** [install_third_party] Error 1
make: *** Waiting for unfinished jobs....

Re Homebrew: All dependencies must be in the official repo. The cbc formula was just merged, so no blockers there.

[Mizux]

install rule seems broken... on my way to fix them

[ankane]

Hey @Mizux, the latest commit seemed to fix it. I'll submit to Homebrew whenever a new version is out.

[Mizux]

@ankane just FYI now or-tools depends on C++17 revision so we have to use abseil-cpp build using C++17 (yes abseil-cpp change according to the C++ revision used).

Unfortunately, as expected, homebrew/homebrew-core only provide abseil-cpp in.... C++11 -> ortools.rb is no longer compiling against the "homebrew ecosystem"
see: https://github.com/Mizux/homebrew-or-tools/runs/1031099691?check_suite_focus=true

[ankane]

Does that just mean we need to wait until the next abseil LTS version is out?

[Mizux]

It means we need to have an abseil package compiled against c++17.

I've just sent a bottle to the homebrew community -> https://discourse.brew.sh/t/abseil-and-c-revision/8641

[ankane]

It doesn't look like any formula currently depend on it (just search for abseil in https://github.com/homebrew/homebrew-core), so should be fine to make the change.

[Mizux]

start to have something working: https://github.com/Mizux/homebrew-or-tools/runs/1035669937?check_suite_focus=true
Next step:

• [ ] Send PR to homebrew-core to modify abseil.rb using my version
• [ ] Send PR to homebrew-core to integrate ortools.rb (CMake based build) once release v8.0 is out...

[ankane]

Hey @Mizux, was able to get v8.0 working with the following changes: https://github.com/Homebrew/homebrew-core/compare/master...ankane:or-tools3. Let me know if you'd like me to submit, or you'd like to do it.

[Mizux]

@ankane well I was waiting to release v8.0 to propose my homebrew package ;)

our ortools.rb and mine differ a little bit (e.g. you don't provide head, you are using archive, i'm using git), i'll try to send you comment then if you want to provide your version, I'm ok (i.e. don't care if you or me send the PR to homebrew community, I just want to have it integrated inside homebrew-core)

FYI: I already ask on homebrew forum if it will be ok to bump abseil to C++17, it should be ok since no one is using it yet (brew reverse dependencies show nothing)
see: https://github.com/or-tools/homebrew_or-tools/issues/1#issuecomment-672968696

[ankane]

Homebrew prefers the tar.gz url over git whenever possible for the url. Just added the head url (I wasn't sure if Homebrew allows this in new formula, but it doesn't generate a warning with the new formula audit). Anyways, feel free to run with your version / steal from this if there are other changes you want to make.

[Mizux]

@ankane few bad news:

• adding revision to abseil.rb will break the build

Emacs Lisp files have been installed to:
  /usr/local/share/emacs/site-lisp/cmake
Error: Testing requires the latest version of abseil
Error: Process completed with exit code 1.

see: https://github.com/or-tools/homebrew_or-tools/actions/runs/303942900

• greping abseil, it seems grpc make it through homebrew-core using c++11 version...
  note: in my github action workflow, I've tried to find reverse deps

Run brew uses --recursive --include-build --include-test abseil
  brew uses --recursive --include-build --include-test abseil
  shell: /bin/bash -e {0}
  env:
    VERSION: --HEAD
apache-arrow // depends on grpc
apache-arrow-glib // depends on apache-arrow
dvc // depends on apache-arrow
grpc // depends directly on abseil

see: https://github.com/or-tools/homebrew_or-tools/runs/1246430593?check_suite_focus=true#step:5:1

Side note

• Your or-tools.rb do not provide a .pc which can be cool for auto-tool users and few others tools (ed even cmake has a module to read them...)
  see: https://github.com/Homebrew/homebrew-core/compare/master...Mizux:ortools

• you are missing -DBUILD_SAMPLES=OFF IIRC

EDIT: I'll send a PR and see what's going on with homebrew team

[ankane]

Looks like it was accepted 🎉 Thanks @Mizux!