Or-tools: Installing or-tools from git?

[ax487]

I was wondering whether it is possible to actually install the C++-library on linux. I looked at the instructions I was able to get my hands on a build for a rather recent version of ubuntu, but as far as I can tell from looking at the Makefiles there is no way to install the C++ includes / libraries to a specific folder. Is there something I am missing?

[dlahlou]

Hello,

The includes are available under "src/", and the libraries are compiled
under "lib/" after running "make all". Is there any specific file you are
looking for?

Regards,

On Fri, Dec 16, 2016 at 3:53 PM, ax487 [e-mail] wrote:

I was wondering whether it is possible to actually install the C++-library
on linux. I looked at the instructions
https://developers.google.com/optimization/installing I was able to get
my hands on a build for a rather recent version of ubuntu, but as far as I
can tell from looking at the Makefiles there is no way to install the C++
includes / libraries to a specific folder. Is there something I am missing?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292, or mute the thread
https://github.com/notifications/unsubscribe-auth/ASwpfCacVyezXHg16_sF--pKWUu7Jc_tks5rIqX8gaJpZM4LPQrP
.

--

driss

[ax487]

Well, that is not exactly what installing traditionally means:

• The "includes" folder should for example not contain "*.cc" files, only the headers.
• The "includes" folder should not contain "*.proto" files (at least imho), instead it should only contain generated headers

So the contents and organization of the "includes" should be different from the "src" folder and generally an additional step is required in order to populate the folder. This has been done in the ubuntu tarball. Generally generating an includes directory is an automated step in most advanced build systems.

[ian-bertolacci]

Agreed.
The way this is packaged makes it difficult to utilize.
Even the prebuilt releases require manual installation.

A make install, or other equivalent, is trivial to implement.

[ax487]

Well, I would not call it trivial, but still. Unfortunately I have come to the conclusion that in its current form the project is not usable for us. This is due to the following reasons:

Disregarding conventions with respect to dependency management.

Linux distributions have tools called package managers. These take care of keeping packages up to date and manage dependencies internally. Many modern build systems, noticeably cmake incorporate package managers by looking for libraries on their own. Hand-written Makefiles (as they are in this project) are generally not able to provide this functionality. Instead of using a proper build system the developers chose to somehow include the dependencies directly. This has the following drawbacks:

• The compile time increases by a huge amount, the dependencies, i.e. their sources / headers / objects clutter the file system
• It forces any developer to use the exact same versions of libraries as this project. This is a real problem as well, it again makes it impossible to auto-detect libraries and their versions. It makes using advanced build tools nearly impossible and requires enormous workarounds (aka version hell).

Not providing an installation routine

As was said, it is impossible for a user to determine how an include directory should look like. It is nearly possible to build a working package to be included in any distribution. It is also nearly impossible to update ortools / to keep track of the git version.

I don't mean this as an insult, I don't mean to offend any of the developers which have undoubtedly put a lot of work into this project. I am just rather exhausted because I spent four hours trying to get a project using ortools as a library working on a student's laptop. __Anyone willing to give ortools a try in a project should be aware that it is almost impossible to get the project to work reliably on different machines without writing custom tooling or performing a lot of manual labor for each machine / update.__
I think this is a fixable problem. I am also aware that nobody likes build systems very much, so probably not a lot is going to happen.

[lperron]

Hi,

I am sorry for your problems. I will try to tell you my side of it.
I am using complex dependencies, they have evolved over time and they are
not very popular (Coin CBC is nice, but not exactly up to date on linux
distros).

So, my issues is that on linux distros, I need to depend on packages that
are incompatible between versions, that are not up to date, and I tried the
global approach and it failed miserably.
And I need to support windows, linux and mac.

This being said, nothing prevent anybody to not call make third_party.
If you are using C++, you need 3 dependencies:
gflags/gflags
protobuf >= 3.0
Recent CBC (even this one is optional)

And then you fill in Makefile.local with the right paths.

Actually, I have stopped precompiling most of the dependencies recently, I
dropped the need for bison/flex (a total nightmare as bison 2 is not
compatible with bison 3).
I stopped trying to compile glpk and scip and asked for the path to the
installed versions.

But protobuf 3.0 is only supported since ubuntu 16.10, not on mac and not
on windows. So I am at a loss.
CBC is 2.8.12 on the latest ubuntu (and 2.9.8 on the coin site), and this
versions contains a lot of fixed for bugs I reported.

The list is long.

I am trying to switch to cmake, but protobuf and gflags are not trivially
supported. And this is a lengthy process.
I have implemented experimental bazel support. This is fantastic, it pulls
gflags and protobuf perfectly and compiles everything fast and without any
fuss. Still, it does not work on windows (I tried) and it is limited to C++.

So, my current answer is use the binary package.
I did nuget and pypi to simplify the process. Pypi is a bag of hurt, but
sometimes it works. Maven is out of the question because binary libraries
are not supported.

And I would love rpm and .deb support, but I do not have the skillset, nor
the time to invest in it.

I am open to suggestions.

Thanks

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-04-26 15:50 GMT+02:00 ax487 [e-mail]:

Well, I would not call it trivial, but still. Unofrtunately I have come to
the conclusion that in its current form the project is not usable for us.
This is due to the following reasons:
Disregarding conventions with respect to dependency management.

Linux distributions have tools called package managers. These take care of
keeping packages up to date and manage dependencies internally. Many modern
build systems, noticeably cmake incorporate package managers by looking
for libraries on their own. Hand-written Makefiles (as they are in this
project) are generally not able to provide this. Instead the developers
chose to somehow include the dependencies directly. This has the following
drawbacks:

• The compile time increases by a huge amount, the dependencies, their
  sources / headers / objects clutter the file system
• It forces anyone using cmake or alike to use the exact same versions
  of libraries as this project. This is a real problem as well, it again
  makes it impossible to auto-detect libraries and their versions. It makes
  using advanced build tools nearly impossible and requires enormous
  workarounds (aka version hell).

Not providing an installation routine

As was said, it is impossible for a user to determine how an include
directory should look like. It is nearly possible to build a working
package to be included in any distribution. It is also nearly impossible to
update ortools / to keep track of the git version.

I don't mean this as an insult, I don't mean to offend any of the
developers which have undoubtedly put a lot of work into this project. I am
just rather exhausted because I spent four hours trying to get a project
using ortools as a library working on a student's laptop. Anyone willing
to give ortools a try in a project should be aware that it is almost
impossible to get the project to work reliably on different machines
without writing custom tooling or performing a lot of manual labor for each
machine / update.
I think this is a fixable problem. I am also aware that nobody likes build
systems very much.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-297414682,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17QkuMbzyH3RPi98z5P4M4DWxPy54ks5rz0uZgaJpZM4LPQrP
.

[lperron]

Trivial :-)

I have spent more that 6 years on or-tools and I would love to see a
trivial make install that works on linux, mac where you need to create
framework, and windows where you do not have make.

I do not want to use configure, ever, and cmake has some learning curve.

Any contrib is more than welcome, I am really doing my best.

Thanks

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-04-14 1:18 GMT+02:00 Ian Bertolacci [e-mail]:

Agreed.
The way this is packaged makes it difficult to utilize.
Even the prebuilt releases require manual installation.

A make install, or other equivalent is trivial.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-294045801,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17fN7WnD7nPliQZrJv8J1sMz8AE49ks5rvq1bgaJpZM4LPQrP
.

[ian-bertolacci]

You're right. 'Trivial' is definitely the wrong word here.
However it seems like there is at least one easy road to go down.

For myself, I build or-tools to get the binaries, and then download the prebuilt package to get the include/ directory.

If the directory hierarchy of the repository was such that lib/ and include/ could (at the very least) be copied into a more suitable location (/usr/ for example), that would make or-tools much easier to distribute (at least for smaller projects like mine).

[ambasta]

Hey @lperron
I've been trying to port the build system to CMake and am making some project structure changes to meet the same. Could you please take a look and let me know if its an acceptable approach? You can find the fork at https://github.com/ambasta/or-tools

Currently, I've only added the ability to build the C++ components alone. But I am working on adding support for python, java and c#

Will also add the ability to fall back to locally compiled third party libraries as the makefiles do now

[lperron]

hi,

nice!!!

do you need the extra src/ directory now that everything is in ortools?

In term of priorities, C++ and python are high priority as pypi is a
nightmare.

The rest seems less important. Java is okayish, C# has nuget support.

PS: We have a local branch on called cmake, you can have a look.

Thanks

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-05-24 13:49 GMT+02:00 Amit Prakash Ambasta [e-mail]:

Hey @lperron https://github.com/lperron
I've been trying to port the build system to CMake and am making some
project structure changes to meet the same. Could you please take a look
and let me know if its an acceptable approach? You can find the fork at
https://github.com/ambasta/or-tools

Currently, I've only added the ability to build the C++ components alone.
But I am working on adding support for python, java and c#

Will also add the ability to fall back to locally compiled third party
libraries as the makefiles do now

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-303700805,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17Z0QrTikEarsxK-PDxErSUCsrNCnks5r9Bk1gaJpZM4LPQrP
.

[ambasta]

Hi @lperron

In my first cut, I've separated the src directory, but it should go away soon enough. Python should be done today, and since CMake does support C#, I try to have a single build tool for all the languages :)

[ax487]

@ambasta: Your work is certainly much appreciated. Just one question: Did you intentionally disable issues in your repository?

[ambasta]

@ax487 Not at all, I just forked this repo. Enabled issues now !

[lperron]

Hi,

I really like it.

I have a few comments:

• Is there an easy way to compile dependencies with cmake, some
  precompiled version? I guess on linux you just install the packages. On
  mac, I do not know.
• include dir: you can remove examples from the search path
• protobuf: version >= 3.0

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-05-29 15:32 GMT+02:00 Amit Prakash Ambasta [e-mail]:

@ax487 https://github.com/ax487 Not at all, I just forked this repo.
Enabled issues now !

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-304662170,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17T4z9yvUbCjIRAtAAia-rczgKkOkks5r-skDgaJpZM4LPQrP
.

[dlahlou]

ExternalProject_Add installs dependencies smoothly.

You can check out this file :
https://github.com/google/or-tools/blob/cmake/build/dependencies/CMakeLists.txt

On 30 May 2017 at 11:41, lperron [e-mail] wrote:

Hi,

I really like it.

I have a few comments:

• Is there an easy way to compile dependencies with cmake, some
  precompiled version? I guess on linux you just install the packages. On
  mac, I do not know.
• include dir: you can remove examples from the search path
• protobuf: version >= 3.0

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68
53
00

2017-05-29 15:32 GMT+02:00 Amit Prakash Ambasta <[email protected]

:

@ax487 https://github.com/ax487 Not at all, I just forked this repo.
Enabled issues now !

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-304662170,
or mute the thread
auth/AKj17T4z9yvUbCjIRAtAAia-rczgKkOkks5r-skDgaJpZM4LPQrP>
.

>

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-304852848,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ASwpfDwBnvDfFVvcJkh2PDEmtoP3kKCBks5r_AB7gaJpZM4LPQrP
.

--
الشعر أنا، وأنا الشعر

[ian-bertolacci]

Is an install script in the works?

[lperron]

Not yet,

But for linux, I have added docker support that will seamlessly build
binary artifacts that you can install anywhere.

cd tools/docker
make utunbu16.04-archive will build a
standalone or-tools_Ubuntu-16.04-64bit_v6.2.4392.tar.gz in the
tools/docker/export subdirectory.

You can then install it anywhere and build from it.

I wish we could build mac and windows version in a purely asynchronous way
like that.

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-07-20 13:04 GMT-07:00 Ian Bertolacci [e-mail]:

Is an install script in the works?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-316814686,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17fcK4QBeAT11JRcoq-cchBf0-mf5ks5sP7LWgaJpZM4LPQrP
.

[ian-bertolacci]

While docker support is great for some users, our needs are much simpler. Is there anything that can be done to help expedite a make install feature? I'd be willing to write that functionality if I knew what needed to be moved where.

How is the prebuilt version of Google Or-Tools generated?

[ambasta]

Have you tried the cmake build?

[ian-bertolacci]

Im checking out the most recent tag of the repository, which appears to use cmake. As far as I can tell, it does not have a make install feature.

[lperron]

I have written a 'crude' make install for C++ libraries

update to latest git, and type

make install TARGET_DIR="path_to_target"

currently, it will mkdir "path_to_target" and copy the relevant files.

Tell me any improvements you would like to see.

Thanks

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-07-20 15:25 GMT-07:00 Ian Bertolacci [e-mail]:

Im checking out the most recent tag of the repository, which appears to
use cmake. As far as I can tell, it does not have a make install feature.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-316847467,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17ZcpAqCMj5IkTxF0cjbfP5nLaqZjks5sP9PQgaJpZM4LPQrP
.

[ian-bertolacci]

Currently theres a build issue with HEAD (f97b36ea51d510f4a7377edb5bbca1a1c93ea539), so I cant try it out.
I'll come back when it gets resolved.

In file included from ./ortools/sat/boolean_problem.cc:14:0:
./ortools/sat/boolean_problem.h:19:37: fatal error: ortools/sat/cp_model.pb.h: No such file or directory
compilation terminated.
makefiles/Makefile.gen.mk:1756: recipe for target 'objs/sat/boolean_problem.o' failed

If anyone's interested.

[lperron]

I just pushed an update to dependencies.
Can you try it out?

Thanks

Laurent Perron | Operations Research | [email protected] | (33) 1 42 68 53
00

2017-07-21 9:15 GMT-07:00 Ian Bertolacci [e-mail]:

Currently theres a build issue with HEAD (f97b36e
https://github.com/google/or-tools/commit/f97b36ea51d510f4a7377edb5bbca1a1c93ea539),
so I cant try it out.
I'll come back when it gets resolved.

In file included from ./ortools/sat/boolean_problem.cc:14:0:
./ortools/sat/boolean_problem.h:19:37: fatal error: ortools/sat/cp_model.pb.h: No such file or directory
compilation terminated.
makefiles/Makefile.gen.mk:1756: recipe for target 'objs/sat/boolean_problem.o' failed

If anyone's interested.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/292#issuecomment-317044258,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKj17dfu9l-TScbOBYbC_NC6fwsT_Fumks5sQM6MgaJpZM4LPQrP
.

[ian-bertolacci]

This looks pretty good. My suggestion is to not install the example apps and readme.
Users like me dont need that installed anywhere.

[lperron]

Good idea.
I removed examples, readme, makefile, kept the license, and renamed the command to install_cc

Thanks

[ian-bertolacci]

Thanks for doing this. Really makes our lives easier!