Bitshares-core: Ubuntu 18.04 LTS Support

[ryanRfox]

User Story

As an Ubuntu User I want to build bitshares-core on the (forthcoming) Ubuntu 18.04 LTS platform because it is the latest version.

Notes

• Ubuntu 18.04 LTS (bionic) Release Notes
• Linux kernel 4.15
• openssl 1.1.0g (apt install openssl)
• boost 1.65.1 (apt install libboost-all-dev)

Task List

• [x] Upgrade to 18.04 (until final release 28 APR 2018)
  
  
  
  • Prerequisite: apt install update-manager-core
  
  
  • Verify: /etc/update-manager/release-upgrades is set to normal
  
  
  • Install: do-release-upgrade -d
  
  
  • follow on screen instructions
  
  
• [x] Add support for OpenSSL 1.1 (see branch openssl-1.1-support)
• [x] Add support for Boost 1.65
• [x] Update build script(s)
  
  
  
  • See @jmjatlanta comment below for complete build script
  
  
• [x] Test building on Ubuntu 18.04 LTS
  
  
  
  • [x] README.md: PR #1212
  
  
  • [x] Wiki: Update on day of release
  
  
  • [x] wiki home page
  
  
  • [x] wiki BUILD_UBUNTU page
  
  
  • [x] bump docs submodule after wiki updated.
  
  
• [x] Update documentation

[jmjatlanta]

I was able to compile and run bitshares-core on Ubuntu 18.04 (latest daily build) with the following caveats:

1. The bitshares-fc and bitshares-core branch openssl-1.1-support was used
2. The changes in bitshares-fc branch boost_1.65 was also merged in.

With a fresh install of Ubuntu 18.04, the following additional packages are necessary:
sudo apt install gcc g++ git cmake libboost-all-dev libssl-dev autoconf libcurl4-openssl-dev

Note 1: With a brand new install of git, certain commands may prompt you to do extra steps, such as git --global user.email or something similar. Read carefully, and follow their instructions.

Note 2: Certain git commands may open an editor, and ask you for a comment. There is no need to add a comment. [CTRL]+[X] will allow you to exit the editor and the process should continue normally.

Once the openssl-1.1-support and boost_1.65 branches are merged in to bitshares-fc and bitshares-core, getting the source code will be easier. But until then, the instructions are:

git clone https://github.com/bitshares/bitshares-core.git
cd bitshares-core
git checkout openssl-1.1-support
git submodule update --init --recursive
cd libraries/fc
git checkout openssl-1.1-support
git pull https://github.com/jmjatlanta/bitshares-fc.git boost_1.65
cd ../../
cmake -DCMAKE_BUILD_TYPE=Release .
make

Update 2018-05-04: boost 1.65 changes merged in, adjusted instructions

[ryanRfox]

I successfully built the project using the steps outlined in the comment above. Next, I intend to build again for the TESTNET, then run my witness_node on the 18.04 OS for further testing.

[abitmore]

While this is a good to have feature and the correct direction to go, I don't think it's very high priority. After Ubuntu 18.04 is out, it may need some time for businesses to try it in production. My estimation on this is about 3-6 months.

[operano02]

hi ;
can you explane the " boost_1.65 branches are merged in to bitshares-fc and bitshares-core, "
what does mean merged into ? how to merge into ? i'm newbie
after git pull https://github.com/jmjatlanta/bitshares-fc.git boost_1.65 command
" fatal: Couldn't find remote ref boost_1.65 " this error.

[jmjatlanta]

The work done on the boost_1.65 has now become part of the official bitshares-fc project. So, that line can be left out. I will adjust the post above.

[jmjatlanta]

To compile for the latest release of TESTNET ( instructions as of 12-May-2018 )

# Pull down the bitshares-core repository and place it in the directory
# bitshares-core (directory should not already exist)
git clone https://[email protected]/bitshares/bitshares-core
# Enter your new copy of the repository
cd bitshares-core
# checkout the testnet branch of bitshares-core
git checkout testnet
# checkout all the submodules needed
git submodule update --init --recursive
# merge in the openssl-1.1-support branch of bitshares-core
git pull https://github.com/bitshares/bitshares-core openssl-1.1-support
# move to the bitshares-fc directory
cd libraries/fc
# grab the latest, if you want to ( this is optional )
git checkout master
# merge in the openssl-1.1-support branch of bitshares-fc
git pull https://github.com/bitshares/bitshares-fc.git openssl-1.1-support
# go back to the bitshares-core directory
cd ../..
# prepare your Makefiles ( as you normally would )
cmake -DCMAKE_BUILD_TYPE=Release .
# compile the applications ( as you normally would )
make

[abitmore]

@jmjatlanta that means the only thing we need to do is to merge the OpenSSL 1.1 PR's?

We need to test:

• sync testnet and mainnet,
• connect a light wallet and make some transactions,
• sign blocks and
• etc.

[jmjatlanta]

@abitmore I have not tested the OpenSSL-1.1-support branch completely.

I would add that we need to test compiling with OpenSSL 1.0.2 with the 1.1-support branch merged in.

If we can get everything tested well and the 1.1-support branch merged in to the other branches, it would be a good win for those running 18.04 LTS, as they would not need anything beyond stuff they can apt-get plus the bitshares-core branch of their choice. The instructions above would be much shorter.

[ryanRfox]

I am failing to build at the same point with both OpenSSL 1.0.2g & 1.1.0g as demonstrated here:

@jmjatlanta I really appreciate the detailed instructions with inline comments above.

[pmconrad]

See #921, it includes the removal of these API calls

[ryanRfox]

Success! Yes, after commenting out the lines related to blind_sign API calls and deterministic_openssl_rand found in #921 the build completed without errors.

[FrankBlabu]

Are the instructions in the comments above still valid ? I get the reply "Couldn't find remote ref openssl-1.1-support" when trying to pull it.

I would appreciate any hint here. I updated my system to Ubuntu 18.04 without checking for the witness node compatibility before and now I'm stuck and cannot go back easily.

[pmconrad]

The branch has been merged into develop, please check out develop branch and build that. Don't forget to git submodule update --init --recursive.

[ryanRfox]

@FrankBlabu you may observe the build process for the develop branch provided by Travis-CI:
https://travis-ci.org/bitshares/bitshares-core/builds/391476199

Focus on lines 418 & 428:

git clone --depth=1 --branch=develop https://github.com/bitshares/bitshares-core.git
git submodule update --init --recursive

As needed, please open a Build Error Issue and complete the template with the relevant details:
https://github.com/bitshares/bitshares-core/issues/new?template=build_error.md

[jmjatlanta]

Updated instructions as of 15-June-2018 for develop branch, based on a fresh install of Ubuntu 18.04:

sudo apt install gcc g++ git cmake libboost-all-dev libssl-dev autoconf libcurl4-openssl-dev
git clone --branch=develop https://github.com/bitshares/bitshares-core.git
cd bitshares-core
git submodule update --init --recursive
cmake -DCMAKE_BUILD_TYPE=Release .
make

[ying2025]

I have the similiar problem. But My openssl is 1.0.2. How I solve the following problems?
/home/zhr/bitshare/bitshares-core/libraries/deterministic_openssl_rand/deterministic_openssl_rand.cpp:92:1: error: invalid conversion from ‘void ()(const void, int)’ to ‘int ()(const void, int)’ [-fpermissive]
};
^
/home/zhr/bitshare/bitshares-core/libraries/deterministic_openssl_rand/deterministic_openssl_rand.cpp:92:1: error: invalid conversion from ‘void ()(const void, int, double)’ to ‘int ()(const void, int, double)’ [-fpermissive]
libraries/deterministic_openssl_rand/CMakeFiles/deterministic_openssl_rand.dir/build.make:62: recipe for target 'libraries/deterministic_openssl_rand/CMakeFiles/deterministic_openssl_rand.dir/deterministic_openssl_rand.cpp.o' failed
make[2]: * [libraries/deterministic_openssl_rand/CMakeFiles/deterministic_openssl_rand.dir/deterministic_openssl_rand.cpp.o] Error 1
CMakeFiles/Makefile2:790: recipe for target 'libraries/deterministic_openssl_rand/CMakeFiles/deterministic_openssl_rand.dir/all' failed
make[1]: [libraries/deterministic_openssl_rand/CMakeFiles/deterministic_openssl_rand.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: ** [all] Error 2
zhr@ubuntu:~/bitshare/bitshares-core$ openssl version
OpenSSL 1.0.2g 1 Mar 2016

[jmjatlanta]

Hrmmm. It seems you are having a different problem @ying2025. Please create a new issue here:

https://github.com/bitshares/bitshares-core/issues/new?template=build_error.md

[ying2025]

ubuntu:~/bitshare/bitshares-core$ git checkout openssl-1.1-support
error: pathspec 'openssl-1.1-support' did not match any file(s) known to git.

[abitmore]

@ying2025 please follow these steps:

git clone https://github.com/bitshares/bitshares-core.git
cd bitshares-core
git checkout develop
git submodule update --init --recursive
# Then run `cmake` and `make`

[rsswlkr]

For some reason this node didn't trust the git ssl certificate/server, so I had to work around the error message.

bitshares-core$ sudo git remote set-head origin --auto
fatal: unable to access 'https://github.com/bitshares/bitshares-core.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
bitshares-core$ sudo export GIT_SSL_NO_VERIFY=1
sudo: export: command not found
bitshares-core$ sudo git config --global http.sslverify false
bitshares-core$ sudo git remote set-head origin --auto
origin/HEAD set to master

and carry on ... switching this setting back to true immediately after the update.

[fvisconti]

I'm on Ubuntu 18.04
openssl version: OpenSSL 1.1.0g 2 Nov 2017
followed all the steps mentioned by @abitmore, but facing the issue as in #409, which is said to be fixed on Ubuntu 17.10.
I configured with cmake -DCMAKE_BUILD_TYPE=Release as @jmjatlanta said.
Don't know exactly how to move (not even why this should work with Ubuntu 17.10 and not with 18.04).

[jmjatlanta]

On Ubuntu 18.04, things should work just fine if you are using the develop branch. In fact, it is slightly easier as there is no need to compile specific boost or openssl versions.

There must be some kind of problem (misconfiguration?) between doxygen and perl, if I am reading issue #409 correctly. I wasn't around when that issue appeared, so I can only speculate. Please post your build log and I'll see if I can help.

Note: Github messes with the formatting of the build log. I've found it better to post on something like pastebin.com and post a link here.

[fvisconti]

This is the last message in the build log.
In #409, they say to comment out lines in CMakeLists.txt, but I found they have been replaced in the develop branch..

[ 81%] Generating api_documentation.cpp
Can't locate doxygen/perlmod/DoxyDocs.pm in @INC (you may need to install the doxygen::perlmod::DoxyDocs module) (@INC contains: /hom
e/francesco/WORK/XXX/bitshares-core/libraries/wallet /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/per
l/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /usr/local/li
b/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /home/francesco/WORK/XXX/bitshares-core/libraries/wallet/generate_api_docum
entation.pl line 6.
libraries/wallet/CMakeFiles/graphene_wallet.dir/build.make:62: recipe for target 'libraries/wallet/api_documentation.cpp' failed
make[2]: *** [libraries/wallet/api_documentation.cpp] Error 2
CMakeFiles/Makefile2:1826: recipe for target 'libraries/wallet/CMakeFiles/graphene_wallet.dir/all' failed
make[1]: *** [libraries/wallet/CMakeFiles/graphene_wallet.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

[jmjatlanta]

If you don't want to create the documentation, you could probably comment out line 4 of libraries/wallet/CMakeLists.txt like so...

# find_package( Doxygen )

I haven't tested it, but that should skip the documentation creation process. You'll have to re-run cmake, then run make, and let me know the results.

[fvisconti]

Ok, going to.

[fvisconti]

Builded succesfully, many thanks.

[abitmore]

If unable to find doxygen, build process should not fail, but skip the step gracefully.

By reading #409 and #479, it seems both doxygen and perl have been found, but the build is still failing? The #479 fix no longer works?

[jmjatlanta]

I agree @abitmore. A small clarification: the package doxygen is being found. The problem seems to be that sometimes it is not finding the perl module in libraries/wallet/doxygen/perlmod/DoxyDocs.pm. I do not know why it finds it on some boxes, but does not find it on others.

[oxarbitrage]

closing by work done before and final updating the docs at https://github.com/bitshares/bitshares-core/pull/1212
some testing was done by the core team today with ubuntu 18.01. gcc 7.3.0, boost 1.65.1(found in default installations) and everything looks good.

[abitmore]

Still some work to be done before final mainnet release. OP updated.

[oxarbitrage]

ok, got it, thanks.

[abitmore]

Done.

[abitmore]

Does the doxygen issue still exist? Will this help? https://github.com/steemit/steem/pull/3119/commits/071f6a997357a116d96c0b0215473d3c20fd5316

[zhangbo-miuNEST]

Does the doxygen issue still exist? Will this help? steemit/steem@071f6a9

It works.