scylla 🚀 - Add ArchLinux Support

Tried to build manually but got an error:

$ yaourt -S libyaml lz4 zlib snappy jsoncpp thrift antlr3 gnutls ninja ragel libaio crypto++ xfsprogs numactl hwloc libpciaccess libxml2 python-pyparsing lksctp-tools protobuf libunwind
# missing: libasan libubsan (i think this already included on the compiler)

$ ./configure.py --mode=release --with=scylla --disable-xen
666.development-0.20170227.0198d8e4
Notice: -fsanitize=vptr is broken, disabling; some debug mode tests are bypassed.
[1/1] GEN build/release/seastar.pc

$ ninja build/release/scylla -j2     
[42/278] CXX build/release/gen/Cassandra.o
FAILED: build/release/gen/Cassandra.o 
g++ -MD -MT build/release/gen/Cassandra.o -MF build/release/gen/Cassandra.o.d -std=gnu++1y -g  -Wall -Werror -fvisibility=hidden -pthread -I/home/asd/MEGA/github.com-kokizzu/scylla/seastar -I/home/asd/MEGA/github.com-kokizzu/scylla/seastar/fmt -I/home/asd/MEGA/github.com-kokizzu/scylla/seastar/build/release/gen  -march=nehalem -Ifmt -DBOOST_TEST_DYN_LINK -Wno-overloaded-virtual -DFMT_HEADER_ONLY -DHAVE_HWLOC -DHAVE_NUMA -DHAVE_LZ4_COMPRESS_DEFAULT  -O2 -DBOOST_TEST_DYN_LINK  -Wno-maybe-uninitialized -DHAVE_LIBSYSTEMD=1 -I. -I build/release/gen -I seastar -I seastar/build/release/gen -c -o build/release/gen/Cassandra.o build/release/gen/Cassandra.cpp
build/release/gen/Cassandra.cpp: In member function ‘uint32_t org::apache::cassandra::Cassandra_login_args::read(apache::thrift::protocol::TProtocol*)’:
build/release/gen/Cassandra.cpp:19:11: error: ‘org::apache::thrift’ has not been declared
   apache::thrift::protocol::TInputRecursionTracker tracker(*iprot);

... lots of similar lines

ninja: build stopped: subcommand failed.

Which thrift version I should use? Currently I install 0.10.0-1

kokizzu on 27 Feb 2017

ah nevermind found it on: https://github.com/scylladb/scylla/issues/841

kokizzu on 27 Feb 2017

Ah no, I've tried with 0.9.1, 0.9.2, 0.9.3 also failed, but with different error:

$ ninja build/release/scylla -j2                           
[1/238] CXX build/release/gen/cassandra_constants.o
FAILED: build/release/gen/cassandra_constants.o 
g++ -MD -MT build/release/gen/cassandra_constants.o -MF build/release/gen/cassandra_constants.o.d -std=gnu++1y -g  -Wall -Werror -fvisibility=hidden -pthread -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar/fmt -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar/build/release/gen  -march=nehalem -Ifmt -DBOOST_TEST_DYN_LINK -Wno-overloaded-virtual -DFMT_HEADER_ONLY -DHAVE_HWLOC -DHAVE_NUMA -DHAVE_LZ4_COMPRESS_DEFAULT  -O2 -DBOOST_TEST_DYN_LINK  -Wno-maybe-uninitialized -DHAVE_LIBSYSTEMD=1 -I. -I build/release/gen -I seastar -I seastar/build/release/gen -c -o build/release/gen/cassandra_constants.o build/release/gen/cassandra_constants.cpp
In file included from build/release/gen/cassandra_constants.h:10:0,
                 from build/release/gen/cassandra_constants.cpp:7:
build/release/gen/cassandra_types.h:14:26: fatal error: thrift/TBase.h: No such file or directory
 #include <thrift/TBase.h>
                          ^
compilation terminated.
[2/238] CXX build/release/gen/Cassandra.o
FAILED: build/release/gen/Cassandra.o 
g++ -MD -MT build/release/gen/Cassandra.o -MF build/release/gen/Cassandra.o.d -std=gnu++1y -g  -Wall -Werror -fvisibility=hidden -pthread -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar/fmt -I/home/kyz/MEGA/github.com-kokizzu/scylla/seastar/build/release/gen  -march=nehalem -Ifmt -DBOOST_TEST_DYN_LINK -Wno-overloaded-virtual -DFMT_HEADER_ONLY -DHAVE_HWLOC -DHAVE_NUMA -DHAVE_LZ4_COMPRESS_DEFAULT  -O2 -DBOOST_TEST_DYN_LINK  -Wno-maybe-uninitialized -DHAVE_LIBSYSTEMD=1 -I. -I build/release/gen -I seastar -I seastar/build/release/gen -c -o build/release/gen/Cassandra.o build/release/gen/Cassandra.cpp
In file included from build/release/gen/Cassandra.cpp:7:0:
build/release/gen/Cassandra.h:17:52: fatal error: thrift/async/TConcurrentClientSyncInfo.h: No such file or directory
 #include <thrift/async/TConcurrentClientSyncInfo.h>
                                                    ^
compilation terminated.
ninja: build stopped: subcommand failed.

There are no thrift/TBase.h on 0.9.1

drwxr-xr-x  10 root root  4096 Feb 27 17:12 ./
drwxr-xr-x 595 root root 36864 Feb 27 16:53 ../
drwxr-xr-x   2 root root  4096 Feb 27 17:12 async/
drwxr-xr-x   6 root root  4096 Feb 27 17:12 c_glib/
drwxr-xr-x   2 root root  4096 Feb 27 17:12 concurrency/
-rw-r--r--   1 root root 12748 Nov  3  2014 config.h
-rw-r--r--   1 root root  4051 Nov  3  2014 cxxfunctional.h
drwxr-xr-x   2 root root  4096 Feb 27 17:12 processor/
drwxr-xr-x   2 root root  4096 Feb 27 17:12 protocol/
drwxr-xr-x   2 root root  4096 Feb 27 17:12 qt/
drwxr-xr-x   2 root root  4096 Feb 27 17:12 server/
-rw-r--r--   1 root root  3635 Nov  3  2014 TApplicationException.h
-rw-r--r--   1 root root  5158 Nov  3  2014 TDispatchProcessor.h
-rw-r--r--   1 root root   896 Nov  3  2014 thrift-config.h
-rw-r--r--   1 root root  5327 Nov  3  2014 Thrift.h
-rw-r--r--   1 root root  8124 Nov  3  2014 TLogging.h
-rw-r--r--   1 root root  6852 Nov  3  2014 TProcessor.h
drwxr-xr-x   2 root root  4096 Feb 27 17:12 transport/
-rw-r--r--   1 root root  2813 Nov  3  2014 TReflectionLocal.h

or 0.9.2

drwxr-xr-x  10 root root  4096 Feb 27 17:14 ./
drwxr-xr-x 595 root root 36864 Feb 27 16:53 ../
drwxr-xr-x   2 root root  4096 Feb 27 17:14 async/
drwxr-xr-x   6 root root  4096 Feb 27 17:14 c_glib/
drwxr-xr-x   2 root root  4096 Feb 27 17:14 concurrency/
-rw-r--r--   1 root root 13283 Dec 27  2014 config.h
-rw-r--r--   1 root root  4051 Dec 27  2014 cxxfunctional.h
drwxr-xr-x   2 root root  4096 Feb 27 17:14 processor/
drwxr-xr-x   2 root root  4096 Feb 27 17:14 protocol/
drwxr-xr-x   2 root root  4096 Feb 27 17:14 qt/
drwxr-xr-x   2 root root  4096 Feb 27 17:14 server/
-rw-r--r--   1 root root  3635 Dec 27  2014 TApplicationException.h
-rw-r--r--   1 root root  5158 Dec 27  2014 TDispatchProcessor.h
-rw-r--r--   1 root root   896 Dec 27  2014 thrift-config.h
-rw-r--r--   1 root root  5383 Dec 27  2014 Thrift.h
-rw-r--r--   1 root root  8124 Dec 27  2014 TLogging.h
-rw-r--r--   1 root root  6852 Dec 27  2014 TProcessor.h
drwxr-xr-x   2 root root  4096 Feb 27 17:14 transport/
-rw-r--r--   1 root root  2813 Dec 27  2014 TReflectionLocal.h
-rw-r--r--   1 root root  2336 Dec 27  2014 TToString.h

or 0.9.3:

drwxr-xr-x  10 root root  4096 Feb 27 17:15 ./
drwxr-xr-x 595 root root 36864 Feb 27 16:53 ../
drwxr-xr-x   2 root root  4096 Feb 27 17:15 async/
drwxr-xr-x   6 root root  4096 Feb 27 17:15 c_glib/
drwxr-xr-x   2 root root  4096 Feb 27 17:15 concurrency/
-rw-r--r--   1 root root 13702 Mar  5  2016 config.h
-rw-r--r--   1 root root  4072 Mar  5  2016 cxxfunctional.h
drwxr-xr-x   2 root root  4096 Feb 27 17:15 processor/
drwxr-xr-x   2 root root  4096 Feb 27 17:15 protocol/
drwxr-xr-x   2 root root  4096 Feb 27 17:15 qt/
drwxr-xr-x   2 root root  4096 Feb 27 17:15 server/
-rw-r--r--   1 root root  3527 Mar  5  2016 TApplicationException.h
-rw-r--r--   1 root root  5163 Mar  5  2016 TDispatchProcessor.h
-rw-r--r--   1 root root   894 Mar  5  2016 thrift-config.h
-rw-r--r--   1 root root  3345 Mar  5  2016 Thrift.h
-rw-r--r--   1 root root 10429 Mar  5  2016 TLogging.h
-rw-r--r--   1 root root  1825 Mar  5  2016 TOutput.h
-rw-r--r--   1 root root  6786 Mar  5  2016 TProcessor.h
drwxr-xr-x   2 root root  4096 Feb 27 17:15 transport/
-rw-r--r--   1 root root  2336 Mar  5  2016 TToString.h

but there are on 0.10.0

drwxr-xr-x  14 root root  4096 Feb 27 17:16 ./
drwxr-xr-x 595 root root 36864 Feb 27 16:53 ../
drwxr-xr-x   2 root root  4096 Feb 27 17:16 async/
drwxr-xr-x   6 root root  4096 Feb 27 17:16 c_glib/
-rw-r--r--   1 root root  1274 Jan  6 07:09 common.h
drwxr-xr-x   2 root root  4096 Feb 27 17:16 concurrency/
-rw-r--r--   1 root root 13705 Jan  6 07:09 config.h
-rw-r--r--   1 root root  4188 Jan  6 07:09 cxxfunctional.h
drwxr-xr-x   2 root root  4096 Jan  6 07:09 generate/
-rw-r--r--   1 root root  3880 Jan  6 07:09 globals.h
-rw-r--r--   1 root root  1223 Jan  6 07:09 logging.h
-rw-r--r--   1 root root  2825 Jan  6 07:09 main.h
drwxr-xr-x   2 root root  4096 Jan  6 07:09 parse/
-rw-r--r--   1 root root  1273 Jan  6 07:09 platform.h
drwxr-xr-x   2 root root  4096 Jan  6 07:09 plugin/
drwxr-xr-x   2 root root  4096 Feb 27 17:16 processor/
drwxr-xr-x   2 root root  4096 Feb 27 17:16 protocol/
drwxr-xr-x   2 root root  4096 Feb 27 17:16 qt/
drwxr-xr-x   2 root root  4096 Feb 27 17:16 server/
-rw-r--r--   1 root root  3533 Jan  6 07:09 TApplicationException.h
-rw-r--r--   1 root root  1190 Jan  6 07:09 TBase.h
-rw-r--r--   1 root root  5163 Jan  6 07:09 TDispatchProcessor.h
-rw-r--r--   1 root root   894 Jan  6 07:09 thrift-config.h
-rw-r--r--   1 root root  3345 Jan  6 07:09 Thrift.h
-rw-r--r--   1 root root 10429 Jan  6 07:09 TLogging.h
-rw-r--r--   1 root root  1825 Jan  6 07:09 TOutput.h
-rw-r--r--   1 root root  6792 Jan  6 07:09 TProcessor.h
drwxr-xr-x   2 root root  4096 Feb 27 17:16 transport/
-rw-r--r--   1 root root  2336 Jan  6 07:09 TToString.h
-rw-r--r--   1 root root  1000 Jan  6 07:09 version.h
drwxr-xr-x   2 root root  4096 Jan  6 07:09 windows/

kokizzu on 27 Feb 2017

Update: apparently I should reconfigure after installing previous version of thrift, now missing :

# clone scylla 1.6.1
git clone https://github.com/scylladb/scylla.git
cd scylla
git checkout tags/scylla-1.6.0

# use old thrift
wget -c 'http://ala.seblu.net/packages/t/thrift/thrift-0.9.1-3-x86_64.pkg.tar.xz'
pacman -U thrift-0.9.1-3-x86_64.pkg.tar.xz

# install deps 
git submodule init
git submodule update --init --recursive
yaourt -S libantlr3c libyaml lz4 zlib snappy jsoncpp gnutls ninja ragel libaio crypto++ xfsprogs numactl hwloc libpciaccess libxml2 python-pyparsing lksctp-tools protobuf libunwind

# install antlr3 cpp headers
pushd .
git clone --depth 1 https://github.com/antlr/antlr3.git
cd antlr3
sudo cp include/* /usr/include/
popd 

# build
./configure.py --mode=release --with=scylla --disable-xen --static-stdc++ --static-thrift --static-boost --static
ninja build/release/scylla -j2

[4/233] NINJA seastar/build/release/libseastar.a seastar/build/release/apps/iotune/i...se/gen/http/request_parser.hh seastar/build/release/gen/http/http_response_parser.hh 
FAILED: seastar/build/release/libseastar.a seastar/build/release/apps/iotune/iotune seastar/build/release/gen/http/request_parser.hh seastar/build/release/gen/http/http_re
sponse_parser.hh  
/usr/bin/ninja -C seastar build/release/libseastar.a build/release/apps/iotune/iotune build/release/gen/http/request_parser.hh build/release/gen/http/http_response_parser.
hh 
ninja: Entering directory `seastar' 
[1/49] CXX build/release/core/systemwide_memory_barrier.o 
[2/49] CXX build/release/http/reply.o 
[3/49] CXX build/release/http/mime_types.o 
[4/49] CXX build/release/http/api_docs.o 
[5/49] CXX build/release/core/reactor.o 
FAILED: build/release/core/reactor.o  
g++ -MMD -MT build/release/core/reactor.o -MF build/release/core/reactor.o.d -O2 -I build/release/gen -std=gnu++1y -g  -Wall -Werror -fvisibility=hidden -pthread -I. -U_FO
RTIFY_SOURCE  -march=nehalem -Ifmt -Wno-overloaded-virtual -DFMT_HEADER_ONLY -DNO_EXCEPTION_INTERCEPT -DHAVE_HWLOC -DHAVE_NUMA -DHAVE_LZ4_COMPRESS_DEFAULT -c -o build/rele
ase/core/reactor.o core/reactor.cc 
core/reactor.cc:85:21: fatal error: sys/sdt.h: No such file or directory 
#include <sys/sdt.h> 
                    ^ 
compilation terminated. 
[6/49] CXX build/release/net/proxy.o 
[7/49] CXX build/release/net/ip_checksum.o 
[8/49] CXX build/release/net/udp.o 
[9/49] CXX build/release/net/dhcp.o 
[10/49] CXX build/release/core/fstream.o 
[11/49] CXX build/release/net/virtio.o 
[12/49] CXX build/release/core/scollectd.o 
[13/49] CXX build/release/net/tcp.o 
[14/49] CXX build/release/net/tls.o 
ninja: build stopped: subcommand failed. 
[5/233] CXX build/release/gen/Cassandra.o 
ninja: build stopped: subcommand failed.

What's the required library for sdt.h?

kokizzu on 28 Feb 2017

Nevermind, found it here: https://github.com/scylladb/seastar/issues/223
It's on systemtap-git package on AUR

kokizzu on 28 Feb 2017

Complete solution, note that cannot statically build thrift because ArchLinux package does not provide the libthrift.a file:

# clone scylla 1.6.1
git clone https://github.com/scylladb/scylla.git
cd scylla
git checkout tags/scylla-1.6.0

# use old thrift
wget -c 'http://ala.seblu.net/packages/t/thrift/thrift-0.9.1-3-x86_64.pkg.tar.xz'
pacFileInstall thrift-0.9.1-3-x86_64.pkg.tar.xz

# install deps 
git submodule init
git submodule update --init --recursive
pacInstall libantlr3c libyaml yaml-cpp lz4 zlib snappy jsoncpp gnutls ninja ragel libaio crypto++ xfsprogs numactl hwloc libpciaccess libxml2 python-pyparsing lksctp-tools protobuf libunwind systemtap-git

# install antlr3 cpp headers
pushd .
git clone --depth 1 https://github.com/antlr/antlr3.git
cd antlr3
sudo cp include/* /usr/include/
popd 

# build
./configure.py --mode=release --with=scylla --disable-xen --static-stdc++ --static-boost --static
ninja build/release/scylla -j7

kokizzu on 28 Feb 2017

Reasons to support ArchLinux: popularity, based on Distrowatch today:

Debian and Ubuntu-based:
1   Mint    2772
2   Debian  1807
3   Ubuntu  1419
6   Zorin   1095
7   elementary  1043
9   deepin  852

ArchLinux based:
4   Manjaro 1388
10  Antergos    801
12  Arch    728

RHEL and Fedora based:
8   Fedora  1029
11  CentOS  793

Misc:
5   openSUSE    1359
14  PCLinuxOS   623
13  Solus   692
15  ReactOS 549

ArchLinux based distro combined is the top-3

kokizzu on 1 Mar 2017

Hi

As an open source project we'd take patches to support additional
distributions if they are not intrusive.
But in terms of official support - meaning we'll test every release and add
it to the support matrix - that's not in our horizon at the moment.
Officially supporting it adds a lot of work to our test environment, which
is time we'd rather spend testing and developing scylla features.

On Tue, Feb 28, 2017 at 10:26 PM, Kiswono Prayogo notifications@github.com
wrote:

Reasons to support ArchLinux: popularity, based on Distrowatch
https://distrowatch.com/ today:

Debian and Ubuntu-based:
1 Mint 2772
2 Debian 1807
3 Ubuntu 1419
6 Zorin 1095
7 elementary 1043
9 deepin 852

ArchLinux based:
4 Manjaro 1388
10 Antergos 801
12 Arch 728

RHEL and Fedora based:
8 Fedora 1029
11 CentOS 793

Misc:
5 openSUSE 1359
14 PCLinuxOS 623
13 Solus 692
15 ReactOS 549

ArchLinux based distro combined is the top-3

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla/issues/2118#issuecomment-283234417,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUNvWRSfysZCBL2_sEJUlAR7hp2aWAvks5rhOVugaJpZM4MM2Or
.

glommer on 1 Mar 2017

Hi,

I run Arch for my personal stuff and if you can whip up a PKGBUILD I am
happy to test it but I agree that the Scylla devs should focus on Scylla.

ons 1 mars 2017 kl 19:44 skrev Glauber Costa notifications@github.com:

Hi

As an open source project we'd take patches to support additional
distributions if they are not intrusive.
But in terms of official support - meaning we'll test every release and add
it to the support matrix - that's not in our horizon at the moment.
Officially supporting it adds a lot of work to our test environment, which
is time we'd rather spend testing and developing scylla features.

On Tue, Feb 28, 2017 at 10:26 PM, Kiswono Prayogo <
[email protected]>
wrote:

Reasons to support ArchLinux: popularity, based on Distrowatch
https://distrowatch.com/ today:

Debian and Ubuntu-based:
1 Mint 2772
2 Debian 1807
3 Ubuntu 1419
6 Zorin 1095
7 elementary 1043
9 deepin 852

ArchLinux based:
4 Manjaro 1388
10 Antergos 801
12 Arch 728

RHEL and Fedora based:
8 Fedora 1029
11 CentOS 793

Misc:
5 openSUSE 1359
14 PCLinuxOS 623
13 Solus 692
15 ReactOS 549

ArchLinux based distro combined is the top-3

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla/issues/2118#issuecomment-283234417,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AAUNvWRSfysZCBL2_sEJUlAR7hp2aWAvks5rhOVugaJpZM4MM2Or

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla/issues/2118#issuecomment-283430796,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKcU1NlMEXAC7ufbAs87F4jJsUwm0jfks5rhbyHgaJpZM4MM2Or
.

dahankzter on 1 Mar 2017

Hi, my friend and i created an aur package (similar to lauchpad on ubuntu), to help others compile and install scylladb:

https://aur.archlinux.org/packages/scylla/

kokizzu on 13 Mar 2017

i don't know why, using latest git checkouting 1.6.1 or 1.6.2 it no longer compilable.
Maybe the code no longer compatible with latest boost?

http://pastebin.com/pNMHwEfS

kokizzu on 13 Mar 2017

It's not an issue with Boost, it's an issue with the fmt submodule:

/home/arch/src/scylla/seastar/fmt/fmt/format.h:1856:20: error: expected unqualified-id before numeric constant
     const unsigned CHAR_WIDTH = 1;

The error message suggests that the code is trying to define a variable with the name CHAR_WIDTH that's already taken by a macro. And indeed, it seems to have been fixed in fmt upstream: https://github.com/fmtlib/fmt/commit/abbefd71666055daac9e14e78262620f9e845850

penberg on 15 Mar 2017

its currently not on our workplan - nevertheless if you have been able to advance, it would be great if you can send patches - it will help once we do add it

slivne on 30 Apr 2017

The development version of ScyllaDB is now available from Arch User Repository:
https://aur.archlinux.org/packages/scylla-git/

The stable release package will be added once Scylla 2.0 is out, which is going to happen soon.
There are some problems with packaging Scylla 1.7.x as it depends on earlier versions of Thrift not easily available for ArchLinux (as you have correctly pointed out, this is mentioned in #841).

DPDK support for Arch Linux packages is in progress.

argenet on 17 Aug 2017

🎉2

FYI, ScyllaDB 2.0 is out today and now can be installed from AUR.

/cc @kokizzu

argenet on 6 Oct 2017

👍2

It takes a bunch of time to install but it seems to work! :)

dahankzter on 12 Apr 2018

👍1

Scylla needs official arch-linux package. Is there an attempt to do this?

morphinz on 26 Jul 2018

Hi @morphinz,

As of today, there are no plans to make an official Scylla package for Arch Linux.
There are AUR packages, such as scylla, scylla-jmx and scylla-tools that are updated periodically as new versions of Scylla arise.

In order to get some package into official Arch repos, its maintainer needs to have advanced permissions which noone from Scylla team possesses today. If you are able to facilitate this, help would be much appreciated.

argenet on 6 Aug 2018

Scylla: Add ArchLinux Support

Most helpful comment

All 18 comments

Related issues