Scylla: Segfault backtracing through clock_gettime() via vdso
Installation details
Scylla version (or git commit hash): 4.3.rc0-0.20201028.bbef05ae3c with build-id adad9e1dae9eb9dbebab38bd5e7258d5c7360350
Cluster size: 6 db nodes
OS (RHEL/CentOS/Ubuntu/AWS AMI): ami-01cc969208fae7a3a
running longevity 5000 tables.
during nemesis called CorruptThenRebuild we do:
- get all non system KS and CF
- stop service on the target node
- choose 5 files and
rm -f <file> - start Scylla service (and wait for service to be UP)
during step 4 we got a coredump:
Job for scylla-server.service failed because a fatal signal was delivered to the control process. See "systemctl status scylla-server.service" and "journalctl -xe" for details.
corefile_url=https://storage.cloud.google.com/upload.scylladb.com/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000.gz
backtrace= PID: 120199 (scylla)
UID: 996 (scylla)
GID: 1001 (scylla)
Signal: 11 (SEGV)
Timestamp: Tue 2020-11-10 20:47:29 UTC (4min 12s ago)
Command Line: /usr/bin/scylla --blocked-reactor-notify-ms 500 --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --network-stack posix --io-properties-file=/etc/scylla.d/io_properties.yaml --cpuset 1-15,17-31 --lock-memory=1
Executable: /opt/scylladb/libexec/scylla
Control Group: /
Boot ID: 0ca3c8b45fd14c26b6ecc7ada161f6db
Machine ID: ec239da046efe33c7665d4508a7d0a61
Hostname: longevity-5000-tables-4-3-db-node-0924e5e2-3
Coredump: /var/lib/systemd/coredump/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000
Message: Process 120199 (scylla) of user 996 dumped core.
Stack trace of thread 120214:
#0 0x00007f3dbaef08ff _Unwind_Backtrace (libgcc_s.so.1)
#1 0x00007f3dbae23df6 __backtrace (libc.so.6)
#2 0x0000000002b5af63 _ZN7seastar9backtraceIZNS_16backtrace_buffer16append_backtraceEvEUlNS_5frameEE_EEvOT_ (scylla)
#3 0x0000000002b5b5f1 _ZN7seastar16backtrace_buffer16append_backtraceEv (scylla)
#4 0x0000000002b5b896 print_with_backtrace (scylla)
#5 0x0000000002b5b931 _ZZN7seastar30install_oneshot_signal_handlerILi11EXadL_ZNS_L14sigsegv_actionEvEEEEvvENUliP9siginfo_tPvE_4_FUNEiS2_S3_ (scylla)
#6 0x00007f3dbbaa5a90 __restore_rt (libpthread.so.0)
#7 0x00007f3dbaef08ff _Unwind_Backtrace (libgcc_s.so.1)
#8 0x00007f3dbae23df6 __backtrace (libc.so.6)
#9 0x0000000002b5af63 _ZN7seastar9backtraceIZNS_16backtrace_buffer16append_backtraceEvEUlNS_5frameEE_EEvOT_ (scylla)
#10 0x0000000002b5b5f1 _ZN7seastar16backtrace_buffer16append_backtraceEv (scylla)
#11 0x0000000002b5bac2 _ZN7seastar8internal18cpu_stall_detector14generate_traceEv (scylla)
#12 0x0000000002b5bce8 _ZN7seastar8internal18cpu_stall_detector12maybe_reportEv (scylla)
#13 0x00007f3dbbaa5a90 __restore_rt (libpthread.so.0)
#14 0x00007ffc672e76d8 n/a (linux-vdso.so.1)
#15 0x00007ffc672e7982 __vdso_clock_gettime (linux-vdso.so.1)
Stack trace of thread 120199:
#0 0x0000000002b9c9e1 _ZN7seastar17smp_message_queue19flush_request_batchEv (scylla)
#1 0x0000000002b9ca56 _ZN7seastar7reactor10smp_pollfn4pollEv (scylla)
#2 0x0000000002b476fd _ZN7seastar7reactor9poll_onceEv (scylla)
#3 0x0000000002b98f79 _ZNKSt8functionIFbvEEclEv (scylla)
#4 0x0000000002aecfcf _ZN7seastar12app_template14run_deprecatedEiPPcOSt8functionIFvvEE (scylla)
#5 0x0000000002aeda5f _ZN7seastar12app_template3runEiPPcOSt8functionIFNS_6futureIiEEvEE (scylla)
#6 0x0000000000d4c993 main (scylla)
#7 0x00007f3dbad3c042 __libc_start_main (libc.so.6)
#8 0x0000000000c2d16e _start (scylla)
Stack trace of thread 120229:
#0 0x00007f3dbbaa49ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#4 0x00007f3dbba9a432 start_thread (libpthread.so.0)
#5 0x00007f3dbae16913 __clone (libc.so.6)
Stack trace of thread 120201:
#0 0x0000000000e6c490 _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7replaceEmmPKcm (scylla)
#1 0x0000000000e84e38 _ZN5tableC2EN7seastar13lw_shared_ptrIK6schemaEENS_6configEPN2db9commitlogER18compaction_managerR17cell_locker_statsR13cache_tracker (scylla)
#2 0x0000000000dd7d08 _ZN5tableC4EN7seastar13lw_shared_ptrIK6schemaEENS_6configERN2db9commitlogER18compaction_managerR17cell_locker_statsR13cache_tracker (scylla)
#3 0x0000000000ddb147 _ZN8database36add_column_family_and_make_directoryEN7seastar13lw_shared_ptrIK6schemaEE (scylla)
#4 0x0000000000ddbf8b operator()<view_ptr&> (scylla)
#5 0x0000000000ddc38c run_and_dispose (scylla)
#6 0x0000000002b593a8 _ZN7seastar7reactor9run_tasksERNS0_10task_queueE (scylla)
#7 0x0000000002b5965f _ZN7seastar7reactor14run_some_tasksEv (scylla)
#8 0x0000000002b98f46 _ZN7seastar7reactor14run_some_tasksEv (scylla)
#9 0x0000000002baa6fb _ZZN7seastar3smp9configureEN5boost15program_options13variables_mapENS_14reactor_configEENKUlvE1_clEv (scylla)
#10 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#11 0x00007f3dbba9a432 start_thread (libpthread.so.0)
#12 0x00007f3dbae16913 __clone (libc.so.6)
Stack trace of thread 120254:
#0 0x00007f3dbbaa49ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#4 0x00007f3dbba9a432 start_thread (libpthread.so.0)
#5 0x00007f3dbae16913 __clone (libc.so.6)
Stack trace of thread 120231:
#0 0x00007f3dbbaa49ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce
download_instructions=gsutil cp gs://upload.scylladb.com/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000.gz .
gunzip /var/lib/systemd/coredump/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.120199.1605041249000000.gz
DB node logs and SCT logs can be downloaded from:
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Log links for testrun with test id 0924e5e2-3404-4dd6-b498-8cbe68f030dd |
+-----------------+------------+-----------------------------------------------------------------------------------------------------------------------------+
| Date | Log type | Link |
+-----------------+------------+-----------------------------------------------------------------------------------------------------------------------------+
| 20201112_163045 | db-cluster | https://cloudius-jenkins-test.s3.amazonaws.com/0924e5e2-3404-4dd6-b498-8cbe68f030dd/20201112_163045/db-cluster-0924e5e2.zip |
| 20201112_163045 | sct-runner | https://cloudius-jenkins-test.s3.amazonaws.com/0924e5e2-3404-4dd6-b498-8cbe68f030dd/20201112_163045/sct-runner-0924e5e2.zip |
+-----------------+------------+-----------------------------------------------------------------------------------------------------------------------------+
for reference, the event SCT raised for the coredump (with the node's timestamp):
2020-11-10 20:56:19.519: (CoreDumpEvent Severity.ERROR)node=Node longevity-5000-tables-4-3-db-node-0924e5e2-3 [34.254.225.10 | 10.0.2.162] (seed: False)
fgelcer
All 39 comments
could it be somehow related to #7439?
or even #7482 ?
fgelcer
on 15 Nov 2020
there is another coredump in this test, that happened during ScyllaKill nemesis.. could it be related to previous one?
corefile_url=https://storage.cloud.google.com/upload.scylladb.com/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000.gz
backtrace= PID: 112456 (scylla)
UID: 996 (scylla)
GID: 1001 (scylla)
Signal: 11 (SEGV)
Timestamp: Wed 2020-11-11 01:10:44 UTC (4min 23s ago)
Command Line: /usr/bin/scylla --blocked-reactor-notify-ms 500 --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --network-stack posix --io-properties-file=/etc/scylla.d/io_properties.yaml --cpuset 1-15,17-31 --lock-memory=1
Executable: /opt/scylladb/libexec/scylla
Control Group: /
Boot ID: 0ca3c8b45fd14c26b6ecc7ada161f6db
Machine ID: ec239da046efe33c7665d4508a7d0a61
Hostname: longevity-5000-tables-4-3-db-node-0924e5e2-3
Coredump: /var/lib/systemd/coredump/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000
Message: Process 112456 (scylla) of user 996 dumped core.
Stack trace of thread 112474:
#0 0x00007feabe9358ff _Unwind_Backtrace (libgcc_s.so.1)
#1 0x00007feabe868df6 __backtrace (libc.so.6)
#2 0x0000000002b5af63 _ZN7seastar9backtraceIZNS_16backtrace_buffer16append_backtraceEvEUlNS_5frameEE_EEvOT_ (scylla)
#3 0x0000000002b5b5f1 _ZN7seastar16backtrace_buffer16append_backtraceEv (scylla)
#4 0x0000000002b5b896 print_with_backtrace (scylla)
#5 0x0000000002b5b931 _ZZN7seastar30install_oneshot_signal_handlerILi11EXadL_ZNS_L14sigsegv_actionEvEEEEvvENUliP9siginfo_tPvE_4_FUNEiS2_S3_ (scylla)
#6 0x00007feabf4eaa90 __restore_rt (libpthread.so.0)
#7 0x00007feabe9358ff _Unwind_Backtrace (libgcc_s.so.1)
#8 0x00007feabe868df6 __backtrace (libc.so.6)
#9 0x0000000002b5af63 _ZN7seastar9backtraceIZNS_16backtrace_buffer16append_backtraceEvEUlNS_5frameEE_EEvOT_ (scylla)
#10 0x0000000002b5b5f1 _ZN7seastar16backtrace_buffer16append_backtraceEv (scylla)
#11 0x0000000002b5bac2 _ZN7seastar8internal18cpu_stall_detector14generate_traceEv (scylla)
#12 0x0000000002b5bce8 _ZN7seastar8internal18cpu_stall_detector12maybe_reportEv (scylla)
#13 0x00007feabf4eaa90 __restore_rt (libpthread.so.0)
#14 0x00007fffc63576d8 n/a (linux-vdso.so.1)
#15 0x00007fffc6357982 __vdso_clock_gettime (linux-vdso.so.1)
Stack trace of thread 112458:
#0 0x00007feabe85637d syscall (libc.so.6)
#1 0x0000000002dcf6dd _ZN7seastar8internal13io_pgeteventsEmllPNS0_9linux_abi8io_eventEPK8timespecPK10__sigset_tb (scylla)
#2 0x0000000002dcb994 _ZN7seastar19reactor_backend_aio12await_eventsEiPK10__sigset_t (scylla)
#3 0x0000000002dcbb55 _ZN7seastar19reactor_backend_aio23wait_and_process_eventsEPK10__sigset_t (scylla)
#4 0x0000000002b59860 _ZN7seastar7reactor5sleepEv (scylla)
#5 0x0000000002b9941a _ZN7seastar7reactor3runEv (scylla)
#6 0x0000000002baa6fb _ZZN7seastar3smp9configureEN5boost15program_options13variables_mapENS_14reactor_configEENKUlvE1_clEv (scylla)
#7 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#8 0x00007feabf4df432 start_thread (libpthread.so.0)
#9 0x00007feabe85b913 __clone (libc.so.6)
Stack trace of thread 112459:
#0 0x00007feabe85637d syscall (libc.so.6)
#1 0x0000000002dcf6dd _ZN7seastar8internal13io_pgeteventsEmllPNS0_9linux_abi8io_eventEPK8timespecPK10__sigset_tb (scylla)
#2 0x0000000002dcb994 _ZN7seastar19reactor_backend_aio12await_eventsEiPK10__sigset_t (scylla)
#3 0x0000000002dcbb55 _ZN7seastar19reactor_backend_aio23wait_and_process_eventsEPK10__sigset_t (scylla)
#4 0x0000000002b59860 _ZN7seastar7reactor5sleepEv (scylla)
#5 0x0000000002b9941a _ZN7seastar7reactor3runEv (scylla)
#6 0x0000000002baa6fb _ZZN7seastar3smp9configureEN5boost15program_options13variables_mapENS_14reactor_configEENKUlvE1_clEv (scylla)
#7 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#8 0x00007feabf4df432 start_thread (libpthread.so.0)
#9 0x00007feabe85b913 __clone (libc.so.6)
Stack trace of thread 112509:
#0 0x00007feabf4e99ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#4 0x00007feabf4df432 start_thread (libpthread.so.0)
#5 0x00007feabe85b913 __clone (libc.so.6)
Stack trace of thread 112490:
#0 0x00007feabf4e99ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#4 0x00007feabf4df432 start_thread (libpthread.so.0)
#5 0x00007feabe85b913 __clone (libc.so.6)
Stack trace of thread 112491:
#0 0x00007feabf4e99ac read (libpthread.so.0)
#1 0x0000000002dc9a27 _ZN7seastar11thread_pool4workENS_13basic_sstringIcjLj15ELb1EEE (scylla)
#2 0x0000000002dc9c98 operator() (scylla)
#3 0x0000000002b232ce _ZNKSt8functionIFvvEEclEv (scylla)
#4 0x00007feabf4df432 start_thread (libpthread.so.0)
#5 0x00007feabe85b913 __clone (libc.so.6)
Stack trace of thread 112460:
#0 0x00007feabe85637d syscall (libc.so.6)
#1 0x000000
download_instructions=gsutil cp gs://upload.scylladb.com/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000.gz .
gunzip /var/lib/systemd/coredump/core.scylla.996.0ca3c8b45fd14c26b6ecc7ada161f6db.112456.1605057044000000.gz
What kernel are you running?
It looks like __vdso_clock_gettime doesn't support stack unwinding correctly. Similar to https://gitlab.com/gnutls/gnutls/-/issues/1111.
avikivity
on 15 Nov 2020
What kernel are you running?
It looks like __vdso_clock_gettime doesn't support stack unwinding correctly. Similar to https://gitlab.com/gnutls/gnutls/-/issues/1111.
Linux version 5.4.68-34.125.amzn2.x86_64 (mockbuild@ip-10-0-1-132) (gcc version 7.3.1 20180712 (Red Hat 7.3.1-10) (GCC)) #1 SMP Fri Oct 16 20:58:44 UTC 2020
fgelcer
on 15 Nov 2020
That should be stable enough.
avikivity
on 15 Nov 2020
That should be stable enough.
can you confirm/deny both coredumps are similar?
fgelcer
on 15 Nov 2020
Yes, it's the same problem.
avikivity
on 15 Nov 2020
/cc @tgrabiec
avikivity
on 15 Nov 2020
vdso-clock_gettime-backtrace.tar.gz
Please test the attached program on the AMI with that kernel.
You can compile it with
gcc -g vdso-clock_gettime-backtrace.c -lrt -o vdso-clock_gettime-backtrace
could it be somehow related to #7439?
or even #7482 ?
no
bhalevy
on 15 Nov 2020
vdso-clock_gettime-backtrace.tar.gz
Please test the attached program on the AMI with that kernel.
You can compile it with
gcc -g vdso-clock_gettime-backtrace.c -lrt -o vdso-clock_gettime-backtrace
@roydahan , should i re-run the test and keep the nodes to run this program?
fgelcer
on 16 Nov 2020
Run it as a "Reproducer" only with one of these nemesis (better ScyllaKill).
Then, try yourself or provide developers the details of the cluster.
roydahan
on 16 Nov 2020
@Avi Kivity avi@scylladb.com whouldn't it be enough to boot the ami on
the same instance and run the test program ?
Do we need to push the machine to a state ?
On Mon, Nov 16, 2020 at 12:35 PM Roy Dahan notifications@github.com wrote:
Run it as a "Reproducer" only with one of these nemesis (better
ScyllaKill).
Then, try yourself or provide developers the details of the cluster.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla/issues/7607#issuecomment-727892508,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA2OCCASJBWIFOTFNBLETRLSQD6ANANCNFSM4TWEIWLQ
.
slivne
on 16 Nov 2020
It's enough to just run that program, no need to start scylla or anything.
avikivity
on 16 Nov 2020
It just has to be exactly the same kernel.
avikivity
on 16 Nov 2020
And, please let me know what the kernel version is.
avikivity
on 16 Nov 2020
ok, easier.
@fgelcer FYI.
roydahan
on 16 Nov 2020
@avikivity , i have a machine (built with the same ami as the test did):
[email protected]
uname -r
5.4.68-34.125.amzn2.x86_64
how do i run the program?
[scyllaadm@ip-10-0-1-190 ~]$ ./vdso-clock_gettime-backtrace
Segmentation fault (core dumped)
[scyllaadm@ip-10-0-1-190 ~]$ ll
total 24
-rwxrwxr-x 1 scyllaadm scyllaadm 13072 Nov 16 11:51 vdso-clock_gettime-backtrace
-rw-rw-r-- 1 scyllaadm scyllaadm 1050 Nov 15 13:43 vdso-clock_gettime-backtrace.c
-rw-rw-r-- 1 scyllaadm scyllaadm 631 Nov 16 11:50 vdso-clock_gettime-backtrace.tar.gz
should i pass flags to it?
fgelcer
on 16 Nov 2020
@fgelcer the crash indicates we reproduced the problem and the bug is in the kernel, not Scylla.
avikivity
on 16 Nov 2020
What instance type did you use?
avikivity
on 16 Nov 2020
Never mind, reproduced on i3.
avikivity
on 16 Nov 2020
Also reproduced with 5.4.74-36.135.amzn2.x86_64
avikivity
on 16 Nov 2020
Never mind, reproduced on i3.
sorry for the delayed response...
i will terminate that instance.
fgelcer
on 16 Nov 2020
Does not reproduce on 4.14.200-155.322.amzn2.x86_64
avikivity
on 16 Nov 2020
Does not reproduce with 5.9.8-1.el7.elrepo.x86_64
avikivity
on 16 Nov 2020
I suspect the fix is
commit b91c8c42ffdd5c983923edb38b3c3e112bfe6263
Author: Christophe Leroy <[email protected]>
Date: Tue Apr 28 13:16:53 2020 +0000
lib/vdso: Force inlining of __cvdso_clock_gettime_common()
When adding gettime64() to a 32 bit architecture (namely powerpc/32)
it has been noticed that GCC doesn't inline anymore
__cvdso_clock_gettime_common() because it is called twice
(Once by __cvdso_clock_gettime() and once by
__cvdso_clock_gettime32).
This has the effect of seriously degrading the performance:
Before the implementation of gettime64(), gettime() runs in:
clock-gettime-monotonic-raw: vdso: 1003 nsec/call
clock-gettime-monotonic-coarse: vdso: 592 nsec/call
clock-gettime-monotonic: vdso: 942 nsec/call
When adding a gettime64() entry point, the standard gettime()
performance is degraded by 30% to 50%:
clock-gettime-monotonic-raw: vdso: 1300 nsec/call
clock-gettime-monotonic-coarse: vdso: 900 nsec/call
clock-gettime-monotonic: vdso: 1232 nsec/call
Adding __always_inline() to __cvdso_clock_gettime_common() regains the
original performance.
In terms of code size, the inlining increases the code size by only 176
bytes. This is in the noise for a kernel image.
Signed-off-by: Christophe Leroy <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lkml.kernel.org/r/1ab6a62c356c3bec35d1623563ef9c636205bcda.1588079622.git.christophe.leroy@c-s.fr
(torvalds/linux@b91c8c42ffdd5c983923edb38b3c3e112bfe6263)
avikivity
on 16 Nov 2020
Did not reproduce with old kernels on a local virtual machine. So it's likely xenclock specific.
avikivity
on 16 Nov 2020
@amoskong how do you change kernels on AWS? The tricks with grub2-set-default did not work for me.
avikivity
on 16 Nov 2020
@amoskong how do you change kernels on AWS? The tricks with grub2-set-default did not work for me.
- Install new kernel, check the new item is added to /boot/grub2/grub.cfg
- change GRUB_DEFAULT to 0 in /etc/default/grub
- sudo grub2-mkconfig -o /boot/grub2/grub.cfg
- reboot
amoskong
on 18 Nov 2020
Wasn't able to make it work. Used kexec instead.
avikivity
on 22 Nov 2020
However, my attempts to get a vanilla kernel to boot on AWS failed. So bisecting will be difficult.
avikivity
on 22 Nov 2020
@avikivity whats the next step - do you want to move to CentOS8 / back to Centos7 ?
slivne
on 25 Nov 2020
Ok - so we are moving back to CentOS7 -like we did in 4.2 - and move master to CentOS8 and work on using amazon kernel (4.14.200-155.322.amzn2.x86_64)
slivne
on 25 Nov 2020
AWS forum link: https://forums.aws.amazon.com/thread.jspa?threadID=331605
avikivity
on 25 Nov 2020
@slivne @avikivity Created PR to revert Amazon Linux 2. https://github.com/scylladb/scylla-machine-image/pull/85
syuu1228
on 2 Dec 2020
@fgelcer 4.3 image will have the change back to centos7 - lets verify the issue is gone.
slivne
on 3 Dec 2020
@fgelcer 4.3 image will have the change back to centos7 - lets verify the issue is gone.
check that latest 4.3.rc2 has a Centos 7 running underneath and tested that it doesn't have the issue as above
ami id - ami-08aaddc100281dc7d (on eu-north-1)
results:
[scyllaadm@ip-172-31-1-95 ~]$ uname -r
5.9.14-1.el7.elrepo.x86_64
[scyllaadm@ip-172-31-1-95 ~]$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
and the result of the script @avikivity provided is:
[scyllaadm@ip-172-31-1-95 ~]$ gcc -g vdso-clock_gettime-backtrace.c -lrt -o vdso-clock_gettime-backtrace
[scyllaadm@ip-172-31-1-95 ~]$ echo $?
0
@fgelcer 4.3 image will have the change back to centos7 - lets verify the issue is gone.
check that latest 4.3.rc2 has a Centos 7 running underneath and tested that it doesn't have the issue as above
ami id - ami-08aaddc100281dc7d (on eu-north-1)
results:[scyllaadm@ip-172-31-1-95 ~]$ uname -r 5.9.14-1.el7.elrepo.x86_64 [scyllaadm@ip-172-31-1-95 ~]$ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"and the result of the script @avikivity provided is:
[scyllaadm@ip-172-31-1-95 ~]$ gcc -g vdso-clock_gettime-backtrace.c -lrt -o vdso-clock_gettime-backtrace [scyllaadm@ip-172-31-1-95 ~]$ echo $? 0
@slivne @roydahan , FYI
fgelcer
on 15 Dec 2020
thanks @fgelcer
slivne
on 20 Dec 2020
Related issues
avikivity
·
4Comments
Felix-zhoux
·
3Comments
amoskong
·
6Comments
amoskong
·
5Comments
pdziepak
·
6Comments