tikv terminated with signal 4, Illegal instruction.

[abo]

Please answer these questions before submitting your issue. Thanks!

1. What version of Rust are you using (rustc --version)?

   bin/tikv-server --version
   TiKV
   Release Version: 2.0.0-rc.3
   Git Commit Hash: a6c08bcdbe747bfd2bc17048f991ed763db29bd7
   Git Commit Branch: master
   UTC Build Time: 2018-03-24 07:51:00
   Rust Version: 1.25.0-nightly (73ac5d6a8 2018-01-11)

2. What operating system and processor architecture are you using?

   Linux tidb03 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

   vendor_id : AuthenticAMD
   cpu family : 16
   model : 6
   model name : AMD Athlon(tm) II X2 250 Processor
   stepping : 3
   microcode : 0x10000c8
   cpu MHz : 800.000
   cache size : 1024 KB
   physical id : 0
   siblings : 2
   core id : 0
   cpu cores : 2
   apicid : 0
   initial apicid : 0
   fpu : yes
   fpu_exception : yes
   cpuid level : 5
   wp : yes
   flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc art rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt hw_pstate npt lbrv svm_lock nrip_save
   bogomips : 6026.96
   TLB size : 1024 4K pages
   clflush size : 64
   cache_alignment : 64
   address sizes : 48 bits physical, 48 bits virtual
   power management: ts ttp tm stc 100mhzsteps hwpstate

3. What did you do?
   If possible, provide a recipe for reproducing the error.
   A complete runnable program is good.

   download tidb from http://download.pingcap.org/tidb-latest-linux-amd64.tar.gz; start pd-server, tikv-server, tidb-server; start syncer;

4. What did you expect to see?

   tikv-server should not segment fault.

5. What did you see instead?

   tikv segment fault with a core dump.

   `Program terminated with signal 4, Illegal instruction.

   0 0x00007fe36a4b27e2 in rocksdb::DBImpl::MarkLogsSynced(unsigned long, bool, rocksdb::Status const&) ()

   (gdb) bt

   0 0x00007fe36a4b27e2 in rocksdb::DBImpl::MarkLogsSynced(unsigned long, bool, rocksdb::Status const&) ()

   1 0x00007fe36a4c89d5 in rocksdb::DBImpl::SyncClosedLogs(rocksdb::JobContext*) ()

   2 0x00007fe36a4ccc0a in rocksdb::DBImpl::FlushMemTableToOutputFile(rocksdb::ColumnFamilyData, rocksdb::MutableCFOptions const&, bool, rocksdb::JobContext, rocksdb::LogBuffer) ()

   3 0x00007fe36a4cd35a in rocksdb::DBImpl::BackgroundFlush(bool, rocksdb::JobContext, rocksdb::LogBuffer*) ()

   4 0x00007fe36a4d07a1 in rocksdb::DBImpl::BackgroundCallFlush() ()

   5 0x00007fe36a663401 in rocksdb::ThreadPoolImpl::Impl::BGThread(unsigned long) ()

   6 0x00007fe36a6635dd in rocksdb::ThreadPoolImpl::Impl::BGThreadWrapper(void*) ()

   7 0x00007fe36a61d740 in execute_native_thread_routine ()

   8 0x00007fe368a63e25 in start_thread () from /lib64/libpthread.so.0

   9 0x00007fe36857b34d in clone () from /lib64/libc.so.6`

[siddontang]

Hi @abo
Does the TiKV panic quickly when you start it or run for a time?

I see that you use AMD, maybe we introduce something only for Intel CPU. How do we build the release version, using make release or make unportable_release @iamxy

[abo]

run for a time.

p.s. another tikv instance works fine with processor:

vendor_id : AuthenticAMD
cpu family : 21
model : 16
model name : AMD A8-5600K APU with Radeon(tm) HD Graphics
stepping : 1
microcode : 0x6001119
cpu MHz : 1900.000
cache size : 2048 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 2
apicid : 19
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm topoext perfctr_core perfctr_nb arat cpb hw_pstate npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold bmi1
bogomips : 7186.38
TLB size : 1536 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro

[siddontang]

oh, PTAL @huachaohuang @zhangjinpeng1987

[breeswish]

Hi, our binary contains SSE 4.1 and SSE 4.2 instructions:

elfx86exts ~/Downloads/tidb-latest-linux-amd64/bin/tikv-server 
MODE64 (call)
CMOV (cmovbe)
SSE2 (pause)
SSE1 (movaps)
SSE3 (movddup)
SSE41 (pinsrq)
SSE42 (crc32)
BMI (tzcnt)

while your CPU only supports sse4a instruction set. that's why there are illegal instructions.

[abo]

thanks @breeswish , i will try to build tikv with ROCKSDB_SYS_SSE=0

[mark988]

./tikv-server --pd="127.0.0.1:2379" --data-dir=tikv --log-file=tikv.log
执行上面的指令，报错如下，
Illegal instruction (core dumped)

[breeswish]

@mark988 Which version are you using?

[mark988]

没看懂解决方案。