Couchdb: arch linux {os_process_error,{exit_status,139}}
Hi there,
I installed couchdb (2.3.0) via pacman on archlinux and played along with it.
My problem is, that I cannot do certain actions, e.g. to create new _design/-Documents or Views.
I get this error while trying to create a new view:
{os_process_error,{exit_status,139}}
I get this error on 3 different arch-machines (laptop, desktop, server)
I found a thread at apache.org blaming SELinux for this error: http://docs.couchdb.org/en/2.3.0/install/troubleshooting.html
but I don't know what to do on arch.
Any help welcome
:=)
Expected Behavior
A new View or _design-Document is created.
Current Behavior
crashes with error 139
Possible Solution
don't know
Steps to Reproduce (for bugs)
- install archlinux
- pacman -S couchdb 3.
curl -H 'Content-Type: application/json' -X PUT http://localhost:5984/mydb/_design/example -d ' {
"_id" : "_design/example",
"views" : {
"foo" : {
"map" : "function(doc){ emit(doc._id, doc._rev)}"
}
}
} '
- Get error 139
Your Environment
- Version used: 2.3.0
- Browser Name and version: Firefox 64.0
- Operating System and version (desktop or mobile): archlinux
produnis
All 20 comments
So ,this is how the log looks like:
Dez 31 16:18:06 Elrond couchdb[4172]: [info] 2018-12-31T15:18:06.479165Z [email protected] <0.223.0> -------- Apache CouchDB 2.3.0 is starting.
Dez 31 16:18:06 Elrond couchdb[4172]: [info] 2018-12-31T15:18:06.479807Z [email protected] <0.224.0> -------- Starting couch_sup
Dez 31 16:18:06 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:06.483586Z [email protected] <0.106.0> -------- config: [features] pluggable-storage-engines set to true for reason nil
Dez 31 16:18:06 Elrond couchdb[4172]: [info] 2018-12-31T15:18:06.520449Z [email protected] <0.223.0> -------- Apache CouchDB has started. Time to relax.
Dez 31 16:18:06 Elrond couchdb[4172]: [info] 2018-12-31T15:18:06.520558Z [email protected] <0.223.0> -------- Apache CouchDB has started on http://127.0.0.1:5986/
Dez 31 16:18:06 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:06.532090Z [email protected] <0.286.0> -------- rexi_server : started servers
Dez 31 16:18:06 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:06.533801Z [email protected] <0.291.0> -------- rexi_buffer : started servers
Dez 31 16:18:06 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:06.573703Z [email protected] <0.106.0> -------- config: [features] scheduler set to true for reason nil
Dez 31 16:18:11 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:11.562159Z [email protected] <0.369.0> -------- couch_replicator_clustering : cluster stable
Dez 31 16:18:11 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:11.568297Z [email protected] <0.421.0> -------- Started replicator db changes listener <0.521.0>
Dez 31 16:18:21 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:21.533217Z [email protected] <0.286.0> -------- rexi_server : cluster stable
Dez 31 16:18:21 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:21.533575Z [email protected] <0.291.0> -------- rexi_buffer : cluster stable
Dez 31 16:18:25 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:25.191992Z [email protected] <0.889.0> 69320659c9 127.0.0.1:5984 127.0.0.1 produnis GET /verifytestdb 200 ok 18
Dez 31 16:18:25 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:25.192104Z [email protected] <0.891.0> 072221a8ac 127.0.0.1:5984 127.0.0.1 produnis GET /verifytestdb_replicate 404 ok 7
Dez 31 16:18:25 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:25.231098Z [email protected] <0.889.0> c63494a27d 127.0.0.1:5984 127.0.0.1 produnis DELETE /verifytestdb 200 ok 28
Dez 31 16:18:25 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:25.541853Z [email protected] <0.889.0> 9cbcdc8d15 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb 201 ok 131
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.005692Z [email protected] <0.891.0> af7ce69780 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb/test_doc_1 201 ok 34
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.355983Z [email protected] <0.891.0> bb791918f4 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb/test_doc_1 201 ok 14
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.402736Z [email protected] <0.891.0> ba33f2a14e 127.0.0.1:5984 127.0.0.1 produnis DELETE /verifytestdb/test_doc_1?rev=2-d588d3e93ee155c5afffdf0247a2c5ef 200 ok 15
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.449777Z [email protected] <0.891.0> 87a28053ac 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb/test_doc_10 201 ok 28
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.459226Z [email protected] <0.890.0> 4f8b4807b5 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb/test_doc_20 201 ok 37
Dez 31 16:18:26 Elrond couchdb[4172]: [notice] 2018-12-31T15:18:26.477647Z [email protected] <0.892.0> d70366474f 127.0.0.1:5984 127.0.0.1 produnis PUT /verifytestdb/test_doc_30 201 ok 51
Dez 31 16:18:26 Elrond couchdb[4172]: [info] 2018-12-31T15:18:26.836534Z [email protected] <0.238.0> -------- couch_proc_manager <0.1072.0> died normal
Dez 31 16:18:26 Elrond couchdb[4172]: [error] 2018-12-31T15:18:26.836603Z [email protected] <0.1070.0> -------- OS Process Error <0.1072.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:27 Elrond couchdb[4172]: [info] 2018-12-31T15:18:27.098445Z [email protected] <0.238.0> -------- couch_proc_manager <0.1105.0> died normal
Dez 31 16:18:27 Elrond couchdb[4172]: [error] 2018-12-31T15:18:27.098559Z [email protected] <0.1070.0> -------- OS Process Error <0.1105.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:27 Elrond couchdb[4172]: [info] 2018-12-31T15:18:27.321895Z [email protected] <0.238.0> -------- couch_proc_manager <0.1108.0> died normal
Dez 31 16:18:27 Elrond couchdb[4172]: [error] 2018-12-31T15:18:27.322010Z [email protected] <0.1070.0> -------- OS Process Error <0.1108.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:27 Elrond couchdb[4172]: [info] 2018-12-31T15:18:27.557800Z [email protected] <0.238.0> -------- couch_proc_manager <0.1113.0> died normal
Dez 31 16:18:27 Elrond couchdb[4172]: [error] 2018-12-31T15:18:27.557958Z [email protected] <0.1070.0> -------- OS Process Error <0.1113.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:27 Elrond couchdb[4172]: [info] 2018-12-31T15:18:27.780737Z [email protected] <0.238.0> -------- couch_proc_manager <0.1116.0> died normal
Dez 31 16:18:27 Elrond couchdb[4172]: [error] 2018-12-31T15:18:27.780836Z [email protected] <0.1070.0> -------- OS Process Error <0.1116.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:28 Elrond couchdb[4172]: [info] 2018-12-31T15:18:28.032640Z [email protected] <0.238.0> -------- couch_proc_manager <0.1119.0> died normal
Dez 31 16:18:28 Elrond couchdb[4172]: [error] 2018-12-31T15:18:28.032745Z [email protected] <0.1070.0> -------- OS Process Error <0.1119.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:28 Elrond couchdb[4172]: [info] 2018-12-31T15:18:28.283102Z [email protected] <0.238.0> -------- couch_proc_manager <0.1122.0> died normal
Dez 31 16:18:28 Elrond couchdb[4172]: [error] 2018-12-31T15:18:28.283144Z [email protected] <0.1070.0> -------- OS Process Error <0.1122.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:28 Elrond couchdb[4172]: [info] 2018-12-31T15:18:28.503788Z [email protected] <0.238.0> -------- couch_proc_manager <0.1125.0> died normal
Dez 31 16:18:28 Elrond couchdb[4172]: [error] 2018-12-31T15:18:28.503898Z [email protected] <0.1070.0> -------- OS Process Error <0.1125.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:28 Elrond couchdb[4172]: [info] 2018-12-31T15:18:28.720950Z [email protected] <0.238.0> -------- couch_proc_manager <0.1131.0> died normal
Dez 31 16:18:28 Elrond couchdb[4172]: [error] 2018-12-31T15:18:28.721074Z [email protected] <0.1070.0> -------- OS Process Error <0.1131.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:28 Elrond couchdb[4172]: [info] 2018-12-31T15:18:28.968368Z [email protected] <0.238.0> -------- couch_proc_manager <0.1161.0> died normal
Dez 31 16:18:28 Elrond couchdb[4172]: [error] 2018-12-31T15:18:28.968477Z [email protected] <0.1070.0> -------- OS Process Error <0.1161.0> :: {os_process_error,{exit_status,139}}
Dez 31 16:18:29 Elrond couchdb[4172]: [info] 2018-12-31T15:18:29.153334Z [email protected] <0.238.0> -------- couch_proc_manager <0.1164.0> died normal
Dez 31 16:18:29 Elrond couchdb[4172]: [error] 2018-12-31T15:18:29.153449Z [email protected] <0.1070.0> -------- OS Process Error <0.1164.0> :: {os_process
The nspr package is installed...
2 cents, anyone?
produnis
on 31 Dec 2018
Well, error 139 is SIGSEV and repeating your steps on Arch in Vagrant gives me the same result with kernel log featuring messages like:
Dec 31 17:24:50 archlinux kernel: couchjs[12088]: segfault at 10 ip 00007f3dc292c244 sp 00007ffc68a4d778 error 4 in libpthread-2.28.so[7f3dc2928000+f000]
I guess this is up to how mozjs185 been compiled. Try to reach to the author of the package?
eiri
on 31 Dec 2018
I contacted the package-maintainer, and today a new version of js185 was ready. Unfortunately, the error still remains.
Is there anything I could point the maintainer to in detail?
produnis
on 1 Jan 2019
The maintainer doesn't have all of the patches we use in our libmozjs compile for Ubuntu/Debian/RedHat/Fedora/SuSE/CentOS. Perhaps one of them will solve the problem.
Point him here: https://github.com/apache/couchdb-pkg/blob/master/js/README.md
and specifically here: https://github.com/apache/couchdb-pkg/tree/master/js/debian/patches
He should try applying all of those patches and see if it improves things.
You can also manually test couchjs itself by running:
path/to/couchjs path/to/share/server/main.js
# then, when it's running, type
["reset"]
# it should respond with true
["map_doc",{}]
# it should respond with []
# exit with CTRL-D
wohali
on 1 Jan 2019
Hi there, and thx for your reply. I sent an email to the package maintainer.
Also, I tried your manual test, and it goes through without errors:
21:32:56|produnis@Elrond:~$ /usr/lib/couchdb/bin/couchjs /usr/lib/couchdb/share/server/main.js
["reset"]
true
["map_doc",{}]
[]
21:33:35|produnis@Elrond:~$
@produnis Were you able to test the latest build I provided to you? If not I’ll try to deploy a new Arch machine to test (I’d rather avoid crashing my running couchdb on a prod machine…).
ArchangeGabriel
on 3 Jan 2019
Hi Bruno,
installed js185 version 1.0.0.5 yesterday (I wrote you an email)butunfortunately, the segfault error is still there... That's why they pointed me/us at these patches (see wohali post)
produnis
on 3 Jan 2019
Yeah, but since then I’ve built 1.0.0-6 with more patches, and send it to you by email. Did you not received it?
ArchangeGabriel
on 4 Jan 2019
No, I didn't get that mail. Where can I find 0.6-Version?
produnis
on 4 Jan 2019
OK, please find it here then: https://paste.xinu.at/tD1hZ/
ArchangeGabriel
on 4 Jan 2019
Thx a lot. I installed 0.6, but the SEGFAULT is still there..... :/
produnis
on 4 Jan 2019
OK, I see I still don’t have --disable-methodjit, so I added that too: https://paste.xinu.at/O2h2S/. But if it still does not work, then I’ll try to reproduce it in gdb with debug symbols to get a better understanding of what is happening.
ArchangeGabriel
on 4 Jan 2019
I’ve just realized that couchjs has been segfaulting massively on my server lately, so the issue might be completely different and related to another change. I’m investigating.
ArchangeGabriel
on 4 Jan 2019
@produnis Could you try editing /usr/lib/systemd/system/couchdb.service, removing the line MemoryDenyWriteExecute=true, then systemctl daemon-reload, systemctl restart couchdb and trying again your test case?
ArchangeGabriel
on 4 Jan 2019
I need to do more tests, but MemoryDenyWriteExecute=true is definitively the culprit. However one of the changes I’ve made to libmozjs in order to try to solve this broke couchjs in a different way:
couchdb[17334]: [error] 2019-01-04T19:42:46.189862Z couchdb@localhost <0.2485.0> -------- OS Process Error <0.4280.0> :: {os_process_error,{exit_status,1}}
Reverting to previous version works OK, but my best guess is the tag patch breaks ABI and requires to recompile CouchDB. Will check that latter tonight and push fixed version of both packages.
ArchangeGabriel
on 4 Jan 2019
Hi Bruno, I uncommented MemoryDenyWriteExecute=true in systemd, and now I do NOT get any more segfaults...
THX A LOT!!!!
produnis
on 4 Jan 2019
On my server, I installed version 0.7, and I edited couchdb.service, and restarted everything, and on this server I get{os_process_error,{exit_status,1}}, like you.
On my desktop machine, after editing couchdb.service, couchdb runs just fine without errors...
Both machines have the same arch-install... I don't know why my desktop runs just fine while my server is still throwing an error...
produnis
on 4 Jan 2019
I downgraded my server to js185-version5 (as in repo), and now error 1is gone, too, and everything runs smooth...
produnis
on 4 Jan 2019
@ArchangeGabriel Yes, one of the patches in our repo does break ABI compatibility because of required changes for aarch64. (An additional high address bit is masked off in the VA space because of limitations on that platform.)
wohali
on 4 Jan 2019
OK, fixed the CouchDB package by rebuilding against ABI-modified mozjs. Everything is in order now, this can be closed. Thanks for your help and pointing me towards your patchset. :)
ArchangeGabriel
on 6 Jan 2019
Related issues
maciozo
·
5Comments
sploders101
·
3Comments
DeylEnergy
·
5Comments
wohali
·
5Comments
popojargo
·
5Comments
Most helpful comment
OK, fixed the CouchDB package by rebuilding against ABI-modified mozjs. Everything is in order now, this can be closed. Thanks for your help and pointing me towards your patchset. :)