The work from @ljo 's PR https://github.com/eXist-db/exist/pull/1141 does not yet for macOS:
(added: same for ubuntu)
During startup the directory tools/yajsw/${env.EXIST_HOME}/ ; the variable does not seem to be substituted?
start with
./tools/yajsw/bin/runConsole.sh
In the attached logfile the following line is relevant:
java.io.FileNotFoundException: ${env.EXIST_HOME}/tools/yajsw/tmp/in_-5209836399611897029$1476732036731 (Permission denied)
at java.io.RandomAccessFile.open0(Native Method)
setting EXIST_HOME manually does not fix the issue.
Thanks, I will check how this can be the case. Did you set EXIST_HOME before building or just running?
Just before executing the command mentioned above
any idea yet about this issue @ljo ? Or maybe a work around?
@ljo https://github.com/eXist-db/exist/issues/1154 contains valuable details....
I tried to install the daemon at my local Arch Linux running systemd. I set up successful $JAVA_HOME to /usr/lib/jvm/default which seems to be right and $EXIST_HOME to a symlinked /opt/exist.
When I try to start systemctl --user start eXist-db I get an error that "the control process exited with error code." Starting the wrapper manually with /usr/lib/jvm/default/bin/java -Dwrapper.pidfile=/opt/exist/tools/yajsw/work/wrapper.eXist-db.pid -Dwrapper.service=true -Dwrapper.visible=false -Djna_tmpdir=/opt/exist/tools/yajsw/tmp -jar /opt/exist/tools/yajsw/wrapper.jar -tx /opt/exist/tools/yajsw/conf/wrapper.conf gives me the following error:
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/groovy/groovy-patch.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/groovy/groovy-all-2.3.11.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/commons/commons-httpclient-3.0.1.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/commons/commons-codec-1.3.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/commons/commons-net-3.2.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/regex/automaton-1.11.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/glazedlists/glazedlists-1.8.0_java15.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/glazedlists/commons-beanutils-1.8.2.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/cron/joda-time-2.7.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/cron/yacron4j-00.03.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/jgoodies/forms-1.2.0.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-webdav/jackrabbit-webdav-1.5.6.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-webdav/xercesImpl.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-webdav/slf4j-jdk14-1.5.0.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-webdav/slf4j-api-1.5.0.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-dbx/vfs-dbx-00.02.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-dbx/dropbox-core-sdk-1.7.7.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/vfs-dbx/jackson-core-2.2.4.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/glazedlists/commons-beanutils-1.8.2.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/glazedlists/glazedlists-1.8.0_java15.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/keystore/keystore.jar
INFO: lib not found: /home/me/exist/tools/yajsw/lib/extended/abeille/formsrt.jar
YAJSW: yajsw-beta-12.06
OS : Linux/4.8.4-1-ARCH/amd64
JVM : Oracle Corporation/1.8.0_92//usr/lib/jvm/java-8-openjdk/jre/64
Oct 28, 2016 12:36:00 PM org.apache.commons.vfs2.VfsLog info
INFO: Using "/tmp/vfs_cache" as temporary files store.
Oct 28, 2016 12:36:00 PM org.rzo.yajsw.os.posix.PosixService init
WARNING: Error /etc/init.d : is not a directory
************* STARTING eXist-db ***********************
Oct 28, 2016 12:36:00 PM org.rzo.yajsw.os.posix.PosixService getPid
INFO: wrapper pid file: null
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.rzo.yajsw.boot.WrapperExeBooter.main(WrapperExeBooter.java:49)
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at org.rzo.yajsw.os.posix.PosixService.state(PosixService.java:649)
at org.rzo.yajsw.wrapper.WrappedService.state(WrappedService.java:618)
at org.rzo.yajsw.wrapper.WrappedService.isRunning(WrappedService.java:382)
at org.rzo.yajsw.WrapperExe.doStartPosix(WrapperExe.java:545)
at org.rzo.yajsw.WrapperExe.executeCommand(WrapperExe.java:162)
at org.rzo.yajsw.WrapperExe.main(WrapperExe.java:124)
... 5 more
The line WARNING: Error /etc/init.d : is not a directory makes me wonder.
A new version of the wrapper has been released: https://sourceforge.net/projects/yajsw/files/yajsw/yajsw-beta-12.07/
quote:
the 3rd item could point to some clues....... but replacing the jar files does not help, unfortunately.
that said, I feel the project is under documented and over complex :-(
Workaround:
wrapper.conf ; replace ${env.EXIST_HOME} with an absolute path to EXIST_HOME (e.g. /Users/me/git/exist./tools/yajsw/bin/runConsole.sh@dizzzz Please remove your faulty generated wrapper.conf and rebuild after applying this. Part of it is just the build where you got the wrong values. Tasmos problem I need to check out more, but at least we should avoid getting the the variable expansion to the non-set value (property name).
@tasmo It looks you just miss the wrapper.pidfile file. that is why you get the error in your manual run.
@ljo I'll do@
@ljo in your ideas, is EXIST_HOME a fixed value? What I mean is: when exist is installed via a JAR file, or a macOS .dmg file, twill then a 'valid' value be filled in?
@ljo The genereated systemd script got an parameter called -Dwrapper.pidfile=/opt/exist/tools/yajsw/work/wrapper.eXist-db.pid which I included in my direct call. Is it not the right param?
[Unit]
Description=eXist-db – Native XML Database –
After=syslog.target
[Service]
Type=forking
User=tom
ExecStart=/usr/lib/jvm/default/bin/java -Dwrapper.pidfile=/opt/exist/tools/yajsw/work/wrapper.eXist-db.pid -Dwrapper.service=true -Dwrapper.visible=false -Djna_tmpdir=/opt/exist/tools/yajsw/tmp -jar /opt/exist/tools/yajsw/wrapper.jar -tx /opt/exist/tools/yajsw/conf/wrapper.conf
ExecStop=/usr/lib/jvm/default/bin/java -Dwrapper.pidfile=/opt/exist/tools/yajsw/work/wrapper.eXist-db.pid -Dwrapper.service=true -Dwrapper.visible=false -Djna_tmpdir=/opt/exist/tools/yajsw/tmp -jar /opt/exist/tools/yajsw/wrapper.jar -px /opt/exist/tools/yajsw/conf/wrapper.conf
Restart=on-abort
StandardOutput=null
[Install]
WantedBy=multi-user.target
@dizzzz yes, a vaild value will be filled in.
@tasmo I assume you have the rights to write the specified work directory and the pid file if running as _the unprivileged user tom_ ... Could you please check:
@ljo Zes I could touch and remove the file /opt/exist/tools/yajsw/work/wrapper.eXist-db.pid easily with that user's rights.
The whole installation got the owner privileges of the user.
@tasmo OK. And you did remove your faulty wrapper.conf and rebuilt it with EXIST_HOME and JAVA_HOME set firstly? Since the error is due to the pid file is missing -- are you using selinux? What if you try to write your pid file somewhere else, eg under /home/tom/tmp if that directory exists?
@ljo On that machine I don't use selinux. Deleted wrapper.conf again and rebuilt exist. Set the PID to the users ~/tmp folder (which exists). EXIST_HOME and JAVA_HOME are set properly. With journalctl -xe I get eXist-db.service: Control process exited, code=exited status=216.
macOS:
git clean -fdx and git reset --hard to be sure all is cleanJAVA_HOME and EXIST_HOME ./tools/yajsw/bin/runConsole.shduring start it shows (progressing much better than previously!)
interestingly: eXist-db does start! second time without errors!!!!
INFO|56930/0|eXist-db|16-10-31 19:02:28|31 Oct 2016 19:02:28,083 [main] INFO (JettyStart.java [run]:150) - [jetty configuration : /.../existdb/tools/jetty/etc/standard.enabled-jetty-configs]
INFO|56930/0|eXist-db|16-10-31 19:02:28|31 Oct 2016 19:02:28,566 [main] INFO (JettyStart.java [run]:159) - Configuring eXist from /.../existdb/conf.xml
INFO|56930/0|eXist-db|16-10-31 19:02:28|org.exist.EXistException: java.lang.ClassCastException: org.exist.yajsw.Main cannot be cast to java.lang.Comparable
INFO|56930/0|eXist-db|16-10-31 19:02:28|31 Oct 2016 19:02:28,595 [main] ERROR (JettyStart.java [run]:172) - configuration error: java.lang.ClassCastException: org.exist.yajsw.Main cannot be cast to java.lang.Comparable
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPools.configure(BrokerPools.java:192)
INFO|56930/0|eXist-db|16-10-31 19:02:28|org.exist.EXistException: java.lang.ClassCastException: org.exist.yajsw.Main cannot be cast to java.lang.Comparable
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPools.configure(BrokerPools.java:110)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPools.configure(BrokerPools.java:192) ~[exist.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.jetty.JettyStart.run(JettyStart.java:164)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPools.configure(BrokerPools.java:110) ~[exist.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.jetty.JettyStart.run(JettyStart.java:164) [exist-optional.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.lang.reflect.Method.invoke(Method.java:498)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.yajsw.Main.start(Main.java:47)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.yajsw.Main.main(Main.java:65)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.yajsw.Main.start(Main.java:47) [classes/:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.yajsw.Main.main(Main.java:65) [classes/:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.lang.reflect.Method.invoke(Method.java:498)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.rzo.yajsw.app.WrapperJVMMain.executeMain(WrapperJVMMain.java:60)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.rzo.yajsw.app.WrapperJVMMain.main(WrapperJVMMain.java:43)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.rzo.yajsw.app.WrapperJVMMain.executeMain(WrapperJVMMain.java:60) [wrapperApp.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28|Caused by: java.lang.ClassCastException: org.exist.yajsw.Main cannot be cast to java.lang.Comparable
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.rzo.yajsw.app.WrapperJVMMain.main(WrapperJVMMain.java:43) [wrapperApp.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655)
INFO|56930/0|eXist-db|16-10-31 19:02:28|Caused by: java.lang.ClassCastException: org.exist.yajsw.Main cannot be cast to java.lang.Comparable
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:899)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.cpr(ConcurrentSkipListMap.java:655) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.doPut(ConcurrentSkipListMap.java:899) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListMap.putIfAbsent(ConcurrentSkipListMap.java:1962) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.Optional.ifPresent(Optional.java:159)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.concurrent.ConcurrentSkipListSet.add(ConcurrentSkipListSet.java:241) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPool.<init>(BrokerPool.java:400)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at java.util.Optional.ifPresent(Optional.java:159) ~[?:1.8.0_101]
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPools.configure(BrokerPools.java:178)
INFO|56930/0|eXist-db|16-10-31 19:02:28| at org.exist.storage.BrokerPool.<init>(BrokerPool.java:400) ~[exist.jar:?]
INFO|56930/0|eXist-db|16-10-31 19:02:28| ... 14 more
@tasmo please try the runConsole.sh first..... (as recommended by the YAJSW project)
@ljo it would be nice if ./build.sh clean or rebuild would clean the generated config files too. Is that possible?
@dizzzz That is the fuzzy part; runConsole.sh runs well.
@ljo would upgrading to the latest version be 'copying jar files' only or will there be more work?
@ljo on macOs, the steps of https://github.com/eXist-db/exist/issues/1147#issuecomment-257371921 result into a working result. 1st time, second time, after installing the deamon with ./tools/yajsw/bin/installDaemon.sh and a reboot.
I have two PRs open right now, one needs to be reworked when the other is pulled in. #1158 #1159 I hope this will contribute to a better working wrapper.
@dizzzz I definitely see the good in if clean and rebuild targets would do this regenerate of config files too. But since the behaviour has been to not overwrite them before I did not change it. So should I?
@dizzzz @ljo Just to remind you, that we build a different target on Travis, so you could always make a rule in the build file to do something slightly different on Travis if that helps you.
@ljo overwriting files is a good idea, it did cost me and @tasmo time because any changes were not reflected in the generated files. And until now a clean did not help either.
@adamretter my latest change will make sure the right value will always be set....
@ljo the behaviour is in https://github.com/eXist-db/exist/pull/1158
@tasmo I hope https://github.com/eXist-db/exist/pull/1161 in combination with the jar-upgrade PR improves the situation
Unfortunately it does not help. Get still the same results.
So I just tried to install the latest HEAD of eXist-db on CentOS, there is a problem with the installation of the wrapper which is clearly reported:
[root@izdanak exist]# pwd
/usr/local/exist
[root@izdanak exist]# tools/yajsw/bin/installDaemon.sh
++++++++++ YAJSW SET ENV ++++++++++
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)
wrapper home : /usr/local/exist/tools/yajsw
configuration: /usr/local/exist/tools/yajsw/conf/wrapper.conf
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)
---------- YAJSW SET ENV ----------
mkdir: cannot create directory ‘/tools/yajsw/work/wrapper.eXist-db.pid’: No such file or directory
Detected systemd running.
Do you want to use it (Y=Run service with non-privileged systemd/N=continue with privileged systemV-init)?
The error above is: mkdir: cannot create directory ‘/tools/yajsw/work/wrapper.eXist-db.pid’: No such file or directory. It does not seem to use the correct wrapper home path for the pid file. I suspect this may also be the problem experienced by @tasmo.
This seems to just be reported as a warning, but should really abort the installation process.
The issue is caused here in installDaemon.sh between lines 66 and 69:
w_wrapper_pid_file="$EXIST_HOME/tools/yajsw/work/wrapper.eXist-db.pid";
if [ ! -e "$w_wrapper_pid_file" ]; then
mkdir "$w_wrapper_pid_file";
fi
EXIST_HOME is never set before that. I think the solutions is to use wrapper_home instead which is set correctly by the installDaemon.sh script and derive an exist_home from that if needed.
yes correct;
I wrote it in the 'develop' room yesterday to @tasmo ; setenv.sh does not set EXIST_HOME
this is the key step to fix......
su it must be mkdir -p basedir($w_wrapper_pid_file) ?
Is there something new on this topic?
I am still not able to run/install yajsw in Ubuntu because of this issue
Souce: Current dev branch
still not working? hmmmm that is bad
I send https://github.com/eXist-db/exist/pull/1345 some time ago, it address env.JAVA_HOME issue by replace it with java (platform independent reference to java)
so you say that @StephanMa should set JAVA_HOME ?
@dizzzz nope, jvm run be run by java command. No need to give full path to it location, until you need it and then you set it manually.
but how can we enforce to use a specific java version? I have both java7 and java8 installed, I only want to use java8....
@dizzzz you have two options:
~99% of users will have only one (default) jvm. So, it must be our target and option to set it manually (by editing config file) for ~1% cases.
@StephanMa Can you tell me, have you set JAVA_HOME?
Echoing JAVA_HOME after a restart of Ubuntu, it seems that it is correctly set. When I enter this path manually to the wrapper, it works like a charm. Sadly not from scratch
@StephanMa What do you mean by manually entering?
I replaced the var in the scripts with hard coded path to the jvm
var = variable? using abbr is smtms a bit hrd to underst.
@shabanovd for me it is important that I can set JAVA_HOME and that then the correct java version is used WITHOUT extending the PATH with $JAVA_HOME/bin .
so scripts could work like
which java, and use the path to find out where JAVA_HOME is.I realise that with 'alternatives' this might not work....
@dizzzz I'm not against JAVA_HOME but I do not like idea that I have to set it. Ant is force it right now at yajsw, why it can't use java if there is no JAVA_HOME?
@shabanovd @dizzzz I think we should check in this order, and it should work for both of you:
java there, then use it, otherwise warning message.java on the PATH and it is valid, then use it.@StephanMa I think perhaps your JAVA_HOME is not correctly set for the user you run the scripts as, as the scripts will use the environment var if it is present.
@dizzzz Can you please comment on the compromise that I suggested to you and @shabanovd two comments up?
@adamretter I missed the comment; yes that would be acceptable I guess....
I m tempted to close and consolidate this with #1154,
@dizzzz is this still an issue that is high-prio and needs resolution?
ok to close and consolidate.
while not 100% identical please continue discussion in #1154 thanks