Sdkman-cli: "file exists" when opening a terminal

Created on 18 Oct 2016 · 15Comments · Source: sdkman/sdkman-cli

When opening the terminal, I have this error:

/home/$USER/.sdkman/bin/sdkman-init.sh:144: file exists: /home/$USER/.sdkman/var/version

The issue is fixed by running sdk selfupdate force but it appears again once the OS is restared.

Environment:

Ubuntu 16.04
shell: /bin/zsh
sdkman: latest version

Source

cazacugmihai

👍1

Most helpful comment

This is because your shell has "noclobber" set, and the sdkman script is trying to clobber a file. You can check by running a command like this:

set -o | grep clobber

See man zshoptions for a discussion of this option. There isn't a good solution, other than setting the "clobber" shell option for any shell that runs sdkman, prior to running any sdkman script. e.g.:

set +o noclobber

setopt clobber

Of course, you can set this in a startup file for all shells, or for a shell that you only run for sdkman.

I suppose you could modify the sdkman scripts so they use ">|" instead of ">" when redirecting standard out to files. If this is done I'm guessing the next update to sdkman would overwrite any changes to the scripts.

Wrapping sdk in a script or alias that enables and disables clobbering could be a solution, but it's often hard to get such scripts right in all situations.

Other shell-script-based utilities also have this problem (i.e., zprezto).

jverne on 28 Oct 2016

👍4

All 15 comments

This is because your shell has "noclobber" set, and the sdkman script is trying to clobber a file. You can check by running a command like this:

set -o | grep clobber

set +o noclobber

setopt clobber

Of course, you can set this in a startup file for all shells, or for a shell that you only run for sdkman.

Wrapping sdk in a script or alias that enables and disables clobbering could be a solution, but it's often hard to get such scripts right in all situations.

Other shell-script-based utilities also have this problem (i.e., zprezto).

jverne on 28 Oct 2016

👍4

@jverne: I'm using zprezto and noclobber option is enabled by default. I have disabled it and everything works as expected now. Thank you!

If there is no other solution (implemented in sdkman), this issue can be closed.

cazacugmihai on 28 Oct 2016

👍1

Closing this issue as the problem lies with prezto being opinionated about file clobbering. I personally use ZSH with ohmyzsh and have no problems.

Worth adding that I also experienced this problem in the past with prezto which ultimately led me to switching to ohmyzsh.

marc0der on 28 Oct 2016

For future spelunkers

This is not a zprezto issue. It is a shell issue. zprezto just also happens to complain when the shell has noclobber set.

Here is what happens on plain bash:

jverne@YKF-JVERNE:~$ echo foo > out
jverne@YKF-JVERNE:~$ echo foo > out
bash: out: cannot overwrite existing file
jverne@YKF-JVERNE:~$ sdk
bash: /home/jverne/.sdkman/var/broadcast_id: cannot overwrite existing file
bash: /home/jverne/.sdkman/var/broadcast: cannot overwrite existing file

Both zprezto and sdk try to clobber files with ">", but the shell sometimes prefers if you use ">!" (or ">|") depending on how it is configured using `set and issues an error message. Each shell has a different way of issuing this message.

jverne on 28 Oct 2016

It occurs to me that a script fix might be to use a subshell to update/clobber files. e.g., in all the places we clobber files, we do something like this:

( set + noclobber; do_the_thing_that_clobbers > some_file )

To be robust we'd probably refactor this into functions; one for Bourne-style shells that support set or setopt with the clobber option, and those shells that do not.

jverne on 29 Oct 2016

This is not a zprezto issue. It is a shell issue. zprezto just also happens to complain when the shell has noclobber set.

@jverne I beg to differ, and can prove it in a few lines in a terminal. Give it a try yourself:

$ docker run --rm -it ubuntu:latest bash
# echo "test" > myfile
# echo "test" > myfile
# apt-get update && apt-get --assume-yes install zsh && zsh
# echo "test" > myfile

You will see _no_ error message in either case. Even when installing ohmyzsh, no errors. Only when using prezto these problems surface. This is due to prezto being opinionated about clobbering (and many other shell behaviours), without allowing you to override these settings. We won't be making any changes in our codebase to accommodate such a framework.

marc0der on 31 Oct 2016

@marc0der you might be seeing a related zprezto issue, or the error being issued might be massaged by zprezto, but my example above was pure, unadulterated POSIX bash as shipped on Linux. All your example shows is that you do not have the noclobber setting enabled, so you are not actually testing the root cause. But noclobber is _not_ some special thing offered by zprezto; it is a shell option present on most modern Bourne shells. It is part of the shell environment shipped with bash, zsh, ksh.

If you use a modern Bourne shell flavour like bash, ksh, or zsh, _and_ you have the "noclobber" option set, some utilities will fail to overwrite files. SDKMAN is one of those utilities. zprezto is another.

The underlying root cause of this report is not zprezto. It is the noclobber option that is present on most or all modern Bourne-style shells. (csh, et al, might have their own version of this, but I don't use csh.)

e.g. with a recent Ubuntu flavour of stock bash:

jverne@YKF-JVERNE:~$ set -o noclobber
jverne@YKF-JVERNE:~$ sdk list
bash: /home/jverne/.sdkman/var/broadcast_id: cannot overwrite existing file
bash: /home/jverne/.sdkman/var/broadcast: cannot overwrite existing file

[...]

 jverne@YKF-JVERNE:~$ set +o noclobber
 jverne@YKF-JVERNE:~$ sdk list

[...]

Once the noclobber option has been turned off, we no longer get the error. This is because SDKMAN uses ">" when redirecting output to files, and the shell (not zprezto -- in this example there is no zprezto at play) wants you to use ">!"

It doesn't matter that zprezto turns noclobber on, because it is a shell switch that can get turned on many different ways. It is even on by default in some Linux distros. The idea is to be clear about the root cause here so future visitors can figure out how to remedy the situation.

And the root cause is that the noclobber option causes truncating redirection to fail in SDKMAN. The immediate remedy is to turn off noclobber. This applies regardless of the presence of zprezto (or the use of the z-shell) or not.

Put another way, if you run zsh with OhMyZsh, and run set -o noclobber before invoking sdk we will see the exact same symptoms.

jverne on 31 Oct 2016

Yes, all distros come with noclobber disabled as sensible default. This is why we have many many happy users who never encounter this problem. If you _do_ want to enable it, do so at your own risk, but we won't be working around it.

marc0der on 31 Oct 2016

👎2

As I said, I don't really care if this isn't something you want to allow for. It's tricky and error prone, like most shellisms.

But we should at least document the root cause instead of pointing fingers at poor zprezto. Just this week I installed a test distro and noticed noclobber was set, and noticed someone else had it set on some build box I was inspecting. So, no, not _all_ distros disable it. And we can't count on it being unset all the time.

In some cases it is perfectly sane to expect to run both sdkman and have noclobber enabled. At least if someone else runs into it they can get confirmation here.

jverne on 31 Oct 2016

👍2

I always set noclobber in my shell environments to avoid accidentally overwriting files. To solve this sdkman issue, I edited ~/.sdkman/bin/sdkman-init.sh and disabled noclobber at the very top, and enabled it again at the very bottom:

# beginning of sdkman-init.sh file
set +o noclobber
... 
# end of sdkman-init.sh file
set -o noclobber

ironcamel on 4 Dec 2017

👍2

@ironcamel Probably a better solution is to save the user's clobber state and restore it at the end of file, so that way the shell's clobber isn't changed

pgilad on 3 Jun 2018

noclobber is a useful safety feature for interactive shells, and sdkman could easily work with it by just changing > redirections to >|. Unfortunately at the moment the only feasible options are disabling that feature, which makes normal shell use more risky, or not using sdkman. Modifying sdkman scripts manually is not a good solution because those changes would get overwritten by updates. So it would be great if sdkman could make this small change to accommodate users' preferences.

This is actually similar to the situation in Vim, where plugins are expected to be able to cope with changed options as it is normal that users want to configure Vim for their preferences.