Fish-shell: cd into symlink results in suprising behavior.

[newhook]

bash

bash-3.2$ cd /usr/local/opt/berkeley-db53
bash-3.2$ pwd
/usr/local/opt/berkeley-db53

fish

matthew@matthew-osx /u/l/opt> cd /usr/local/opt/berkeley-db53
matthew@matthew-osx /u/l/C/b/5.3.28> pwd
/usr/local/Cellar/berkeley-db53/5.3.28

[ridiculousfish]

This is addressed in the FAQ.

The short story is that the true working directory (i.e. chdir(2)) is always resolved by the kernel. The shell can pretend to not resolve symlinks, but this fiction unravels when an external command is invoked. Try this in bash:

cd /usr/local/opt/berkeley-db53
ls ..
cd .. ; ls

The differences between the output of the second and third lines is what Axel hoped to avoid.

[ridiculousfish]

I don't want to close this since I think it's worth tracking the discussion, but I don't think we'd take a fix unless someone made a convincing argument that some other behavior is better than always-resolve.

[newhook]

I really dislike this behavior. It is quite disconcerting that if you:

cd /usr/local/opt/berkeley-db53
cd ..

you don't end up in /usr/local/opt.

[externl]

I have to agree with @newhook, it's pretty annoying and disorienting. I think it's a convincing enough argument that (pwd)/foo != cd foo; and pwd.

The current logic also also makes any serious scripting using symlinks a real pain.

[nakal]

Most shells try not to confuse users by resolving softlinks. fish should not do it, either. The reason is that you can always find the resolved path by asking the system to do it, but you will never find out the other direction. When you have already a resolved path, you cannot find out which way through the softlinks you have taken.

On FreeBSD there is a command called realpath that resolves a path for you. There should be something equivalent for most operating systems, I think.

[devlinzed]

Fish will always resolve symlinks because chdir resolves symlinks. It can try to (inconsistently) mask that fact but it's much better to get adjusted to the semantically correct behaviour than try to hide it with inconsistent, incorrect behaviour.

[jdoss]

@devlinzed How is it much better for the user? It doesn't really matter if it is the semantically correct behavior if it will cause some confusion and pain for new users. I know it caused frustration for me when I gave fish a try over bash for a month or so and it was part of the reason why I switched back.

Maybe a setting for those that want the "incorrect" behavior so symlinks behave how they expect it to would be a good middle ground?

[nakal]

chdir should _not_ make absolute path. This is wrong and annoying and a part of the problem.

Watch this and tell me which is more intuitive:

zsh (and basically every shell except fish):

# pwd
/home/nakal
# cd pics
# pwd
/home/nakal/pics
# cd ..
# pwd
/home/nakal

Now the same in fish:

# pwd
/home/nakal
# cd pics
# pwd
/net/server/pics
# cd ..
# pwd
/net/server

Now it's obvious that I made a symlink from my home to an NFS tree. But I made this symlink, because I don't want to care about the external tree. Now I need to deal with it, when I use fish.

Also tab completion is not resolving symlinks.

cd pics/..<TAB>

Gives me other results on fish than I expect. symlinks are an abstraction, you remove parts of this abstraction and remove many features with this decision.

[auscompgeek]

chdir should _not_ make absolute path.

@nakal, you're not understanding the problem correctly. chdir(3p) resolves symlinks. Fish isn't doing anything special in this regard. The reasons why bash behaviour is less intuitive have already been described in this thread.

Tab completion not resolving symlinks correctly would be a real issue though, and a new issue should probably be filed.

[nakal]

I described the last one wrong. Tab completion shows me the contents of the /home/nakal directory, but when I press return, I am in /net/server (and there are other files than previously shown by tab completion).

[lloeki]

To wit (mocking a capistrano release structure):

Sandbox> mkdir cap
Sandbox> cd cap
cap> mkdir -p shared releases/{1,2,3}
cap> ln -s "$PWD"/releases/3 current
cap> cd current
current> ls ..
current/  releases/
current> ls ..<TAB>        # bash completes to ../
current> ls ../<TAB>
current/  releases/
current> ls ..<TAB>         # zsh does not complete
Completing `files' or `corrections'              # nor provides results
current> ls ../<TAB>       # zsh provides results
Completing files
1/  2/  3/
current> cd ..     # both go back up the symlink

Bash is extremely pernicious, because when you swap ls for e.g vim and try to edit a file in shared from inside current, it will autocomplete but it won't work.

Zsh does it right though (compared to bash), by clearly separating interactive navigation from command execution, thus only using the symlink as pwd for navigation and in $PWD, otherwise resolving the symlink for command and argument autocompletion, and by making use of the convention or method that consists of resolving symlinks when using a final slash (foo vs foo/, see point 3).

The reasoning is that the mental model of the user doing cd foo; cd bar; cd baz is equivalent to doing cd foo/bar/baz by way of pushing levels of the tree hierarchy onto some stack, and that doing cd .. is equivalent to popping a logical level. The consequence is that bash/zsh lazily resolves the symlink, whereas fish eagerly resolves it.

When scripting, the most robust solution to 'go back up' is certainly to never use cd .. but cd while inside a subshell, or leverage a real stack using pushd/popd/cd - (does fish have those?).

When interactive, should I cast a vote, I'd probably go with the above mental model, however 'unpure' it may look like.

Either way, this can't be an option, or it'll subtly break script behaviour consistency across setups with different settings, unless they start with some shopt-like stuff forcefully, which I'm quite sure is a territory the fish authors don't want to enter.

[nakal]

From what I see, the shell command cd will never resolve symlinks in other shells, even with a trailing slash. The portable solution to resolve paths is readlink -f <path>.

One of the implications when you always resolve symlinks when changing directory is that you cannot have a symlink in your path for your $HOME, which is possible (and sometimes used) on FreeBSD (absolute path is usually /usr/home and symlinked /home). fish will simply never show ~ when you are at home (and this particular fact is ok, because when you are not using the specified symlinked path, you are not at home).

Btw, on zsh cd .. won't complete to cd ../ if you haven't configured it to do it, but this is more csh-like behavior and I prefer it, personally. I am not a bash user, btw, but if you always criticize it, even when bash seems to behave more consistently here, it sounds rather unprofessional. And we are not talking about how you have configured your completion on zsh. We are talking about how cd is handled in all common shells with respect to symlinks and how fish is different/annoying here.

[zanchey]

For reference, readlink -f is not supported on OS X, and the FreeBSD issue noted regarding home directories is fixed in #1133.

[lloeki]

if you always criticize it, even when bash seems to behave more consistently here.

Bash is inconsistent, and dangerous:

Sandbox> mkdir cap
Sandbox> cd cap
cap> mkdir -p shared releases/{1,2,3}
cap> ln -s "$PWD"/releases/3 current
cap> touch .ruby-version releases/.ruby-version
cap> cd current
current> rm ../<TAB>
1 2 3
current> rm ../.ruby-version
# oops, wrong one

And we are not talking about how you have configured your completion on zsh.

I don't care how one's shell is configured regarding completion, I was just being exhaustive in logging the interactive session. What I do care about though is that completion, builtins such as cd and external commands respectively produce and are fed arguments in a sane manner. This is a very complex problem with lots of corner cases and finding a good solution requires as much feedback as possible.

[jlgerber]

This behavior is going to be a deal breaker for me. I just found fish, and was interested, but I am promptly uninstalling it. I rely on symlinks to abstract directory layouts. And so does the company i work for. And it has been working well enough for 20+ years. Given the fact that this feature exists in all of the other shells in use, you would think that there is a compelling user story here.

[kopischke]

Given the fact that this feature exists in all of the other shells in use, you would think that there is a compelling user story here.

I am not sure that is not too sweeping a statement. Personally, I always have found the fact I could not climb up the physical directory hierarchy symlinked to in bash maddeningly counterintuitive.

[newhook]

I'd say thats a pretty personal opinion. This behavior of fish was terrible annoying to me and broke my workflow pretty badly.

[kopischke]

I'd say thats a pretty personal opinion.

Absolutely – but so are the dissenting ones on this issue. For what it is worth, the GUI file managers on both Windows and OS X resolve the linked path to the physical layout, so there is obviously more than one compelling user story at play here, which is all I am trying to say.

[ghost]

Just want to share my perspective. I never really learned bash or zsh well enough, so never realized fish behavior was not the norm here. Now after reading this, I think I find fish behavior more intuitive, but you could argue that's because I am so used to it.

[faho]

This behavior of fish was terrible annoying to me and broke my workflow pretty badly.

@newhook: So... what _is_ your workflow? I mean I know how to see the behavior you're talking about but you've never said how exactly this impacts you. What are you trying to do? How does it hinder or confuse you?

It would be possible to change this, even to make it configurable, but we'd need an actual case for it, not just "other shells do it this way" (which doesn't carry that much weight since fish doesn't have compatibility as a goal) and vague "my workflow" (I could link the xkcd here).

[externl]

I experience @newhook's example which he posted when opening this bug all the time with Homebrew.

> cd /usr/local/opt/fish/
> pwd
/usr/local/Cellar/fish/2.3.0
> cd ..

You would expect to be in /usr/local/opt, but you are not. While it's not really a workflow, I often have to cd in and out of folders in /usr/local/opt. Every folder there is a symlink, so it's quite painful.

[ghost]

@externl OS X user here too. Do you often need to go into the Cellar? and just out of curiosity, why? I know it's all cozy in there, but I never cd into that directory. I may list its contents if I want to see what a brew package installed in my system, but that's about it.

[kopischke]

@externl also, for just “cd[ing] in and out of folders”, Fish has prevd.

[newhook]

I have lots of symlinks on my system for a whole host of issues. For example, I use pow.cx extensively and that wants to have symlinks to link from ~/.pow into your applications. The symlink issue made navigation in there frustrating.

To be honest I gave up on fish and went back to zsh. The two major issues for me were the symlink issue and the vi editing mode somewhat broken. Overall it was just too frustrating.

[faho]

What I had initially thought was that it might be possible to _allow_ someone to show an unresolved PWD by not resolving the variable but adding a call to fish_realpath in prompt_pwd, but that would break e.g. tab completion - we'd be showing the wrong directory. So the only way to allow someone to use it that way would be to offer an actual configuration option, and we try not to do those. As I understand it, we'd also need to "unresolve" paths as opposed to not resolving them.

The problem is that showing an unresolved PWD is pretending. It's just not how things work - and in bash it seems the only thing that behaves like PWD will not be resolved is cd (try echo ../*). The way everything else (including most of bash) works is that symlinks are a one-way trip.

The symlink issue made navigation in there frustrating.

This might be possible to solve better, but for that we'd need a better description - we have a few niceties related to changing paths, including $CDPATH and cd - (and prevd and such), and I'm not sure if you've used them or not, or if there's anything we can improve there.

But since everyone asking for this seems to have moved on, I'm going to close this again.

[newhook]

This might be possible to solve better, but for that we'd need a better description

What do you need a better description of? I think lots of people have said what is frustrating and no amount of cd -, prevd and stuff will sort that out. If you cd /usr/local, cd bin, pwd (/usr/local/bin), cd .. you expect to go back /usr/local not some other wacky location just because of a symlink. Thats very confusing, not how most other unix shells work, and frustrating behavior.

[ghost]

@newhook You said that your workflow was broken because you use pow extensively, which symlinks stuff under ~/.pow, and fish behavior to resolve symlinks when changing dir was throwing you off.

So, problem was, you often went into ~/.pow/xxx which would would teleport you into xxx's realpath (which in itself is not problematic), but then... when you did .. or the like, you'd end up in a different location.

The solution is then obviously to not .. or the like, instead, write the full qualified path, which in your case seems as simple as ~ since pow lives in your home directory by default.

[faho]

What do you need a better description of?

Well, cd .. is _not_ going to go back through the symlink. We've discussed that, and the inconsistency isn't worth it - see the various examples of bash's weirdness here.

So what we'd need is a description of _what_ you want to do, not _how_ you'd want to do it, to find out if there's a different solution. Where are you trying to navigate to? Just to throw an idea around: if we allowed something like cd -/SOMEWHERE to go to the last directory and then the subdirectory SOMEWHERE (with full completion)... would that help?

Of course if you're dead set on "cd .. goes back through the symlink, ls .. doesn't" (because we have no control over ls)... then we'll have to disagree.

[newhook]

I don't want to be rude, but I'm afraid you don't get it. I don't want a different solution. I want cd .. to work just like all the other shells that I've been using for 20+ years.

[faho]

Then: Sorry, but no.

[grrinch]

I completely agree with @newhook.

If you cd /usr/local, cd bin, pwd (/usr/local/bin), cd .. you expect to go back /usr/local not some other wacky location just because of a symlink.

I've tried Fish, the cd behaviour with symlinks annoys me in exactly the same way newhook described and I went back to zsh. This is exactly why Fish is a NO-GO for me for the time being. Maybe I'll use a prevd command, which seems perfect to what I need, however cd .. is just something you can get used to for XX yrs.

If you guys want a "use case" to decide on implementing this so called feature (which TBH is a standard behaviour for some people), make a poll and see how many users want to resolve the symlinks and how many don't. Otherwise it just looks like a proper argument-against-argument shitstorm. Pitty that contributors exile users only over their own mindset and conviction that something is good for the users - it just frustrates both sides.

[newhook]

I'm all for improving things, however, changing the standard behaviour of 35+ years of a core shell interaction such as chdir does not, imo, improve anything. It makes the shell feel confusing, broken and annoying. You are, of course, free to ignore me and tell me that I'm using the shell incorrectly but don't then be shocked when your hear complaints about the shell being broken.

[ItamarGronich]

i too have a hard time with this behavior. i need to use drush in a symlinked directory and always end up in the wrong one. i want the ability to choose the way the shell acts and configure it however i see fit. it damages my productivity.... is that a good enough "actual case" for you?

[ridiculousfish]

Wanting a shell that you can configure how you see fit is certainly reasonable, but fish is not that shell (see "Configurability is the root of all evil")

[newhook]

Whenever possible without breaking the above goals, fish should follow the Posix syntax.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/cd.html

Read the section:

The use of the CDPATH was introduced in the System V shell. Its use is analogous to the use of the PATH variable in the shell. The BSD C shell used a shell parameter cdpath for this purpose.

A common extension when HOME is undefined is to get the login directory from the user database for the invoking user. This does not occur on System V implementations.
...

[mattgodbolt]

Just some colour here from an enthusiastic fish user (for many years now). This behaviour is my only gripe. My use case; I work at a company where we have many many log directories archived on an NFS mount. The path structure is of the form /logs/year/month/date/application/... and so on, which makes it very easy to find the log files one needs.

However, behind the scenes these are all symlinks to various places around the network. My company isn't going to change that (boo) but it's one way to manage this kind of data.

fish makes this hard to use, as once I cd to a directory to poke around in logs, I'm left in the "real" directory of /log/raw-net/something/unrelated/but/managed/by/storage/people which bears no relation to the nice logical place I cded to. Going to the "previous day" isn't cd ../11, it requires me having to type the full path again.

Also once I find a spot, to grep in related locations I am used to doing things like grep interesting_thing ../../**/foo.log and similar. Which also doesn't work.

I understand this feature isn't going to be made optional, but it's a real pain point and a use case I hadn't seen discussed here, so for posterity I thought I'd post it.

[lattenwald]

Installed "fish" said "wow" but then:

cd .config-directory # which is a symlink to my git repo ~/Dotfiles/config-directory
~/Dotfiles/config-directory>

and understood something is wrong. Found this task and understood I am fucked.

Make symlink resolving optional, please.

[kah-ell]

Well, I just took the 10 Minutes to set up a Github account to satisfy my urge to post this comment.

I really, _really_ like fish, but this "feature" makes it unusable for me. My $HOME is organized via symlinks to keep the paths tidy and manageable - which fish complete and utterly negates. It's a shame.

The stated "reasons" for this behavior ignore a 20+ years consensus on how to work with symlinks. It's just beyond bizarre to even have to argue it. This petty adherence to some abstract principle can almost serve as a stereotype for a narrow mindset ruining an otherwise great product. And what about those repeated demands concerning "user stories"? User stories about an expected default behavior in shells for the past decades? Most users just quit. I'm sure for everybody who took the time to reply 10 others simply left. Make a poll about this and I'm pretty sure it's not even a contest.

Thanks to the other posters for repeatedly mentioning zsh, guess I'll check it out after fish got me started on a bash alternative.

As for the developers, I hope you know that those harsh words are motivated by a fondness for your product and work. It's just so damn frustrating.

[externl]

@kah-ell Can you elaborate more on your $HOME dir symlink setup, just curious.

[kah-ell]

It's a really simple affair.
My / and /home are on an different partitions of a SSD, and I have several other large HDDs connected. So everything that doesn't have to be in $HOME directly gets symlinked like ~/Downloads => /media/3TB/HOME/downloads for about everything I want to have accessible from a $HOME prompt.

[krader1961]

I'm really tired of commenting on this issue but having read it again in response to @kah-ell's comment I saw this statement:

I am used to doing things like grep interesting_thing ../../**/foo.log and similar. Which also doesn't work.

That doesn't work in bash or zsh either if you have cd'd through a symlink. That's the primary reason we have no interest in special-casing symlinks and .. solely for the builtin cd command. Because that would be the only context in which .. magically traverses back through the symlink. As has been stated numerous times.

I'm also bemused by statements like this one:

... changing the standard behaviour of 35+ years of a core shell interaction such as chdir ...

I'm old enough to remember when symlinks were first added to UNIX. The behavior that so many people are asking for has not been available for 35 years. Note that bash 2.02 added cd -P to ensure $PWD does not contain symlinks. Why? Presumably because some people realized that not resolving symlinks causes problems. And it wasn't until bash 2.05 that cd and pwd handled symlinks specially (at least according to the CHANGELOG) and that was released in 2001 -- 16 years ago, not 35.

I too make extensive use of symlinks. Both on my home server and laptop and on the systems at work. When I started using fish I too found it initially disconcerting that I couldn't cd .. to go back thru a symlink. But it took me all of one week to get used to the fish behavior and adapt my behavior accordingly.

[lapcchan]

well i would expect this feature, if not by default should be optional. but i found no solution yet. anyone willing to create a plugin or workaround for it?

[externl]

I know it does not follow the _fish way_, but I've written a plugin to handle cd'ing into symbolic links: fish-symnav.

It has some caveats and could use more work, PRs welcome.

Edit: Renamed symdir to symnav

[lapcchan]

thx @externl for fish-symdir, it saved my time from zsh configuration. i can now use fish again. all i can say is, its just user preferences.

[SanjayVyas]

Thanks @externl .. How do I get symnav to show the symlink path instead of real path in prompt and how do I get it to go back to symlink parent and not real parent?

I tried all the symnav vars but I must be confused, because it keeps showing real paths

[externl]

Hi @SanjayVyas, symnav will try to update your prompt automatically when it initializes, but sometimes it needs help. What does your fish_prompt function look like? It may be best for you to open an issue on fish-symnav and we can discuss it further.

Edit: fish-symdir has been renamed to fish-symnav in case anyone was confused.

[suzaku]

Just trying out fish for the second day. cd into one of the projects I inherited that depends heavily on symlinks to work like the other shells. Now I'm uninstalling.

[j-f1]

@suzaku It looks like the 3.0 beta handles symlinks differently.

[ridiculousfish]

Yes, as of 3.0 fish "no longer resolves" symlinks (the CWD is virtual).