Terminal: Consider using OSC 9;9 to inform Windows Terminal about CWD

[skyline75489]

Description of the new feature/enhancement

About a month ago I've filed #7668 , in the seek of a solution for #3158 . It raised a lot of discussion and I can totally understand why #7668 is acceptable to some people but unpalatable to others.

Recently with #8055 being accepted as a solution, I thought why not accept more sequences from ConEmu, which is a fantastic product I've personally used for a long time. So here I purpose we implement another OSC 9 custom sequence from ConEmu:

|Sequence|Descriptoin|
| :------------- | :----------: |
|ESC ] 9 ; 9 ; “cwd” ST | Inform ConEmu about shell current working directory.|

The source can be found in the ConEmu doc.

Proposed technical implementation details (optional)

I hope I could dig more about ConEmu's implementation but I think it should be pretty straight-forward and Windows-friendly as it is a native Windows application.

Update: I found more about this feature here, which takes basically the same approach as #7668, that is to configure the shell to generate correct OSC 9 sequences. A critical thing that is missing is the username@hostname part comparing to OSC 7, which is useful in remote connections. Still I think OSC 9 is a realistic starting point and should be acceptable to most people out there.

[skyline75489]

CC @Maximus5 for any suggestions on the remote connections.

[zadjii-msft]

I owe you a longer writeup on both this and OSC 7, and now that I'm back from leave, I don't really have any excuses 😝 I'll try to write something up later today. Thanks for your patience!

[skyline75489]

@zadjii-msft It's absolutely OK. Take you time, man.

I've been experimenting with ConEmu and OSC 9;9 sequences. From what I can tell, the result of a OSC 9;9 is an option in the "Startup directory" when creating a new console:

So yeah, I guess we could use it for #3158 without breaking the original meaning of the this sequence.

[zadjii-msft]

(copypasta from my comment in #7668)

This one doesn't seem that bad. Since it's a windows-first sequence, it doesn't need to know anything else about who's emitting the path. We can always assume it has been emitted as a Windows path. On the surface, this seems to work great, save for _one_ scenario - ssh. This sequence doesn't accept any sort of hostname, it only takes the path. So if you were ssh'd into another machine, and that one emitted a OSC9;9, then there's no way of determining that sequence wasn't intended for the current machine.

I think I'm okay with us implementing this as-is for the time being though. It's not a perfect sequence, but it's _good enough_, and doesn't come with the compat baggage that OSC7 does. It can certainly be supported with the caveat "This won't work in an ssh session, or in any scenario where it's emitted by a remote connection. It will always be treated as a path on the local machine".

[skyline75489]

I think as a very Win32 specific sequence, I can understand why HOSTNAME does not exist in OSC 9;9: sshing into Win32 machines is simply not possible at the time the sequence was invented. And WSL was also not a thing.

Note that for most cases, it’s still good enough. We’ll always check the availability of the CWD before passing it to shell program, to make sure it actually exists on the local machine.

Having a working but not perfect solution is better than have nothing at all. Besides we can always extent OSC 9;9 as long as we align the movement with ConEmu in the future.

获取 Outlook for iOShttps://aka.ms/o0ukef

[Maximus5]

Just a couple of things to mention. From ConEmu perspective.

The "cwd" could be unix encoded (/mnt/c/path, /cyddrive/c/path, or simply /c/path) or Windows encoded (c:\path). And there are also network paths too... Depends on the shell emitting the sequence and settings.

It does not have much sense to check path existence upfront, we could have e.g. /c/users/admin on remote and localhost both.

In ConEmu the cwd is used not only for starting new shell, but also for processing "hyperlinks", such as produced in compilers errors output. ConEmu uses cwd to expand relative path. Interesting point here is that we could have in backscroll different points where cwd was changed (make was started from different folders). So it could be helpful if Terminal saves cwd for range of lines.

[TBBle]

Apart from the path format (file:// URL vs potentially DOS-style path) and not having a 'hostname', what's the difference between this and OSC 7? Both notify of CWD, as I understand.

Is the advantage simply that we know a Windows-unaware program isn't going to generate this? And hence that if one needs to use cygpath or wslpath because one is in a virtualised-filesystem environment under Windows, then one sends the result via OSC 9;9 rather than OSC 7, and so Windows Terminal can implement only this, and leave OSC 7 unimplemented, as coming from a Windows-unaware shell and most-likely incorrect anyway.

Although ConEmu docs mentioned in the original ticket just send the locally-visible path out of WSL/Cygwin to OSC 9;9... It's not clear how that can work, perhaps it relies on something in the ConEmu cygwin/msys terminal connector, which means that OSC 7 and OSC 9;9 remain semantically the same, and WSL/Cygwin are still problematic.

[Maximus5]

I'm not sure when OSC7 appeared. The VT100 and xterm sequences does not have this code, and these standards I generally took into account implementing ConEmu parser.

Connector is not required to process the OSC9;9, ConEmu does not care who actually emits the cwd to terminal. On the other hand, without connector no ANSI is written to console, they are processed by cygwin1.dll. But Terminal should not care about it I believe. If sequence is emitted, terminal should note the cwd change.

[j4james]

I'm not sure when OSC7 appeared. The VT100 and xterm sequences does not have this code

Yeah, XTerm doesn't support OSC 7. I believe it was originally developed by Apple for the OSX terminal app in 2011. I think VTE was next to support it in 2012, and over the years several other terminals have followed suit, including Mintty, Konsole, and iTerm2. Some of those terminals also had their own sequences for setting the directory, but they added OSC 7 support for compatibility. At this point, I'd probably consider it a de facto standard.

[TBBle]

To be clear, I wasn't trying to be critical of ConEmu for ignoring an existing standard etc.

What I'm really wondering is that given the ConEmu documentation suggests this for $PS1 in WSL:

if [[ -n "${ConEmuPID}" ]]; then
  PS1="$PS1\[\e]9;9;\"\w\"\007\e]9;12\007\]"
fi

and \w will be generating something like /home/myInWSLUser, how does that actually _work_ for "Create new console"? This was one of the major issues in the OSC 7 discussion #7668, and since I'm not a ConEmu user, I'm interested to know if ConEmu had a solution for that.

I had expected to find it using something like:

if [[ -n "${ConEmuPID}" ]]; then
  PS1="$PS1\[\e]9;9;$(wslpath -m \"\w\")\007\e]9;12\007\]"
fi

[skyline75489]

@Maximus5 Thanks for the reply. One thing I'd like to point out is that, we are mainly targeting #3158 as the primary usage of OSC 9;9.

It does not have much sense to check path existence upfront, we could have e.g. /c/users/admin on remote and localhost both.

Generally I get your point. But as I said we will mainly use the CWD to open new terminal tab in same directory. We don't really have an dialog to choose the starting directory, at least for now. In my initial implementation, the newly spawned shell will just arbitrarily use the CWD set by previous shells. This is why we want the CWD to be a valid local path (also why we take remote connections into consideration).

@TBBle The screenshot in my previous comment somewhat explains how it works. As far as I can see, ConEmu will not enforce users to use CWD to open new tabs. Instead it just adds an option in the dropdown. I guess this is one of the reasons why ConEmu is spared from the issues discussed in #7668.

[TBBle]

Ah, thank you, I understand now. I hadn't realised "Create New Console" mentioned in the ConEmu docs had an intervening dialog before actually launching a new console.

[zadjii-msft]

and \w will be generating something like /home/myInWSLUser, how does that actually _work_ for "Create new console"? This was one of the major issues in the OSC 7 discussion #7668, and since I'm not a ConEmu user, I'm interested to know if ConEmu had a solution for that.

I'd certainly be curious how that's supposed to work as well. I _presumed_ that this was only being used for emit Windows-style paths for ConEmu. I'm okay with supporting this sequence, but only with the caveat "these paths must be the Windows relative path". If this isn't the typical usage, then it's certainly no better than OSC7.

Forcing the emitter to use Windows root-relative paths definitely puts the onus on the emitter to ensure that they're self-aware if they're running in WSL, cygwin, etc. It's not great, but for now it's better than OSC7 where the precedent has already been set that the emitter _won't_ be aware of such scenarios.

---

As far as path validation goes, I'd start by treating this the same way wt -d c:\some\invalid\path works today. Maybe in the future, we could add some validation at the time we create the connection to check if the path exists (instead of just returning a [error 0x8007010b when launching 'cmd.exe']), but let's focus on the plumbing first IMO.

[Maximus5]

As far as I can see, ConEmu will not enforce users to use CWD to open new tabs. Instead it just adds an option in the dropdown.

That's almost true. User may configure task to use %CD% as task startup directory. I didn't make it default behavior, but think to do from time to time.

and \w will be generating something like /home/myInWSLUser

ConEmu can (at least should) do automatic translation of Unix path to Windows path to maintain PWD. But with /home folders it's problematic of course.
At some point it was not working, but now as I can see, starting wsl in the folders like \\wsl$\Ubuntu-18.04\home\maximus\myInWslUser and C:\SRC works properly.
So I tend to change \w into wslpath -m "\w" in docs for wsl.

[TBBle]

Quick note on wslpath -m, it's _possible_ that there's no -m, and it needs to be -w. I know cygpath -m works for MSYS/Cygwin, but I don't recall if wslpath supports it.

It probably should be wslpath -w and cygpath -w for _OSC 9;9_ anyway, since the -m is "Windows path but with /", i.e. useful for appending to file://localhost/ to buld an _OSC 7_-valid URL.