Invoke: pty-bearing subprocesses always appear to be 80x24, which stinks for interactive work

[matleh]

I have a invoke task, that enters a docker container in interactive mode. Basically, it does

ctx.run("docker start -it ubuntu bash", pty=True)

But inside that, the terminal is not really usable. tput cols and tput lines return default values of 80 and 24, but really these should be like 195 and 70. As a consequence, many commands like _less_ or _top_ or even bash line editing become unusable.

When I run the docker command directly (not through invoke) everything works as expected.

This can be reproduced with

from invoke import ctask

ctask
def foo(ctx):
   ctx.run("bash -c 'tput cols; tput lines'", pty=True)

will output 80\n24 no matter what size the surrounding terminal window is.

Is there some additional initialisation of the pty needed?

[sophacles]

I've reproduced @matleh's test case issue on my Mac in a term window of various sizes.

I've noticed in the past that there are strange pty things associated with docker as well, but those have been with and without inv in the loop. Irrespective of that, the term size an issue.

In pexpect there are the getwinsize and setwinsize methods to the spawn class- and they don't appear to be used unless the calling program does so. Also note that there are some issues with cygwin (and maybe others) mentioned in the comments and docstrings of the pexpect module.

[sophacles]

Quick update: a thought struck me to check term windows with a size _smaller_ than 80x24, and in that case the size is still reported as 80x24 as well.

[bitprophet]

I was going to say something may be buggy in https://github.com/pyinvoke/invoke/blob/89e147e80b8c955c835115c32b283b750f3452c8/invoke/platform.py#L18-L60 but then I checked and we only use that for our own "print stuff to terminal & format appropriately" code (e.g. inv --list).

So I checked pexpect because that's what runs pty=True commands, and sho nuff: https://github.com/pyinvoke/invoke/blob/89e147e80b8c955c835115c32b283b750f3452c8/invoke/vendor/pexpect.py#L566

I'm hoping that we can just drop something like p.setwinsize(*pty_size()) or similar, after this line: https://github.com/pyinvoke/invoke/blob/89e147e80b8c955c835115c32b283b750f3452c8/invoke/runner.py#L132 and that would work? Have not tried yet myself.

[frol]

@matleh How about

ctx.run("docker run -it ubuntu bash -i")

? (The trick is -i option to bash)

This works for me.

Though, yes, pty=True is definitely broken for interaction. My guess is that it is because of wrong stdin/stdout setting.

Note that tput will report that it is run on 80x24 terminal if not using pty, but you can hack this around with stty cols "$(tput cols)" and stty rows "$(tput lines)"

[susu]

I also have this problem, "bash -i" did not help. Any updates on this?

[frol]

Here is my workaround using pexpect instead of run(pty=True):

BEFORE

context.run(run_command, pty=True)

AFTER

def _run_with_pty(context, command):
    """
    This helper workarounds lack of complete PTY support in Invoke:
    https://github.com/pyinvoke/invoke/issues/216
    """
    import pexpect
    import signal
    from invoke.exceptions import Failure
    from invoke.platform import pty_size
    from invoke.runners import Result

    def autoupdate_pty_size(pexpect_process):
        def update_pty_size(sig, data):
            pexpect_process.setwinsize(*pty_size()[::-1])
        signal.signal(signal.SIGWINCH, update_pty_size)
        update_pty_size(None, None)

    command_process = pexpect.spawn(context['run'].shell, ['-c', command])
    autoupdate_pty_size(command_process)

    command_process.interact()

    command_result = Result(
        command=command,
        shell=context['run'].shell,
        env=None,
        stdout=command_process.read(),
        stderr=None,
        exited=command_process.exitstatus,
        pty=True
    )
    if not command_result:
        raise Failure(command_result)
    return command_result

_run_wuth_pty(context, run_command)

[svetlyak40wt]

@frol thank you for the snippet with workaround. It works for me with a little change. I use getattr(context['run'], 'shell', 'bash') instead of context['run'].shell. And also, I removed shell and env arguments from call to Result, because this class's constructor doesn't handle them in Invoke 0.12.x.

[iago1460]

@frol Thanks, the workaround seems working

[bitprophet]

This issue's a bit old and I'm wondering if it's been incidentally fixed in the interim:

• On Invoke 1.0.x, and I can (macOS 10.12, Python 3.6.4) do eg run("vim", pty=True) inside a 154x41 tmux pane no problem; can edit all the way up to the 154th-or-so column, and so forth. Even running :!stty size inside that vim shows the right dimensions in the subprocess' shell.
• So at least as of 1.0, we're transmitting the data correctly. Changelog mentions us doing so in 0.11.x, which would have come out before this ticket was filed, so not sure what's up with that.
• Figured maybe this was some bizarre interaction between Invoke and Docker (or at least something Docker happens to be doing re: terminals and pipes) but then I remembered I have some private tiny tasks file specifically for managing my own local containers, which does ctx.run("docker run -it ...", pty=True) all day long with no apparent problems.

So I'm guessing this is solved now and will close as "can no longer reproduce"...if anyone _can_ still reproduce this with Invoke 1.0 or up, please comment!