Click: ValueError: I/O operation on closed file in testing

[ResidentMario]

I am getting a ValueError when using CliRunner to test my CLI tool. Unfortunately I can't isolate, but I can give a reproduction:

pip install git+git://github.com/ResidentMario/datablocks.git@d6601c8321e7fb050bb62760067dc63ca4600003
pip install pytest
cd datablocks/tests
pytest cli_tests.py

Which raises the following traceback:

=================================== FAILURES ===================================
_________________________ TestLink.test_depositor_link _________________________

self = <cli_tests.TestLink testMethod=test_depositor_link>

    def test_depositor_link(self):
>       result = runner.invoke(cli.link, ["test_depositor.py", "--outputs", "['bar2.txt']"])

cli_tests.py:41: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <click.testing.CliRunner object at 0x7f2c833a6cc0>
cli = <click.core.Command object at 0x7f2c82e7a5f8>
args = ['test_depositor.py', '--outputs', "['bar2.txt']"], input = None
env = None, catch_exceptions = True, color = False, extra = {}
exc_info = (<class 'SystemExit'>, SystemExit(0,), <traceback object at 0x7f2c82e1eb08>)
out = <_io.BytesIO object at 0x7f2c82e0a5c8>, exception = None, exit_code = 0

    def invoke(self, cli, args=None, input=None, env=None,
               catch_exceptions=True, color=False, **extra):
        """Invokes a command in an isolated environment.  The arguments are
            forwarded directly to the command line script, the `extra` keyword
            arguments are passed to the :meth:`~clickpkg.Command.main` function of
            the command.

            This returns a :class:`Result` object.

            .. versionadded:: 3.0
               The ``catch_exceptions`` parameter was added.

            .. versionchanged:: 3.0
               The result object now has an `exc_info` attribute with the
               traceback if available.

            .. versionadded:: 4.0
               The ``color`` parameter was added.

            :param cli: the command to invoke
            :param args: the arguments to invoke
            :param input: the input data for `sys.stdin`.
            :param env: the environment overrides.
            :param catch_exceptions: Whether to catch any other exceptions than
                                     ``SystemExit``.
            :param extra: the keyword arguments to pass to :meth:`main`.
            :param color: whether the output should contain color codes. The
                          application can still override this explicitly.
            """
        exc_info = None
        with self.isolation(input=input, env=env, color=color) as out:
            exception = None
            exit_code = 0

            try:
                cli.main(args=args or (),
                         prog_name=self.get_default_prog_name(cli), **extra)
            except SystemExit as e:
                if e.code != 0:
                    exception = e

                exc_info = sys.exc_info()

                exit_code = e.code
                if not isinstance(exit_code, int):
                    sys.stdout.write(str(exit_code))
                    sys.stdout.write('\n')
                    exit_code = 1
            except Exception as e:
                if not catch_exceptions:
                    raise
                exception = e
                exit_code = -1
                exc_info = sys.exc_info()
            finally:
                sys.stdout.flush()
>               output = out.getvalue()
E               ValueError: I/O operation on closed file.

../../../miniconda3/envs/datablocks-dev/lib/python3.6/site-packages/click/testing.py:299: ValueError
========================== 1 failed in 37.10 seconds ===========================

This is on Ubuntu 16.04.

[blueyed]

I am seeing this also when using __import__('pdb').set_trace() somewhere inside of the runner.
This is likely related to pytest turning IO-capturing off / changing it.

[blueyed]

Reminded me of an old stash I had: https://github.com/pallets/click/pull/951.

[blueyed]

As for pytest itself I've created https://github.com/pytest-dev/pytest/issues/3344.

[blueyed]

See also https://github.com/pallets/click/issues/654 (duplicate?!).

[jesserobertson]

I hit a similar error when writing a CLI of my own & thought I'd share my fix in the hope it helps isolate where there might be a fix (and this is the top hit in Google for me).

In one of my commands I was opening stdout using click.get_text_stream - something like

with click.open_file(input) as src, click.get_text_stream('stdout') as sink:
    data = parse_input()
    sink.write(json.dumps(data) + '\n')

...and I think the closing of the sink stream at the end of the context was killing the mocked version of stdout in the test?

Anyway, the workaround was to shift to click.echo which is much better behaved (and doesn't require the context manager). Hopefully this helps someone else (I certainly wasted a fair bit of time working it out!).

[glemaitre]

I got the issue recently as well. It was due to the use of a logger within the function.
Using click.echo can as well solve the issue if the logging was to only display some info to the user.

[thomasleveil]

The workaround I found is to write my pytest function with the caplog fixture and making sure no logger will produce any message.

def test_xxxx(caplog):
    caplog.set_level(100000)  
    ...

[cdleonard]

I recently ran into this and created a repo which reproduces the issue standalone: https://github.com/cdleonard/click-pytest-issue

The problem seems to be that the stdio capturing of pytest and click does not mix well. My workaround is to just avoid CliRunner and call main() directly because pytest caplog/capsys already covers everything.

Maybe there could be an option to disable the stdio isolation feature of CliRunner?