Docker-py: Pipeing using exec_create, exec_start stdin, stdout

[carlskii]

If I use the docker CL this works perfectly for me.

echo "hello" | docker exec -i $3 sh -c 'cat >/text.txt'

Now I want to use docker-py and have this so far:

ex = cli.exec_create(container='nginx-ssl', cmd='cat >/carlskii.txt')
print cli.exec_inspect(ex)
ls = cli.exec_start(exec_id=ex["Id"], tty=True)

So how do I pass the "hello" or whatever data into the exec command so that it replicates the CL command ?

Incidentally this also works perfectly locally:

p = Popen(('docker', 'exec', '-i', 'nginx-ssl', 'sh', '-c', 'cat >text.txt'), stdin=subprocess.PIPE)
p.communicate('hello\n')
p.stdin.close()

[TomasTomecek]

You need to pass stdin=True to exec_create, then get socket with exec_start(..., socket=True) and feed the data via that socket.

Code in dockerpty could help you a bit: https://github.com/d11wtq/dockerpty/blob/81361cd90607a45298806db6968b45e75421ff2f/dockerpty/pty.py#L212

[carlskii]

Thanks for the update - Now things are getting complicated. I've no idea where to start here. Any chance of a simple example ?

Alternatively is there a better way to push data into a running container ?

[TomasTomecek]

Yep. docker cp -- client.copy() See comment from @shin- below.

[carlskii]

Thats what I thought but has this not been deprecated ?

Client.copy() is deprecated for API version >= 1.20, ' 'please use get_archive() instead',

[shin-]

use Client.put_archive instead :)

[zbyte64]

When I try to write to the resulting socket I get: File or stream is not writable. I made sure to set stdin=True when specifying exec_create.

[TomasTomecek]

@zbyte64 could you please post a complete reproducer and your environment (ideally in a new issue)? it's hard to help with so few information

[zbyte64]

Sorry, let me clarify. To write to stdin one calls attach_socket with the appropriate flags:

socket = client.attach_socket(container_id, {
    'stdin': 1,
    'stream': 1,
})

But the socket you get back cannot be written to. Instead you must use os.write to send data to stdin:

os.write(socket.fileno(), b'echo hello world')

This usage seems awkward. But If you use TLS then you instead do:

socket.send(b'echo hello world')

[TomasTomecek]

You're correct. dockerpty handles such situation like this: https://github.com/d11wtq/dockerpty/blob/f8d17d893c6758b7cc25825e99f6b02202632a97/dockerpty/io.py#L150

It would be very convenient if docker-py provided helper functions for this usecase.

[carlskii]

@zbyte64 Could you provide a complete example here. I'm still struggling to get this to work.

Going back to my original question, I need to mimic this but using Python:

echo "hello" | docker exec -i $3 sh -c 'cat >/text.txt'

[monperrus]

This may be the complete example you're looking for: https://stackoverflow.com/questions/46521166/how-to-write-to-stdin-in-a-tls-enabled-docker-using-the-python-docker-api

[sbrki]

@zbyte64 using
os.write(socket.fileno(), b'echo hello world') doesn't work for me. All that ever gets passed to the containers stdin , for some to me unknown reason, is \x01.

I'm literally using a copypasta of your (above) code, and inside of the container there is a simple c program that prints whatever scanf("%s",&s)captures. All that ever gets printed is \x01.

I also see that this is a relatively old issue, have you given any thoughts to fixing/implementing this in the docker-py API?

[james-callahan]

I ended up using:

sock = container.attach_socket(params={"stdin": 1, "stdout": 1, "stderr": 1, "stream": 1})
sock._writing = True
def write(sock, str):
    while len(str) > 0:
        written = sock.write(str)
        str = str[written:]
write(sock, b"data")

This isn't too bad, but if the stdin parameter is true could you return a writable socket instead of needing me to dig around in internals (sock._writing)?

[KevinTHU]

@zbyte64 you are using python3 right? look at this code https://github.com/docker/docker-py/blob/8acd2c3d084805df0bbeda4697c8ad7215ff7247/docker/api/client.py#L316

elif six.PY3:
            sock = response.raw._fp.fp.raw
            if self.base_url.startswith("https://"):
                sock = sock._sock

response.raw._fp.fp is a file-like object, and it is maked by Python36/Lib/http/client.py with self.fp = sock.makefile("rb"), so the fp.raw is only readable, you can not use sock.write, this will cause exception File or stream is not writable., so if you want to read and wirte, you need to do as https way above, using sock._sock is ok. you can have a try, but of cause you need to do it as a socket, use sock.recv and sock.send to read and write.

BTW: @shin- why for python3, the sock for http is fp.raw ? just for safe? PTAL, thanks a lot.