Python: Losing the "new line character" when use watch stream

[mikai21]

What happened (please include outputs or screenshots):

I lost the new line character when use watch stream like below.
If not using the follow parameter, then fine. But I want to use stream with new line character.

Use

api = client.CoreV1Api(api_client=self.client)
w = watch.Watch()
return w.stream(api.read_namespaced_pod_log, name=pod_name, namespace=namespace, follow=True)

Output

10.216.74.1 - [26/Sep/2019:12:22:36 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612814"10.216.74.1 - [26/Sep/2019:12:22:46 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612838"10.216.74.1 - [26/Sep/2019:12:22:56 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612866"10.216.74.1 - [26/Sep/2019:12:23:06 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612824"

What you expected to happen:

I want to keep the new line

10.216.74.1 - [26/Sep/2019:12:22:36 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612814"
10.216.74.1 - [26/Sep/2019:12:22:46 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612838"
10.216.74.1 - [26/Sep/2019:12:22:56 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612866"
10.216.74.1 - [26/Sep/2019:12:23:06 +0900] "GET /_hcheck.hdn HTTP/1.1" 200 612824"

How to reproduce it (as minimally and precisely as possible):

api = client.CoreV1Api(api_client=self.client)
w = watch.Watch()
return w.stream(api.read_namespaced_pod_log, name=pod_name, namespace=namespace, follow=True)

Anything else we need to know?:

How about fix like this?

watch.py

        timeouts = ('timeout_seconds' in kwargs)
        while True:
            resp = func(*args, **kwargs)
            try:
                for line in iter_resp_lines(resp):
                    yield self.unmarshal_event(line, return_type) + '\n' # <- here!
                    if self._stop:
                        break

Environment:

• Kubernetes version (kubectl version): 1.15
• OS (e.g., MacOS 10.13.6): Mac 10.14.6(18G95)
• Python version (python --version) python 3.7.3
• Python client version (pip list | grep kubernetes) 10.0.1

[roycaihw]

/assign

[roycaihw]

we had a test case for watching logs with newline: https://github.com/kubernetes-client/python-base/pull/93/files

I wonder if it's some usage not being well-documented. Could you check if you're using the API differently from the test?

(now I look back and feel we should separate the client-side implementation for watch and logs. They share similar stream concepts, but the client-side handling can be different)

[mikai21]

You're right! The watch object don't need any separator because a generator is already iterable.
Thank you. I got insight from your test case.

Actually, My use case was below. For the log stream viewer.

server (flask for stream)

    api = client.CoreV1Api(api_client=self.client)
    w = watch.Watch()
    st = w.stream(api.read_namespaced_pod_log, name=pod_name, namespace=namespace, follow=True)
    return flask.Response(st)

client (web ajax)

        fetch(url, { method: 'GET', headers })
            .then((r) => {
                const reader = r.body.getReader()
                this.setState({ text: '' })
                reader.read()
                    .then(function processText({ done, value }) {
                        if (done || context.state.stop) {
                            reader.releaseLock()
                            console.log('done')
                            return
                        }
                        context.setState({ text: context.state.text + decoder.decode(value) })
                        return reader.read().then(processText)
                    })
            })

I resolve the problem like this

server (flask for stream)

    api = client.CoreV1Api(api_client=self.client)
    w = watch.Watch()
    st = w.stream(api.read_namespaced_pod_log, name=pod_name, namespace=namespace, follow=True)
    def generator_wrapper():
        for g in st:
            yield g + '\n'

    return flask.Response(generator_wrapper())