Lektor: Deploy not working from Lektor Admin on Python 3.6

[zupo]

I'm trying to deploy Lektor from the admin interface running on Python 3.6. I get the following error in the terminal:

Started build
Error on request:
Traceback (most recent call last):
  File "/private/tmp/test/lib/python3.6/site-packages/werkzeug/serving.py", line 270, in run_wsgi
    execute(self.server.app)
  File "/private/tmp/test/lib/python3.6/site-packages/werkzeug/serving.py", line 261, in execute
    write(data)
  File "/private/tmp/test/lib/python3.6/site-packages/werkzeug/serving.py", line 241, in write
    assert isinstance(data, bytes), 'applications must write bytes'
AssertionError: applications must write bytes
Finished build in 0.02 sec

These are the steps to reproduce:

$ virtualenv -p python3.6 test
$ cd test/
$ source bin/activate
$ pip install lektor
$ lektor quickstart  # answer all with "test" and Yes
$ cd test

# add the following to test.lektorproject
[servers.ghpages]
name = GHPages
enabled = yes
target = ghpages://test/test

$ lektor server

# go to http://localhost:5000/admin/root/edit
# click Deploy icon
# observe error above in the terminal

I tried other deploy options as well and I get the same error.

What else can I try? What additional information can I provide to make this easier to debug?

[nixjdm]

This was duplicated and reported on #530. Commonality is GitHub pages, but that might be a red herring. Based on the traceback seemingly not hitting Lektor python code, I'm suspicious of our React code that is driving the admin. It might be using an incorrect API call.

[lexicalbits]

Just a quick tack-on - the same error occurs with lektor-s3 and Python 3.6.5. Building and deploying from the CLI is fine.

[eljoest]

found the problem: Any output that gets sent to the client has to be encoded.

--- utils.py    2018-08-08 15:09:21.000838994 +0200
+++ lib/python3.5/site-packages/lektor/admin/utils.py   2018-08-08 17:19:44.864373577 +0200
@@ -16,7 +16,7 @@
     def new_func(*args, **kwargs):
         def generate():
             for event in chain(f(*args, **kwargs), (None,)):
-                yield 'data: %s\n\n' % json.dumps(event)
+                yield ('data: %s\n\n' % json.dumps(event)).encode()
         return Response(generate(), mimetype='text/event-stream',
                         direct_passthrough=True)
     return update_wrapper(new_func, f)

Disclaimer: I didn't confirm whether this change would break Python 2 deployment but based on unrelated experiments I'm quite sure that it'll be fine.

[nixjdm]

Great! Good job finding it @eljoest ! Do you know what changed in 3.6 though? IIRC I did try to duplicate the bug in 3.5 and couldn't - it really seemed 3.6 specific. Something changed that's not jumping out to me.

I would super appreciate a PR with a test :)

[eljoest]

I'm actually running this with Python 3.5.2. From the change I made it should always happen if there's any error and/or a call to self.fail(msg) in the publisher. Any output that gets passed to the UI (e. g. the yield line in RsyncPublisher) should produce exactly the same error, though I didn't check the latter part.

[msom]

@nixjdm can you make a release?

[nixjdm]

Yes, this is definitely release worthy. Give me a couple days and I'll have one out. :)

[nixjdm]

@msom Here you go! https://github.com/lektor/lektor/releases/tag/3.1.2