Flask: Flask 0.12 incompatible with Google App Engine?

[cinchent]

A very simple boilerplate Google App Engine example using Flask fails when upgrading from 0.10 to 0.12. The problem is something cryptic relating to module path resolution.

The straight-off-the-web GAE Flask Hello World example works fine when deployed to a live GAE project. The reason why this works ok is that it is so simple that it makes no use of third-party libraries outside of those that are natively installed along with Flask as a standard 'pip' install.

But when I extended this example to make the simplest use of the Google Cloud Datastore, then when the code is deployed, and a GET request is made, it fails with the following stack traceback:

Traceback (most recent call last):
File "/google/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/google/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/google/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/cinchent/flask-skel/main.py", line 8, in
from google.cloud import datastore
ImportError: cannot import name datastore

Digging deeper into this, I tried to use the GAE development application server on my local Windows box, and lo! then even the stock Hello World example fails: Flask is unable to even successfully import the native Python 'msvcrt' module from within its own Click initializer. Here's the traceback from that:

Traceback (most recent call last):
File "C:Program Files (x86)GoogleCloud SDKgoogle-cloud-sdkplatformgoogle_appenginegoogleappengineruntimewsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "C:Program Files (x86)GoogleCloud SDKgoogle-cloud-sdkplatformgoogle_appenginegoogleappengineruntimewsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "C:Program Files (x86)GoogleCloud SDKgoogle-cloud-sdkplatformgoogle_appenginegoogleappengineruntimewsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "C:CINCHFreewaveGCPappengine-flask-skeletonmain.py", line 8, in
from flask import Flask
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibflask__init__.py", line 21, in
from .app import Flask, Request, Response
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibflaskapp.py", line 26, in
from . import json, cli
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibflaskcli.py", line 17, in
import click
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibclick__init__.py", line 18, in
from .core import Context, BaseCommand, Command, MultiCommand, Group,
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibclickcore.py", line 7, in
from .types import convert_type, IntRange, BOOL
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibclicktypes.py", line 4, in
from ._compat import open_stream, text_type, filename_to_ui,
File "C:CINCHFreewaveGCPappengine-flask-skeletonlibclick_compat.py", line 164, in
import msvcrt
File "C:Program Files (x86)GoogleCloud SDKgoogle-cloud-sdkplatformgoogle_appenginegoogleappenginetoolsdevappserver2pythonsandbox.py", line 964, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named msvcrt

GAE runs the Flask webapp within a sandbox, but it provides the proper module paths to the installed modules, provided those obey a somewhat orthodox Python path resolution technique, but whatever exotic technique Flask now uses in 0.12 fails within that sandbox. (Other third-party libraries can be imported from within the sandbox just fine.)

These problems are completely absent in Flask 0.10. (Once you're past the problem of importing 'protobuf', that is, a separate but known incompatibility with GAE's internal use, which has a workaround.)

I've googled high and low for any information about what I may be missing in the Flask initialization recipe -- surely I must not be the only person to have encountered this! -- but have turned up nothing.

---

In the enclosed 'main.py', setting the DATASTORE_EXAMPLE global to False causes it to behave as the stock GAE Hello World example app, and setting it to True extends that app to attempt use of the Cloud Datastore.

bundle.zip

[cdman]

I'm no Flask/GAE expert, but here is my quick diagnosis:

• msvcrt is not available on Linux (or rather: it is only available on Windows)
• since the GAE development server tries to simulate Linux (since in production your application runs on Linux), it disallows importing msvcrt
• unfortunately click detects that it's on Windows and tries to use the windows specific functionality: https://github.com/pallets/click/blob/master/click/_compat.py#L166

I don't see any good workaround besides try/catching the import in click and not doing anything if running under GAE (and perhaps logging a warning?)

[cinchent]

Yes, it appeared to me as well that integration of 'cllck' is implicated in this problem, at least as pertains to the development application server. That module was not present in the 0.10 version, which works fine. However, the problem is not confined only to the DAS; it also manifests after code is deployed onto on the real GAE cloud server, which the first traceback attests to.

I had to freeze on flask==0.10 for my GAE app to be deliverable, and that's ok for now -- there's nothing in v0.12 I can't live without. I was simply making the creators aware of having left GAE behind in the wake of intervening version(s), in case they care. (And I presume they do...)

[davidism]

This probably needs to be reported against Click, not Flask.

[cdman]

@davidism - reported here: https://github.com/pallets/click/issues/711

[cinchent]

Well, inasmuch as flask uses click, it's certainly applicable here. Also,
I'm not convinced the problem is solely with the user of click -- witness
the same problem manifesting in the development server with other modules.

On Jan 5, 2017 10:41 PM, "Attila-Mihaly Balazs" [e-mail]
wrote:

@davidism https://github.com/davidism - reported here: pallets/click#711
https://github.com/pallets/click/issues/711

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pallets/flask/issues/2139#issuecomment-270835879, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEQgSlf3FKKCoiRn8JRHljKX6vBjbVToks5rPdQcgaJpZM4LZLnh
.

[untitaker]

I'm pretty sure it's a click issue. Click 6.7 is out, please try.

[alemag1986]

Any news regarding this topic? On Windows the same error persist using click-6.7

[davidism]

It continues to be an issue with Click then. Did you report it?

[yairohayon]

I just reported it again
https://github.com/pallets/click/issues/711

[sorgloomer]

Created App Engine issue, maybe they can push this better or provide a builtin workaround: https://issuetracker.google.com/issues/72043776