Ckan: Write a Flask and Python 3 migration guide for extensions

[amercader]

There is a minimal page in the docs that needs to be expanded. Originally it was just targeted at Flask specific issues but I think it makes sense to expand the scope and include wider Python 3 support issues.

Things it should cover:

• Adding Flake8 tests / formatters to Travis tests to prepare code for Python 3 support, provide examples and update the extension template.
• Use the toolkit / ckantoolkit for imports
• url_for changes
• IMapper + BaseController migration to IBlueprint + Blueprints

Additionally we should update the toolkit reference to mark properties that will be deprecated (like BaseController).

Approach

Write the docs :) This is likely going to be done alongside the actual migration of some extensions.

Story points

3

[smotornyuk]

_Just keeping notes in one place_

1) create py3 branch
2) move plugin.py into the plugin/__init__.py. This won't break anything, as import path for plugin won't be changed. But you need to update relative paths. enable absolute_imports for python modules, and check, whether you registering any new directories(templates, resources, translations). For example, you'll likely make following change:

p.toolkit.add_template_directory(config, "templates")
# -> 
p.toolkit.add_template_directory(config, "../templates")

3) remove the version-dependent code from the original plugin(IMapper/IBlueprint)
4) create plugin/flask_plugin.py and plugin/pylons_plugin.py. Each of those files contains version-specific plugin-mixin(just ordinary CKAN plugin). In our example case, flask_plugin adds new routes via blueprint, and pytlons_plugin is doing it via before_map from IMapper.
5) Depending on current CKAN version, import one of the sub-plugins into main plugin

try:
    tk.requires_ckan_version("2.9")
except CkanVersionException:
    from ckanext.my_ext.plugin.pylons_plugin import MixinPlugin
else:
    from ckanext.my_ext.plugin.flask_plugin import MixinPlugin

6) Add version-specific plugin as a mixin to the main plugin class

class MyPlugin(MixinPlugin, p.SingletonPlugin):

7) If plugin registers CLI commands, add cli.py with relevant logic. Follow #5112 as manual for registering CLI commands.
8) Create webassets.yml for resources (there is no documentation for webassets yet:( )
9) Optionally. Create pylons/flask subfolders for templates. Or, at least, replace resource tags with asset tags and make sure you are not using any global variables in templates and use correct url_for arguments(controller-action for <2.8, and dot-named routes for newer CKAN versions).
10) Futurize code! (pip install future; futurize ckanext -w) . Not sure about this step, but extensions should work well with different versions of CKAN and python, so using if + six may be too cumbersome and there will be more chances to miss something. In addition, it(theoretically) will be much easier to make clean-up for future, when the extension's maintainer decides to drop Python2 support.

[davidread]

Let's move this content to a wiki for a couple of months. As we blitz the migration of extensions, we'll no doubt need to update with a lot of small things, work in parallel, want to suggest changes informally etc. (As discussed on gitter today, with some agreement)

[smotornyuk]

Done - https://github.com/ckan/ckan/wiki/Migration-guide-for-extensions

[amercader]

I made a final review and updated several sections and the overall formatting of the wiki page, we can keep updating it but it's a good initial resource for contributors:

https://github.com/ckan/ckan/wiki/Python-3-migration-guide-for-extensions

[davidread]

Looks great, many thanks @amercader