Products.cmfplone: Start migrating Plone to Python 3

[pbauer]

Meta-ticket to collect todos for this task.

Initial work was documented in https://docs.google.com/document/d/1rYN817-9SjAQulpes36k1veTt4pABZRbl0T0BbuZrHA/edit#

Follow the best practice doumentation in https://github.com/plone/Products.CMFPlone/issues/2184#issuecomment-359445243 when working on this!

Tasks

• [ ] Document setup of different Python 3 versions, tox and related tools to test and maintain Python 3 compatibility.
• [ ] Migrate GenericSetup and MailHost to Python 3 [@gotcha and @witsch] (Note https://github.com/zopefoundation/Products.GenericSetup/pull/52 for a small fix that is needed.)
• [X] Update the jenkins-job for python3 (http://jenkins.plone.org/view/py3)
• [ ] Migrate packages without CMFPlone dependencies (e.g. finish migration of plone.rfc822)
• [ ] Fix all import-errors everywhere (use 2to3 and six to create backwards-compatible imports)
• [ ] Look at plone.testing and plone.app.testing if we can get the lowermost testlayers to start on python 3

Packages that were worked on regarding Python 3

• [x] AccessControl [@rudaporto] - https://github.com/zopefoundation/AccessControl/pull/48
• [x] archetypes.schemaextender [@mauritsvanrees] - https://github.com/plone/archetypes.schemaextender/pull/4
• [x] borg.localrole [@jensens] - https://github.com/plone/borg.localrole/pull/6
• [x] collective.monkeypatcher [@mauritsvanrees] - nothing to do
• [x] collective.xmltestreport [@mauritsvanrees] - nothing to do
• [x] plone.alterego [@pbauer] - nothing to do
• [x] plone.api [@rudaporto] - https://github.com/plone/plone.api/pull/394
• [x] plone.app.blob [@rudaporto, @pbauer, @davilima6] - https://github.com/plone/plone.app.blob/pull/43 & https://github.com/plone/plone.app.blob/pull/48
• [x] plone.app.caching [@b4oshany, @jensens, @davilima6] https://github.com/plone/plone.app.caching/pull/40 & https://github.com/plone/plone.app.caching/pull/41
• [x] plone.app.collection [@rudaporto] - nothing to do
• [x] plone.app.content [@ale-rt, @davilima6] - https://github.com/plone/plone.app.content/pull/143 & https://github.com/plone/plone.app.content/pull/151
• [x] plone.app.contentlisting [@rudaporto] - nothing to do
• [x] plone.app.contentmenu [@pbauer] - nothing to do
• [x] plone.app.contentrules [@pbauer] - https://github.com/plone/plone.app.contentrules/pull/35
• [x] plone.app.contenttypes [@pbauer] - https://github.com/plone/plone.app.contenttypes/pull/447
• [x] plone.app.controlpanel [@vincero] - nothing to do
• [x] plone.app.customerize [@vincero] - nothing to do
• [x] plone.app.dexterity [@vincero] - https://github.com/plone/plone.app.dexterity/pull/265
• [x] plone.app.discussion [@pbauer] - https://github.com/plone/plone.app.discussion/pull/129
• [x] plone.app.event [@pbauer] - https://github.com/plone/plone.app.event/pull/274
• [x] plone.app.folder [@pbauer] - https://github.com/plone/plone.app.folder/pull/15
• [x] plone.app.i18n [@pbauer] - nothing to do
• [x] plone.app.imaging [@ale-rt] - https://github.com/plone/plone.app.imaging/pull/34
• [x] plone.app.intid [@pbauer] - nothing to do
• [x] plone.app.iterate [@vincero] - nothing to do
• [x] plone.app.layout [@ale-rt, @jensens] - https://github.com/plone/plone.app.layout/pull/134
• [x] plone.app.layout part2 [@pbauer] - https://github.com/plone/plone.app.layout/pull/140
• [x] plone.app.lockingbehavior [@pbauer] - nothing to do
• [x] plone.app.ldap [ale-@rt] - https://github.com/plone/plone.app.ldap/pull/33
• [x] plone.app.linkintegrity - [ale-rt] https://github.com/plone/plone.app.linkintegrity/pull/61
• [x] plone.app.locales [@rudaporto] - nothing to do
• [x] plone.app.mosaic [@b4oshany] - https://github.com/plone/plone.app.mosaic/pull/390
• [x] plone.app.multilingual [@ale-rt, @jensens] - WIP https://github.com/plone/plone.app.multilingual/pull/296
• [x] plone.app.openid [@b4oshany] - https://github.com/plone/plone.app.openid/pull/12
• [x] plone.app.portlets [@ale-rt] - https://github.com/plone/plone.app.portlets/pull/104
• [x] plone.app.portlets part2 [@pbauer] - https://github.com/plone/plone.app.portlets/pull/107
• [x] plone.app.querystring [@pbauer] - https://github.com/plone/plone.app.querystring/pull/84
• [x] plone.app.redirector [@vincero] - https://github.com/plone/plone.app.redirector/pull/9
• [x] plone.app.registry [@pbauer] - https://github.com/plone/plone.app.registry/pull/29
• [x] plone.app.relationfield [@rudaporto] - nothing to do
• [x] plone.app.referenceablebehavior [@pbauer] - nothing to do
• [x] plone.app.robotframework [@ale-rt, @b4oshany, @davilima6] - https://github.com/plone/plone.app.robotframework/pull/70
• [x] plone.app.standardtiles [@b4oshany] - https://github.com/plone/plone.app.standardtiles/pull/82
• [x] plone.app.testing [@rudaporto] - nothing to do
• [x] plone.app.textfield [@b4oshany] - https://github.com/plone/plone.app.textfield/pull/26
• [x] plone.app.theming [@pbauer] - https://github.com/plone/plone.app.theming/pull/139
• [x] plone.app.upgrade [@rudaporto] - nothing to do
• [x] plone.app.users [@rudaporto] - nothing to do
• [x] plone.app.uuid [@pbauer] - nothing to do
• [x] plone.app.versioningbehavior [@ale-rt @pbauer] - https://github.com/plone/plone.app.versioningbehavior/pull/35
• [x] plone.app.vocabularies [vincero] - https://github.com/plone/plone.app.vocabularies/pull/53
• [x] plone.app.widgets [@pbauer] - https://github.com/plone/plone.app.widgets/pull/178
• [x] plone.app.workflow [@rudaporto] - nothing to do
• [x] plone.app.z3cform [@pbauer] - https://github.com/plone/plone.app.z3cform/pull/85
• [x] plone.app.z3cform part 2 [@pbauer] - https://github.com/plone/plone.app.z3cform/pull/86
• [x] plone.autoform [@pbauer] - https://github.com/plone/plone.autoform/pull/31
• [x] plone.batching [@pbauer - nothing to do
• [x] plone.behavior [@rudaporto] - nothing to do
• [x] plone.browserlayer [@pbauer] - https://github.com/plone/plone.browserlayer/pull/7
• [x] plone.cachepurging [@pbauer] - https://github.com/plone/plone.cachepurging/pull/9
• [x] plone.caching [@vincero] - nothing to do
• [x] plone.contentrules [@pbauer] - https://github.com/plone/plone.contentrules/pull/7
• [x] plone.dexterity [@pbauer] - https://github.com/plone/plone.dexterity/pull/76
• [x] plone.dexterity part 2 [@pbauer] - https://github.com/plone/plone.dexterity/pull/77
• [x] plone.event [@vincero] - nothing to do
• [x] plone.folder [@vincero] - https://github.com/plone/plone.folder/pull/5
• [x] plone.formwidget.namedfile - [@ale-rt, @pbauer] - https://github.com/plone/plone.formwidget.namedfile/pull/30
• [x] plone.formwidget.recurrence [@pbauer] - https://github.com/plone/plone.formwidget.recurrence/pull/23
• [x] plone.i18n [@pbauer] - https://github.com/plone/plone.i18n/pull/19
• [x] plone.indexer [@vincero] - nothing to do
• [x] plone.intelligenttext - nothing to do
• [x] plone.keyring [@pbauer] - nothing to do
• [x] plone.locking [@pbauer] - https://github.com/plone/plone.locking/pull/15
• [x] plone.memoize [@vincero] - nothing to do
• [x] plone.namedfile [@ale-rt] - https://github.com/plone/plone.namedfile/pull/60
• [x] plone.namedfile part 2 [@pbauer] - https://github.com/plone/plone.namedfile/pull/61
• [x] plone.namedfile part 3 [@pbauer] - https://github.com/plone/plone.namedfile/pull/62
• [x] plone.outputfilters [@pbauer] - https://github.com/plone/plone.outputfilters/pull/26
• [x] plone.portlet.collection [@vincero] - nothing to do
• [x] plone.portlet.static [@vincero] - https://github.com/plone/plone.portlet.static/pull/15
• [x] plone.portlets [@pbauer] - nothing to do
• [x] plone.protect [@vincero] - https://github.com/plone/plone.protect/pull/70
• [x] plone.registry [@gogobd] - nothing to do
• [x] plone.resource [@gogobd] - https://github.com/plone/plone.resource/pull/22
• [x] plone.resourceeditor [@pbauer] - https://github.com/plone/plone.resourceeditor/pull/20
• [x] plone.rfc822 [@ale-rt, @pbauer, @davilima6] - WIP https://github.com/plone/plone.rfc822/pull/4
• [x] plone.scale [@pbauer] - nothing to do
• [x] plone.schema [@vincero] - nothing to do
• [x] plone.schemaeditor [@pbauer] - https://github.com/plone/plone.schemaeditor/pull/59
• [x] plone.session [@vincero] - https://github.com/plone/plone.session/pull/10
• [x] plone.stringinterp [@pbauer] - nothing to do
• [x] plone.subrequest [@pbauer] - https://github.com/plone/plone.subrequest/pull/16
• [x] plone.supermodel [@pbauer, @b4oshany, @davilima6] -
  https://github.com/plone/plone.supermodel/pull/21
• [x] plone.synchronize [@vincero] - nothing to do
• [ ] plone.testing [@rudaporto, @pbauer, @icemac, @loechel] - https://github.com/plone/plone.testing/pull/48
• [x] plone.theme [@pbauer] - nothing to do
• [x] plone.transformchain [@vincero] - https://github.com/plone/plone.transformchain/pull/7
• [x] plone.uuid - nothing to do
• [x] plone.z3cform [@pbauer, @davilima6, @ale-rt] - https://github.com/zopefoundation/plone.z3cform/pull/11, https://github.com/zopefoundation/plone.z3cform/pull/12
• [x] plonetheme.barceloneta - nothing to do
• [ ] Products.Archetypes [@pbauer] - (keep your hands off this)
• [ ] Products.ATContentTypes - TODO
• [x] Products.CMFCore [@rudaporto, @davilima6] - https://github.com/zopefoundation/Products.CMFCore/pull/29 & https://github.com/zopefoundation/Products.CMFCore/pull/32
• [x] Products.CMFDynamicViewFTI [@davilima6] - https://github.com/plone/Products.CMFDynamicViewFTI/pull/11
• [x] Products.CMFDynamicViewFTI part 2 [@pbauer] - https://github.com/plone/Products.CMFDynamicViewFTI/pull/12
• [x] Products.CMFEditions [@davilima6] - https://github.com/plone/Products.CMFEditions/pull/47/ https://github.com/plone/Products.CMFEditions/pull/49
• [x] Products.CMFFormController [@rudaporto, @pbauer] - https://github.com/plone/Products.CMFFormController/pull/13
• [x] Products.CMFPlacefulWorkflow [@davilima6] - https://github.com/plone/Products.CMFPlacefulWorkflow/commit/b1746ce9637ad37082637585aa23c255aadd1add
• [x] Products.CMFUid part2 [@pbauer] - https://github.com/zopefoundation/Products.CMFUid/pull/3
• [x] Products.CMFPlone [@pbauer] - https://github.com/plone/Products.CMFPlone/pull/2181
• [x] Products.CMFPlone part 2 [@pbauer] - https://github.com/plone/Products.CMFPlone/pull/2277
• [x] Products.CMFQuickInstallerTool [@mauritsvanrees] - https://github.com/plone/Products.CMFQuickInstallerTool/pull/17
• [x] Products.contentmigration [@davilima6] - https://github.com/plone/Products.contentmigration/pull/17 & https://github.com/plone/Products.contentmigration/pull/18
• [x] Products.DateRecurringIndex [@vincero] - nothing to do
• [x] Products.DCWorkflow [@davilima6] - https://github.com/zopefoundation/Products.DCWorkflow/pull/9
• [x] Products.DiffTool [@davilima6] - https://github.com/plone/Products.CMFDiffTool/pull/28
• [x] Products.ExtendedPathIndex [@vincero] - nothing to do
• [x] Products.ExternalEditor [@davilima6] - https://github.com/zopefoundation/Products.ExternalEditor/pull/8 & https://github.com/zopefoundation/Products.ExternalEditor/pull/9
• [x] Products.ExternalMethod [@vincero] - nothing to do
• [ ] Products.GenericSetup [@gotcha, @davisagli, @pbauer, @ale-rt] - ~https://github.com/zopefoundation/Products.GenericSetup/pull/60~ https://github.com/zopefoundation/Products.GenericSetup/pull/69
• [x] Products.i18ntestcase [@b4oshany, @davilima6] - https://github.com/plone/Products.i18ntestcase/pull/1 & https://github.com/plone/Products.i18ntestcase/pull/2
• [x] Products.Marshall [@pbauer] - https://github.com/plone/Products.Marshall/pull/5
• [ ] Products.PlacelessTranslationService [@ksuess] - WIP
• [x] Products.PlonePAS [@pbauer] - https://github.com/plone/Products.PlonePAS/pull/26
• [x] Products.PlonePAS part2 [@pbauer] - https://github.com/plone/Products.PlonePAS/pull/30
• [x] Products.PluggableAuthService [@rudaporto] - https://github.com/zopefoundation/Products.PluggableAuthService/pull/10
• [x] Products.PluginRegistry [@mauritsvanrees] - https://github.com/zopefoundation/Products.PluginRegistry/pull/4
• [x] Products.PortalTransforms [@mauritsvanrees] - https://github.com/plone/Products.PortalTransforms/pull/26
• [x] Products.PortalTransforms part2 [@pbauer] - https://github.com/plone/Products.PortalTransforms/pull/28
• [x] Products.ResourceRegistries [@mauritsvanrees] - https://github.com/plone/Products.ResourceRegistries/pull/25
• [x] Products.ResourceRegistries part2 [@pbauer] - https://github.com/plone/Products.ResourceRegistries/pull/26
• [x] Products.Sessions [@pbauer] - https://github.com/zopefoundation/Products.Sessions/pull/3
• [x] Products.SiteErrorLog [@pbauer, @icemac] - https://github.com/zopefoundation/Products.SiteErrorLog/pull/4
• [x] Products.statusmessages [@davilima6] - https://github.com/plone/Products.statusmessages/pull/8
• [x] Products.TemporaryFolder [@davilima6] - nothing to do - Released as 5.0 [@icemac]
• [x] Products.validation [@davilima6] - nothing to do
• [x] Products.ZCatalog [@davilima6] - nothing to do
• [x] Products.ZCTextIndex [@davilima6] - nothing to do
• [ ] Products.ZopeVersionControl [@rudaporto] - https://github.com/zopefoundation/Products.ZopeVersionControl/pull/1
• [x] Products.ZSQLMethods [@davilima6] - nothing to do
• [x] z3c.objpath - [@ale-rt] https://github.com/zopefoundation/z3c.objpath/pull/1
• [x] z3c.relationfield - TODO [@ale-rt] [undefined name 'cmp'] https://github.com/zopefoundation/z3c.relationfield/pull/9
• [x] zc.relation [@ale-rt] - https://github.com/zopefoundation/zc.relation/pull/3
• [x] zc.relationship [@ale-rt] - https://github.com/zopefoundation/zc.relationship/pull/1

[ale-rt]

Take care that many of that PR use the _compat module approach instead of sixer.
We should not merge them as they are, see:

• https://github.com/plone/Products.CMFPlone/pull/2181

[pbauer]

Packages that still need to be looked at for this ticket: https://public.etherpad-mozilla.org/p/Plone_on_Python_3
Many of those are already ok (like WebOb). It is simply a diff between the list above and the packages used in ./bin/test
[Edit: the relevant items of the etherpad list have been copied to the big list here above.]

[pbauer]

Some pull-requests need to replace replace _compat with six:

• [x] https://github.com/plone/plone.app.imaging/pull/34
• [x] https://github.com/plone/plone.app.linkintegrity/pull/61
• [x] https://github.com/plone/plone.app.theming/pull/137
• [x] https://github.com/plone/plone.app.z3cform/pull/85
• [x] https://github.com/plone/plone.dexterity/pull/74

Please update this list when you encounter any.

[jensens]

I close this in favour of keeping thing central at #2368

[andreasma]

I worked on the package cioppino.twothumbs and updated it locally with sixer and python-modernize and some fixes of imports. The package works in my local environment. I tried to test my pull request for python 2 to 3 with Travis-CI, but I can't get Travis to work. I need some hints how I could fix this. My current work is available here: https://github.com/collective/cioppino.twothumbs/pull/23 (I used branch python3 inside my own fork).