Products.cmfplone: Port robot-server to Python 3

[pbauer]

robot-server is not yet ported to python 3 but is a sometimes required for debugging robot-tests.

It depends on Signals (https://github.com/zopefoundation/Zope/tree/2.13/src/Signals) which was moved from Zope to ZServer and is not ported to Python 3.

[davisagli]

What does it do that can't be done when using the Zope testrunner?

[pbauer]

I'm not sure but I was under the impression that it was best practive to use robot-server when debugging robot-tests (with reload).

I wanted to debug the unfound keyword enable_autologin_as:

[ ERROR ] Error in file '/home/jenkins/workspace/plip-py3/src/plone.app.robotframework/src/plone/app/robotframework/tests/test_i18n_library.robot': Getting keyword names from library 'Remote' failed: Calling dynamic method 'get_keyword_names' failed: Connecting remote server at http://localhost:52576/plone/RobotRemote failed: <ProtocolError for localhost:52576/plone/RobotRemote: 404 Not Found>

Could that be related to the change to default to picking a free port?

I found that if I disable reload (e.g. by setting HAS_RELOAD to false when ZServer is not importable) the robot-server starts butI cannot connect:

Error in file '/Users/pbauer/workspace/plip/src/Products.CMFPlone/Products/CMFPlone/tests/robot/test_actionmenu.robot': Getting keyword names from library 'Remote' failed: Calling dynamic method 'get_keyword_names' failed: InvalidURL: nonnumeric port: 'None'

When hard-coding a port I get Connecting remote server at http://localhost:55001/plone/RobotRemote failed: [Errno 61] Connection refused

[pbauer]

@Rotonen do you have any idea about the cause of
[ ERROR ] Error in file '/home/jenkins/workspace/plip-py3/src/plone.app.robotframework/src/plone/app/robotframework/tests/test_i18n_library.robot': Getting keyword names from library 'Remote' failed: Calling dynamic method 'get_keyword_names' failed: Connecting remote server at http://localhost:52576/plone/RobotRemote failed: <ProtocolError for localhost:52576/plone/RobotRemote: 404 Not Found>
(e.g. https://jenkins.plone.org/view/PLIPs/job/plip-py3/924/consoleText)

[Rotonen]

No.

[davisagli]

Ah, reload is a fine answer. I don't trust plone.reload, so didn't think about that.

From my debugging, the missing RobotRemote is not related to the port changes but has to do with zope test layer ordering issues -- so you're definitely going to have to run with the zope testrunner to figure this one out. For some reason that I haven't found yet, one of the test layers in plone.app.robotframework tries to reuse the same base layer that added RobotRemote but it is no longer there.

[Rotonen]

@davisagli oh, that actually fully explains why we have different teardown issues in:

1) with the old zope.testrunner in a sequential run
2) with the new zope.testrunner, which optimised layer sharing, and thus the run order, in a sequential run
3) with parallel runs as they do not share layer setups or teardowns between executors, but just run the layers independent of each other

This is probably not the only symptom from layer setup, teardown and test/class/layer leak issues in regards to Robot either.

[pbauer]

@davisagli thanks for fixing the issue with RobotRemote in https://github.com/plone/plone.app.robotframework/pull/94!
In Python 3 we're now down to only 4 failing robot-tests and no failing python-tests!

In Python 2 chrome does not start:

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/lib/chromium-browser/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.40,platform=Linux 4.15.0-36-generic x86_64)

[Rotonen]

That's probably the CI job not setting $PATH correctly.

[pbauer]

Thanks! I adapted the config to be almost the same as the py3 job and now they seem to run.

[pbauer]

https://jenkins.plone.org/view/PLIPs/job/py3_on_py2/ is green \o/

[ale-rt]

Please upload a picture of your happy face :)
And also tweet something!
This is a great achievement and a good prize for all the effort you invested. Congratulation to you and everybody who made this possible :)

[davisagli]

I was momentarily excited that plip-py3 is green too, but it turns out that build only ran 145 tests... 🤔 Looks like you temporarily edited the build to only run the robot tests, @pbauer?

[Rotonen]

Yes, it was temporarily switched to have a quick run which screenshots the failures, but it seems there is some layer leak from a non-Robot layer breaking those tests.

[pbauer]

Yes, that was an experiment to get the screenshots earlier to debug the failing p.a.m tests. But only running robot-tests they passed.

[pbauer]

They fail and the screenshot is inconclusive. Maybe another test removes the permissions to see the language-independent switch and does not get torn down.
https://jenkins.plone.org/view/PLIPs/job/plip-py3/932/robot/report/robot_log.html#s1-s26-s2-t1

[davisagli]

I debugged and found that the getAdapters call in field/edit.py in plone.schemaeditor does not actually find the IFieldEditorExtender adapter that it should. But I see the adapter factory in getSiteManager().registeredAdapters(), so it's not obvious what is going wrong with the lookup.

If you remove the provideAdapter calls in plone.app.multilingual.dx.schemaeditor, you can get the error even when running the test in isolation. Those provideAdapter calls really shouldn't be there anyway.

[pbauer]

Yes, I looked hard at that file and especially at these two calls and learned nothing except that ZCA can create headaches. Ping me as soon as you figure out a fix 😄

[pbauer]

By running ROBOT_BROWSER=chrome ./bin/test --all -s Products.CMFPlone -s plone.app.multilingual -t Products.CMFPlone -t 'test_schemaeditor.robot' -t '!CMFPlone*.robot' I found that the issue is at least not in one of the python-tests in CMFPlone.

[pbauer]

It is in plone.schemaeditor. With ROBOT_BROWSER=chrome ./bin/test --all -s plone.schemaeditor -s plone.app.multilingual -t plone.schemaeditor -t 'test_schemaeditor.robot' I can reproduce it.

[davisagli]

I won't have a chance to try this until later but I suspect we should convert plone/schemaeditor/tests/tests.py to use plone.testing

[pbauer]

Damn! False alarm, it's not plone.schemaeditor (I commented out the two provideAdapter calls and forgot to save after undoing that...)
I'll try a couple more packages but in half an hour I'll have to call it quits.

[pbauer]

plone.app.users it is.

[pbauer]

It seems to be a a mix of the two layers plone.testing.z2.INTEGRATION_TESTING and plone.app.users.testing.PLONE_APP_USERS_FUNCTIONAL_TESTING

When running the following three tests it fails:
ROBOT_BROWSER=chrome ./bin/test --all -s plone.app.users -s plone.app.multilingual -t 'test_schemaeditor.robot' -t test_uuid_disabled_email_as_login_enabled_no_full_name_uppercase -t test__init__userid_in_request_form_for_manager

The tests themselves are not the issue, when I change test_uuid_disabled_email_as_login_enabled_no_full_name_uppercase and test__init__userid_in_request_form_for_manager to return True the issue in the robot-test still appears.

I guess the teardown of layer for TestAccountPanelSchemaAdapter is the issue but I was not able to pinpoint it now.

[pbauer]

With https://github.com/plone/plone.app.multilingual/commit/6af53c49cedd4cf076204b59f7fcd77de41fa214 I added a ugly hack to redo the provideAdapter in the test-setup. That call indeed needs to be replaced.

[davisagli]

And... 🍏📗💚🥗!

Ship it! 🚢:shipit:

[pbauer]

Sadly, the py3_on_py2 job is not really green. For some reaons it ran 10000 less tests than it should have 😭. There are some failures in plone.restapi that do not happen locally (not again 😱).
And: Zope 4.0b6 needs to be merged. So shipping has to wait a little 😄

[pbauer]

Let's track remaining issues and tasks in #2592 so that this ticket can go back to be about porting robot-server to Python 3, which IMHO is not a hard requirement for 5.2a1

[ale-rt]

Using this branch:
https://github.com/plone/plone.app.robotframework/pull/98
I can do the following:

💣 ~/Code/plone/buildout.coredev.alpine (5.2) 
[ale@emily buildout.coredev.alpine]$ ss -ntupla|grep robot
tcp  LISTEN     0       128             127.0.0.1:36007           0.0.0.0:*      users:(("robot-server",pid=3860,fd=7))                                         
tcp  LISTEN     0       5               127.0.0.1:49999           0.0.0.0:*      users:(("robot-server",pid=3860,fd=8))                                         
✓ ~/Code/plone/buildout.coredev.alpine (5.2) 
[ale@emily buildout.coredev.alpine]$ ZSERVER_HOST=127.0.0.1 ZSERVER_PORT=36007 ./bin/pybot -t "*" src/Products.CMFPlone/Products/CMFPlone/tests/robot/test_controlpanel_language.robot
==============================================================================
Test Controlpanel Language                                                    
==============================================================================
Scenario: Set Site Language in the Language Control Panel             | PASS |
------------------------------------------------------------------------------
Test Controlpanel Language                                            | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/ale/Code/plone/buildout.coredev.alpine/output.xml
Log:     /home/ale/Code/plone/buildout.coredev.alpine/log.html
Report:  /home/ale/Code/plone/buildout.coredev.alpine/report.html

So the robot tests run properly for me.
The only issue I have is that I need to specify the server host and port, before I think it was alway 55001.
Now if I do not set the environment variables I see this error:

[ale@emily buildout.coredev.alpine]$ ./bin/pybot -t "*" src/Products.CMFPlone/Products/CMFPlone/tests/robot/test_controlpanel_language.robot
[ ERROR ] Error in file '/home/ale/Code/plone/buildout.coredev.alpine/src/Products.CMFPlone/Products/CMFPlone/tests/robot/test_controlpanel_language.robot': Getting keyword names from library 'Remote' failed: Calling dynamic method 'get_keyword_names' failed: InvalidURL: nonnumeric port: 'None'
==============================================================================
Test Controlpanel Language                                                    
==============================================================================
Scenario: Set Site Language in the Language Control Panel             | FAIL |
Setup failed:
InvalidArgumentException: Message: Malformed URL: http://None:None/plone is not a valid URL.
------------------------------------------------------------------------------
Test Controlpanel Language                                            | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output:  /home/ale/Code/plone/buildout.coredev.alpine/output.xml
Log:     /home/ale/Code/plone/buildout.coredev.alpine/log.html
Report:  /home/ale/Code/plone/buildout.coredev.alpine/report.html

as you can see the robotframework is trying to access http://None:None/plone

[Rotonen]

@ale-rt is the test layer setup somehow broken? That is where those are stored.

https://github.com/plone/plone.testing/blob/c7de92448022ca97764e26a96a3af7482db68bfd/src/plone/testing/zserver.py#L526-L535

[Rotonen]

Huh, did we have a rebase oopsie in Halle or did I really only do it for the FTP server?

[ale-rt]

Thanks for checking it. Tomorrow I will check where to copy that code from the FTP server class

[Rotonen]

Here's how it works on the Robot side of the stack - it just grabs them from the layer:

https://github.com/plone/plone.app.robotframework/commit/b8d9344f8d9ef1d871205b55b72641b79eda108d

[Rotonen]

No, the backport branches still have it.

https://github.com/plone/plone.testing/pull/52/files
https://github.com/plone/plone.testing/pull/55/files

What happened on the plone.testing master?

[ale-rt]

@Rotonen it seems they are there as expected: https://github.com/plone/plone.testing/blob/master/src/plone/testing/zserver.py#L439-L445
am I missing something?