Cylc-flow: KeyError: 'CYLC_SUITE_PUBLISH_PORT' after running suites with Cylc 7 and Cylc 8

[kinow]

Describe the bug

Playing with two venv's, one with Cylc 7 and another with Cylc 8. I am quite sure I had stopped everything that was running with Cylc 8.

Then started running new suites with Cylc 7. But at one point decided to test suite five. After a while I stopped my Cylc 7 tests, and went back to another desktop workspace (XFCE), where I had the Cylc 8 terminals.

Running cylc run --no-detach five, resulted in errors on the terminal, that I ignored and simply restarted the terminals. Then source'd the venvs again, and tried to start JupyterHub and the suites.

five was running OK, but on JupyterHub I started seeing a lot of these errors:

2020-03-10 15:48:00,241 tornado.application ERROR    Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7f824cdd34e0>>, <Task finished coro=<WorkflowsManager.gather_workflows() done, defined at /home/kinow/Development/python/workspace/cylc-uiserver/cylc/uiserver/workflows_mgr.py:112> exception=KeyError('CYLC_SUITE_PUBLISH_PORT')>)
Traceback (most recent call last):
  File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/tornado/ioloop.py", line 743, in _run_callback
    ret = callback()
  File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/tornado/ioloop.py", line 767, in _discard_future_result
    future.result()
  File "/home/kinow/Development/python/workspace/cylc-uiserver/cylc/uiserver/workflows_mgr.py", line 125, in gather_workflows
    for arg in scan_args:
  File "/home/kinow/Development/python/workspace/cylc-uiserver/cylc/uiserver/workflows_mgr.py", line 117, in <genexpr>
    (reg, host, port, pub_port, self.context, CLIENT_TIMEOUT)
  File "/home/kinow/Development/python/workspace/cylc-flow/cylc/flow/network/scan.py", line 283, in get_scan_items_from_fs
    contact_data[ContactFileFields.PUBLISH_PORT],
KeyError: 'CYLC_SUITE_PUBLISH_PORT'

Haven't investigated what could be the problem. But thought it was worth creating an issue just in case someone else has a similar error and searches for it here.

Tomorrow will re-create my virtual environments and reset my npm environment too (was using a messy branch for some tests this week). Then will update this ticket in case the issue persists. Already rm -rf ~/cylc-run/five then re-registered it, but no luck.

Bruno

Release version(s) and/or repository branch(es) affected?

Steps to reproduce the bug

Expected behavior

Screenshots

Additional context

Pull requests welcome!
This is an Open Source project - please consider contributing a bug fix
yourself (please read CONTRIBUTING.md before starting any work though).

[dwsutherland]

@kinow - CYLC_SUITE_PUBLISH_PORT is new to Cylc8 and is written in the contact file.. So it's not really a bug, more a transitional issue when using the installed workflow of 8 with 7 (or vice versa)..

I don't think we need to account for this, right? So can close?

[kinow]

@dwsutherland just learned that too after debugging the exception. My families2 suite was not running, but had a contact file generated by Cylc 7, which caused the traceback in the JupyterHub logs.

I think the only issue here is that if you have a great number of Cylc 7 suites, and run JupyterHub, then your logs will be cluttered with these logs.

If the error is expected, then I think we could fail silently as we are doing in other cases. WDYT @dwsutherland ?

[datamel]

This should be fixed by https://github.com/cylc/cylc-flow/pull/3509
I have added logic which skips pre cylc8 suites, as requested in the Authentication: wash up #3449 issue.
Since CYLC_SUITE_PUBLISH_PORT doesn't get set pre-cylc8 it was causing errors with traceback. I think it should be fixed by the draft pr.

[dwsutherland]

Has anyone checked the reverse? Cylc7 checking Cylc8 contact file? .. Does it matter?

[kinow]

Has anyone checked the reverse? Cylc7 checking Cylc8 contact file? .. Does it matter?

Good question. I had some Cylc 7 and Cylc 8 suites running in parallel, with both Cylc8 UI+hub+uiserver and cylc7 gui/gscan. So far I have seen only this one error, and another one where I broke my Cylc 8 after starting cylc gui five, while five was running with Cylc 8: https://github.com/cylc/cylc-flow/issues/3414

I think we will need to do some testing before users start migrating suites, or when we start writing the documentation on how to do so.

[kinow]

This should be fixed by #3509
I have added logic which skips pre cylc8 suites, as requested in the Authentication: wash up #3449 issue.
Since CYLC_SUITE_PUBLISH_PORT doesn't get set pre-cylc8 it was causing errors with traceback. I think it should be fixed by the draft pr.

That's excellent news @datamel ! I can help testing/reviewing the PR for this issue once that has been marked as ready for review.

[datamel]

Running a Cylc 8 suite and then scanning in Cylc 7, I get the following error:

I am happy to add an issue to cylc7 for forwards compatability if that is the sensible thing to do?