Cylc-flow: PoC - GraphQL endpoint Design & Implementation

Created on 3 Dec 2018 · 38Comments · Source: cylc/cylc-flow

Note: this is a Proof of Concept

I thought it would be a good idea to present this mock pull-request to provide insight and provoke discussion on the capability/design/implementation of GraphQL with Cylc, before the looming architectural decisions in the next workshop.

I provided an informal presentation to Bruno & Hilary a month ago, but was tied up with operations work until last week, so have found time to do this now.. It is a Work in Progress, happy to have more input.

GraphQL is agnostic to the server implementation, however, as Cherrypy does not have support (Tornado may have only recently added support via 3rd party) for GraphQL/Graphene (a python implementation of GraphQL) I chose Flask, with it's Flask-GraphQL extension (which includes the GraphiQL interface)..
I paired this with gevent for the purposes of this PoC; being at least as performant as Tornado, but easier to implement, leaving less of a footprint on Cylc's code. It also features web-socket capability, and proven to work with subscription type GraphQL queries (althoughonly HTTP is implemented in this branch (so far)).

The old REST endpoints remain in place, and I haven't migrated anything (gui or httpclient.py), to using this. I focused on the most important feature; mapping the tree to n depth with node data in one query, which will satisfy our requirement for a data driven web-gui (as @oliver-sanders outlined). The three main files where all the magic happens is GraphQL resource definition network/schema.py, the resolver filter functions scheduler.py, and the data cache created in state_summary_mgr.py.

I'll run through some examples, and encourage you to have a play ! :smiley:

The extra requirements:

Python:Flask-GraphQL (any).......................................................FOUND (?)
Python:Flask (any)...........................................................FOUND (1.0.2)
Python:Flask-HTTPAuth (any)..................................................FOUND (3.2.4)
Python:gevent (any)..........................................................FOUND (1.3.6)
Python:graphene (any)........................................................FOUND (2.1.3)

If you start a suite and visit the endpoint '/graphql' from your browser, i.e:
'https://niwa-35595lvm.niwa.local:43005/graphql' (using cylc:passphrase credentials)
you'll be presented with an interface to discover and query your suite. Enter the following query, or even start typing it in (there's auto complete, drop down info available), but you can include or exclude as many fields as you desire;

{
  globalInfo {
    suite
    owner
    host
    title
    description
    url
    group
    reloading
    lastUpdated
    status
    runMode
    newestRunaheadCyclePoint
    newestCyclePoint
    oldestCyclePoint
    stateTotals{
      held
      queued
      ready
      waiting
      submitted
      submitFailed
      submitRetrying
      succeeded
      failed
      retrying
      expired
      running
      runahead
    }
    treeDepth
  }
}

then run it (ctrl+enter), and you'll see the result:

Although you can always use curl :wink:
curl -v -s -u cylc:$(cat /home/sutherlanddw/cylc-run/baz/.service/passphrase) --digest --cookie-jar cookietemp --anyauth --insecure --header "Content-Type: application/graphql" --data 'query{allTasks{edges{node{name label state}}}}' 'https://niwa-35595lvm.niwa.local:43005/graphql'
The documentation is very useful for discovering all the available data and filter fields.
Now, we could have a flat structure where we query all the tasks, and I've added some filters in addition to the usual task.point:state, you can include a list of these items or the converse exid & exitems, there is also states list and depth (aka node_depth, range from zero to specified):

{
  allTasks(id: "[fqb]*.2017*", states: ["succeeded","waiting"], depth: 2) {
    edges {
      node {
        id
        name
        label
        state
        title
        description
        URL
        spawned
        submittedTime
        startedTime
        finishedTime
        meanElapsedTime
        host
        jobHosts{
          submitNum
          jobHost
        }
        outputs {
          submitted
          submitFailed
          started
          failed
          succeeded
          expired
        }
        nodeDepth        
      }
    }
  }
}

{
  "data": {
    "allTasks": {
      "edges": [
        {
          "node": {
            "id": "UUxUYXNrOmJhYS4yMDE3MDIwMVQwMDAwKzEz",
            "name": "baa",
            "label": "20170201T0000+13",
            "state": "succeeded",
            "title": "",
            "description": "some task baa",
            "URL": "",
            "spawned": true,
            "submittedTime": null,
            "startedTime": null,
            "finishedTime": null,
            "meanElapsedTime": 10,
            "host": "localhost",
            "jobHosts": [
              {
                "submitNum": 1,
                "jobHost": "niwa-35595lvm.niwa.local"
              }
            ],
            "outputs": {
              "submitted": true,
              "submitFailed": false,
              "started": true,
              "failed": false,
              "succeeded": true,
              "expired": false
            },
            "nodeDepth": 1
          }
        },
        {
          "node": {
            "id": "UUxUYXNrOnF1eC4yMDE3MDEwMVQwMDAwKzEz",
            "name": "qux",
            "label": "20170101T0000+13",
            "state": "succeeded",
            "title": "Some Top family",
            "description": "some task qux",
            "URL": "",
            "spawned": true,
            "submittedTime": null,
            "startedTime": null,
            "finishedTime": null,
            "meanElapsedTime": 20,
            "host": "localhost",
            "jobHosts": [
              {
                "submitNum": 1,
                "jobHost": "niwa-35595lvm.niwa.local"
              }
            ],
            "outputs": {
              "submitted": true,
              "submitFailed": false,
              "started": true,
              "failed": false,
              "succeeded": true,
              "expired": false
            },
            "nodeDepth": 2
          }
        },
        {
          "node": {
            "id": "UUxUYXNrOmJhYS4yMDE3MDEwMVQwMDAwKzEz",
            "name": "baa",
            "label": "20170101T0000+13",
            "state": "succeeded",
            "title": "",
            "description": "some task baa",
            "URL": "",
            "spawned": true,
            "submittedTime": null,
            "startedTime": null,
            "finishedTime": null,
            "meanElapsedTime": 10,
            "host": "localhost",
            "jobHosts": [
              {
                "submitNum": 1,
                "jobHost": "niwa-35595lvm.niwa.local"
              }
            ],
            "outputs": {
              "submitted": true,
              "submitFailed": false,
              "started": true,
              "failed": false,
              "succeeded": true,
              "expired": false
            },
            "nodeDepth": 1
          }
        }
      ]
    }
  }
}



md5-e831183700ee001042664572702adcba



{
  allTasks(states: ["succeeded", "waiting"], first: 2, after: "YXJyYXljb25uZWN0aW9uOjA=") {
    edges {
      node {
        name
        label
        state
        title
        nodeDepth
      }
    }
    pageInfo{
      hasPreviousPage
      hasNextPage
      startCursor
      endCursor
    }
  }
}



md5-7cd4c0ff76ac2fb9cfcb4385b6d25782



{
  "data": {
    "allTasks": {
      "edges": [
        {
          "node": {
            "name": "baa",
            "label": "20170201T0000+13",
            "state": "succeeded",
            "title": "",
            "nodeDepth": 1
          }
        },
        {
          "node": {
            "name": "foo",
            "label": "20170101T0000+13",
            "state": "succeeded",
            "title": "Some Top family",
            "nodeDepth": 4
          }
        }
      ],
      "pageInfo": {
        "hasPreviousPage": false,
        "hasNextPage": true,
        "startCursor": "YXJyYXljb25uZWN0aW9uOjE=",
        "endCursor": "YXJyYXljb25uZWN0aW9uOjI="
      }
    }
  }
}



md5-1ba432400751be4f40710b09f8522190



query allFamilies($vstates: [String]){
  allFamilies(states: $vstates ){
    edges{
      node{
        name
        label
        tasks(states: $vstates) {
          edges {
            node {
              name
              label
              state
            }
          }
        }
        families(states: $vstates) {
          edges {
            node {
              name
              state
            }
          }
        }
        parents{
          edges{
            node{
              name
            }
          }
        }
      }
    }
  }
}



md5-7cd4c0ff76ac2fb9cfcb4385b6d25782



{
  "vstates": ["held", "succeeded"]
}



md5-4c00aa8f42908047e5cc0f7eb8f16d7a



query allFamilies($vstates: [String], $ndepth: Int){
  allFamilies(states: $vstates, depth: $ndepth, items: ["root.*"]){
    edges{
      node{
        name
        label
        tasks(states: $vstates, depth: $ndepth) {
          edges {
            node {
              name
              state
            }
          }
        }
        families(states: $vstates, depth: $ndepth) {
          edges {
            node {
              name
              state
              tasks(states: $vstates, depth: $ndepth) {
                edges {
                  node {
                  name
                  state
                  }
                }
              }
              families(states: $vstates, depth: $ndepth) {
                edges {
                  node {
                    name
                    state
                    tasks(states: $vstates) {
                      edges {
                        node {
                          name
                          state
                        }
                      }
                    }
                    families(states: $vstates, depth: $ndepth) {
                      edges {
                        node {
                          name
                          state
                          tasks(states: $vstates, depth: $ndepth) {
                            edges {
                              node {
                                name
                                state
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}



md5-7cd4c0ff76ac2fb9cfcb4385b6d25782



{
  "vstates": ["held", "succeeded", "waiting"],
  "ndepth": 4
}

The query could just be template'd out from the max tree depth given in the globalInfo query.. BTW it doesn't matter how deep your query is, the data will fill it out as far as it can (according to filters or w/e).

So, that's where I'm at, but there's not reason why we can't push forward with the GraphQL development.. There's still a lot I haven't tried yet, but just this alone is enough to convince me of it's utility...

POC

Source

dwsutherland

Most helpful comment

Update: I've added all the Mutations.

Some of the commands are redundant for the gui:
ping task - information available in taskProxies query (exists and running)
ping suite - information available in suiteInfo query.
remove_cycle - just remove task proxies with glob for name (taskActions).
put_message - put_messages does the same and more (no need for the back compat).

The task actions have been collated into on mutation, i.e.:

(note the use of aliases to use the same mutation multiple times in one request)

TODO for mutation & queries:

Revert task outputs, as the messages may be bespoke
Revisit the dependency graph data-provision/query
Pagination

NEXT: Implementing for Cylc8 using Tornado web sockets, and add subscriptions.

dwsutherland on 13 Mar 2019

🎉3

All 38 comments

This is a re-posting of the pull request:
https://github.com/cylc/cylc/pull/2873

dwsutherland on 3 Dec 2018

@dwsutherland , having to learn a bit more of Relay in order to get it working with Vue. Looking at other Vue projects, looks like most devs are adopting Vue + Apollo.

Q1/ do you know if the only part that we are using of Relay is pagination?
Q2/ do you think it would be too much to implement pagination for Apollo instead?

The reasons for Q2 are for it being framework agnostic, but also due to the better integration with Vue (though the Vue + Relay might work, just need more time, but the project is maintained by 1 dev I think...).

I think Graphene offers easy support for Relay, which is nice. But if that's not too complicated to replace.... :grimacing:

kinow on 4 Dec 2018

As we will likely use Tornado, we can combine

In the Tornado branch also had to change the scheduler.py due to the way Tornado's main loop works. So that would have to replicated (and tested) with GraphQL.

The Tornado + GraphQL seems pretty simple, and quite similar to FLask + GraphQL I think.

kinow on 4 Dec 2018

@dwsutherland it worked! :tada:

Server with Tornado + graphene. Of course querying won't return anything as there's nothing in the scheduler.py. But I think if we go with the approach you suggested (i.e. Python3 => Tornado), then perhaps it would be just a matter of you replicating some changes from your flask branch over the new work.

Minor note; the packaging might be useful... tornado has 2 dependencies that I added manually here. But tornado-graphql has another three that would have to be added too... adding these dependencies manually is starting to look a bit weird.

kinow on 5 Dec 2018

👍1

As we will likely use Tornado, we can combine ... the flask branch; the tornado branch; ....

@dwsutherland @kinow - just to note (for the record on this PR Issue) as discussed we don't know yet if GraphQL will be needed or used in the suite server program - as opposed the new UI Server component ... so if the aforementioned combining is done it will be to investigate the technologies further but not intended for merge. ~~(And this PR should be closed and moved to an "exploratory" Issue with a link to the dev branch).~~

hjoliver on 5 Dec 2018

I think that's what @dwsutherland did, @hjoliver.

kinow on 5 Dec 2018

@kinow - you're right - apologies, my mistake! (The perils of checking in too late at night...). Amending previous comment...

hjoliver on 5 Dec 2018

Not a problem. I did double check as I was replying in early morning too, pre coffee.

And

so if the aforementioned combining is done it will be to investigate the technologies further but not intended for merge

Well noted. And the changes that I did for Tornado's main loop in scheduler.py have not been well tested. Definitely not ready for merge.

kinow on 5 Dec 2018

@dwsutherland , having to learn a bit more of Relay in order to get it working with Vue. Looking at other Vue projects, looks like most devs are adopting Vue + Apollo.

Q1/ do you know if the only part that we are using of Relay is pagination?
Q2/ do you think it would be too much to implement pagination for Apollo instead?

The reasons for Q2 are for it being framework agnostic, but also due to the better integration with Vue (though the Vue + Relay might work, just need more time, but the project is maintained by 1 dev I think...).

I think Graphene offers easy support for Relay, which is nice. But if that's not too complicated to replace.... grimacing

Yes to both Q1 and Q2 (but proof is "in the pudding" so to speak), I guess it's just Vue that's the show stopper for the Relay compliant endpoint being used. But it appears Apollo claims to work with any endpoint:
https://github.com/apollographql/apollo-client

Universally compatible, so that Apollo works with any build setup, any GraphQL server, and any GraphQL schema.

We may need to just implement our own pagination and cursors.

Well done on the Tornado GraphQL-endpoint implement!

dwsutherland on 5 Dec 2018

I've dropped the use of Relay;
SameBranch
for ease of use with Apollo-Vue (can be put back in place easily).

I've also tidied up the resolvers, so the Task/Family queries/Types all use the same function (at top of schema).

dwsutherland on 18 Dec 2018

There’s one issue I’m trying to get my head around; meta data (suite/family/task) has a mix of predefined & custom/arbitrary defined fields, so you cannot specify them in general for suites in the schema definition (if it was an individual suite, then it might be possible to do it on start (but not reload, so not desirable)).. So we have a few options:

Specify the whole meta as a JSON blob:
meta = graphene.JSON()
Which would mean you lose control of only pulling individual fields (i.e. just the title & description), and it means any query filters in the back end would need to parse the JSON.
Specify the default fields, and a custom JSON fields:

class QLMeta(graphene.ObjectType):
    """Meta data fields, and custom fields JSON blob"""
    class Meta:
        default_resolver = dict_resolver
    title = graphene.String(default=None)
    description = graphene.String(default=None)
    group = graphene.String(default=None)
    URL = graphene.String(default=None)
    custom = graphene.JSON()

This would just mean an extra level in the data, and a json.dump of all fields that aren’t default.

Having meta composed of key/value list:
[{“key”: “title”,”value”: “some suite”},{“key”….]
Which is worse than a JSON dump, but easier to fill..
Having meta and custom_meta fields in the node def.

meta = graphene.Field(QLMeta)
custom_meta = graphene.JSON()

This would just reduce the depth while leaving meta more granular.

Perhaps to start 1 or 4.
I don’t think [meta]group has a corresponding [[[meta]]]group under runtime (but could include it as predefined for both I suppose).

There are other fields ([[[environment]]], [[[directives]]] ..etc) that more easily fit into the JSON blob category, if they are desired at the gui.

BTW - WRT the QL in front of the schema types; I can drop them and just use schema.Task externally for the sake of namespace (it was just an easy way to recognize what they were at a glance).

dwsutherland on 18 Dec 2018

Some quick comments (with an old dinosaur hat on):

[meta]group should probably be retired? It was used to group suites in gscan but is not widely used. These days, suites can be registered with a directory hierarchy, so the functionality of group is less obvious.

[[[environment]]] and [[[directives]]] both need to be ordered. (E.g. an environment variable setting may reference another environment variable defined earlier. If our configuration file format has native support for list, these settings should probably be defined as lists.)

matthewrmshin on 18 Dec 2018

So we have a few options

If I understand it correctly option 2 looks superior to me.

We aught to be able to pull the default fields individually (as we are likely to want to use this data in the GUI), if users want custom fields I see no harm in giving them the whole dictionary. I doubt anyone is likely to use the API for this anyway.

[meta]group should probably be retired?

Yep, see https://github.com/cylc/cylc/issues/2776.

oliver-sanders on 2 Jan 2019

👍1

Been a long time coming, but I've made a lot of data structure changes (most satisfy recommendations), and now need more feedback/review:
(Note: all the nomenclature is up for change/review, i.e. if you don't like the use of proxy for task cycle point instance)

Task-Job Separation
This is actually a pseudo separation, although a true separation may be desirable in the future, and it involved:

Creating a job type, stripping job specific fields from graphql task [proxy] type, and adding new job conf fields.
Creating a new job pool job_pool.py, to store and manage job data objects.
Populate the job objects on the back of job config and task proxy state/message management.

The full field query result being

{
  "data": {
    "jobs": [
      {
        "id": "20170101T0000+13/baa/01",
        "batchSysJobId": "3338",
        "batchSysName": "background",
        "batchSysConf": {},
        "directives": {},
        "environment": {
          "GREETING": "Hello from baa!"
        },
        "envScript": "echo \"Hi first, I'm second\"",
        "errScript": "echo 'Boo!'",
        "exitScript": "echo 'Yay!'",
        "extraLogs": [
          "/home/sutherlander/startrek/captains.log"
        ],
        "executionTimeLimit": null,
        "finishedTime": 1551516906,
        "finishedTimeString": "2019-03-02T21:55:06+13:00",
        "host": "localhost",
        "initScript": "echo 'Me first'",
        "jobLogDir": "/home/sutherlander/cylc-run/baz/log/job/20170101T0000+13/baa/01",
        "owner": null,
        "paramEnvTmpl": {},
        "paramVar": {},
        "postScript": "sleep 10",
        "preScript": "sleep 10",
        "script": "sleep 10; echo \"$GREETING\"",
        "shell": "/bin/bash",
        "startedTime": 1551516876,
        "startedTimeString": "2019-03-02T21:54:36+13:00",
        "state": "succeeded",
        "submitNum": 1,
        "submittedTime": 1551516876,
        "submittedTimeString": "2019-03-02T21:54:36+13:00",
        "workSubDir": null,
        "taskProxy": {
          "id": "baa.20170101T0000+13"
        }
      }
    ]
  }
}

Of course you'd query from the task:

{
  taskProxies(id: "baa.*") {
    id
    jobs {
      id
      state
      submitNum
    }
  }
}

Result:

{
  "data": {
    "taskProxies": [
      {
        "id": "baa.20170201T0000+13",
        "jobs": []
      },
      {
        "id": "baa.20170101T0000+13",
        "jobs": [
          {
            "id": "20170101T0000+13/baa/01",
            "state": "succeeded",
            "submitNum": 1
          },
          {
            "id": "20170101T0000+13/baa/02",
            "state": "failed",
            "submitNum": 2
          },
          {
            "id": "20170101T0000+13/baa/03",
            "state": "running",
            "submitNum": 3
          }
        ]
      }
    ]
  }
}

Perhaps if the job objects were created prior to run, then they could be directly modified via the web gui (say for trigger-edit), and job script created from the object (perhaps with a true job-task separation).

Separation of Task/Family to definition & proxy/instance
This is to reduce the duplication of information, and distinguish between the abstract task/family and it's cycle point instance/proxy...

The task/family types were split into def and proxy types and populated state_summary_mgr.py..
Meta type created and expanded on task def.
Prerequisite type granulated/expanded on the task proxy
Queries, arguments and resolver filter funtions added.

Query

{
  tasks(id: "bar") {
    meta {
      title
      description
      URL
      userDefined
    }
    proxies{
      id
      jobs {
        id
        state
        submitNum
      }
      prerequisites {
        expression
        conditions{
          taskId
          exprAlias
          reqState
          satisfied
          message
          taskProxy{
            state
          }
        }
        satisfied
        cyclePoints
      }
    }
  }
}

Result

{
  "data": {
    "tasks": [
      {
        "meta": {
          "title": "Some Top family",
          "description": "some task bar",
          "URL": "https://github.com/dwsutherland/cylc",
          "userDefined": {
            "importance": "Critical",
            "alerts": "none"
          }
        },
        "proxies": [
          {
            "id": "bar.20180101T0000+13",
            "jobs": [
              {
                "id": "20180101T0000+13/bar/01",
                "state": null,
                "submitNum": 1
              }
            ],
            "prerequisites": [
              {
                "expression": "c0 | c1",
                "conditions": [
                  {
                    "taskId": "foo.20180101T0000+13",
                    "exprAlias": "c0",
                    "reqState": "succeeded",
                    "satisfied": true,
                    "message": "unsatisfied",
                    "taskProxy": {
                      "state": "running"
                    }
                  },
                  {
                    "taskId": "qux.20180101T0000+13",
                    "exprAlias": "c1",
                    "reqState": "succeeded",
                    "satisfied": true,
                    "message": "satisfied naturally",
                    "taskProxy": {
                      "state": "succeeded"
                    }
                  }
                ],
                "satisfied": true,
                "cyclePoints": [
                  "20180101T0000+13"
                ]
              },
              {
                "expression": "c0",
                "conditions": [
                  {
                    "taskId": "bar.20171201T0000+13",
                    "exprAlias": "c0",
                    "reqState": "succeeded",
                    "satisfied": true,
                    "message": "satisfied naturally",
                    "taskProxy": {
                      "state": "succeeded"
                    }
                  }
                ],
                "satisfied": true,
                "cyclePoints": [
                  "20171201T0000+13"
                ]
              }
            ]
          },
          {
            "id": "bar.20171201T0000+13",
            "jobs": [
.
.
.

So a query like:

{
  taskProxies {
    id
    state
    namespace
    prerequisites {
      conditions {
        taskId
      }
    }
  }
}

Would give you all the information required for the dependency graph.

The previously-mentioned/initial capabilities are still in place for the most part. And there are other optimisations I've made, and obviously more to come.. But next, and while waiting for review, I'll be working on:

Pagination
Mutations. And, as a sneak peak, I've added the all-in-one stop suite mutation:

class StopSuite(graphene.Mutation):
    """Stop the suite."""
    class Arguments:
        stop_type = graphene.String(required=True)
        stop_item = graphene.String()
        stop_args = graphene.List(graphene.String)

    command_queued = graphene.Boolean()

    def mutate(self, info, stop_type, stop_item=None, stop_args=[]):
        if stop_type in ['now']:
            stop_cmd = 'stop_now'
        else:
            stop_cmd = 'set_stop_' + stop_type
        action = {}
        for key in stop_args:
            action[key] = True
        item = ()
        if stop_item:
            item = (stop_item,)
        schd = info.context.get('schd_obj')
        schd.command_queue.put((stop_cmd,item,action))
        return StopSuite(command_queued=True)


class Mutation(graphene.ObjectType):
    stop_suite = StopSuite.Field()

mutation {
    stopSuite(stopType: "after_task", stopItem: "baa.20170101T0000+13"){
    commandQueued
  }
}

Websockets & Subscriptions (probably with a Tornado implementation)
Authorisation/Privilege-Checking

I'll make general improvements along they way including; documentation/descriptions on the objects (which are available via the endpoint), functionality, sophistication/features and approach.. (perhaps protobuf objects instead of GraphQL objects to hold the data)

The repo will shift to cylc/wip-graphql at some point soon, but it's still here.

dwsutherland on 1 Mar 2019

👍2 🎉1

@dwsutherland Cool! I'm going to have to read your comment in detail.

A first minor suggestion. Perhaps change submitMethodId to batchSysJobId? (To align with fields in job.status.)

matthewrmshin on 1 Mar 2019

@dwsutherland Cool! I'm going to have to read your comment in detail.

A first minor suggestion. Perhaps change submitMethodId to batchSysJobId? (To align with fields in job.status.)

Done. (updated above)

dwsutherland on 2 Mar 2019

Hi @dwsutherland , I'm trying your branch flask-gevent-graphql, and as always I'm trying to run my all-time favourite etc/examples/tutorial/cycling/five/suite.rc.

I am running it with cylc run --no-detach --verbose --debug five as I normally do, but it failed due to a local variable temp used before assignment.

kinow@kinow-VirtualBox:~/Development/python/workspace/cylc$ cylc run --no-detach five
            ._.                                                       
            | |           The Cylc Suite Engine [7.8.1-25-gb22c27b]   
._____._. ._| |_____.           Copyright (C) 2008-2019 NIWA          
| .___| | | | | .___|   & British Crown (Met Office) & Contributors.  
| !___| !_! | | !___.  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
!_____!___. |_!_____!  This program comes with ABSOLUTELY NO WARRANTY;
      .___! |          see `cylc warranty`.  It is free software, you 
      !_____!           are welcome to redistribute it under certain  
2019-03-04T00:31:11Z INFO - Suite server: url=http://kinow-VirtualBox:43082/ pid=29417
2019-03-04T00:31:11Z INFO - Run: (re)start=0 log=1
2019-03-04T00:31:11Z INFO - Cylc version: 7.8.1-25-gb22c27b
2019-03-04T00:31:11Z INFO - Run mode: live
2019-03-04T00:31:11Z INFO - Initial point: 20130808T0000Z
2019-03-04T00:31:11Z INFO - Final point: 20130812T0000Z
2019-03-04T00:31:11Z INFO - Cold Start 20130808T0000Z
2019-03-04T00:31:11Z INFO - [prep.20130808T0000Z] -submit-num=1, owner@host=kinow-VirtualBox
2019-03-04T00:31:11Z ERROR - local variable 'temp' referenced before assignment
    Traceback (most recent call last):
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/scheduler.py", line 269, in start
        self.run()
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/scheduler.py", line 1783, in run
        has_updated = self.update_state_summary()
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/scheduler.py", line 1826, in update_state_summary
        self.state_summary_mgr.update(self)
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/state_summary_mgr.py", line 79, in update
        self._get_tasks_info(schd, parents_dict, ancestors_dict))
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/state_summary_mgr.py", line 332, in _get_tasks_info
        prereq_list.append(prereq.api_dump())
      File "/home/kinow/Development/python/workspace/cylc/lib/cylc/prerequisite.py", line 261, in api_dump
        expression = temp,
    UnboundLocalError: local variable 'temp' referenced before assignment
2019-03-04T00:31:11Z ERROR - error caught: cleaning up before exit
2019-03-04T00:31:11Z INFO - Suite shutting down - ERROR: local variable 'temp' referenced before assignment
2019-03-04T00:31:11Z INFO - DONE
Traceback (most recent call last):
  File "/home/kinow/Development/python/workspace/cylc/bin/cylc-run", line 25, in <module>
    main(is_restart=False)
  File "/home/kinow/Development/python/workspace/cylc/lib/cylc/scheduler_cli.py", line 134, in main
    scheduler.start()
  File "/home/kinow/Development/python/workspace/cylc/lib/cylc/scheduler.py", line 300, in start
    raise exc
UnboundLocalError: local variable 'temp' referenced before assignment

kinow on 4 Mar 2019

Mutations. And, as a sneak peak, I've added the all-in-one stop suite mutation:

Cool!

I've separated the network implementation and interface in the python3 branch so (with a small change) you could write a simple adapter to map the network endpoints onto your GraphQL layer which would save having to duplicate the schd.command_queue.put((stop_cmd,item,action)) type logic.

Something along the lines of:

-class SuiteRuntimeServer(ZMQServer):
-    """Suite runtime service API facade exposed via zmq."""
+class SuiteRuntimeInterface:
+    """Suite runtime service API facade."""

     API = 4  # cylc API version

@@ -652,7 +652,7 @@ class SuiteRuntimeServer(ZMQServer):
         return (True, 'Command queued')

     @authorise(Priv.CONTROL)
-    @ZMQServer.expose
+    @expose  # and so on for all the others
     def trigger_tasks(self, items, back_out=False):
         """Trigger submission of task jobs where possible.

@@ -664,3 +664,17 @@ class SuiteRuntimeServer(ZMQServer):
         self.schd.command_queue.put(
             ("trigger_tasks", (items,), {"back_out": back_out}))
         return (True, 'Command queued')
+
+
+class SuiteRuntimeServer(ZMQServer, SuiteRuntimeInterface):
+
+    @staticmethod
+    def expose(fcn):
+        return ZMQServer.expose(fcn)
+
+
+class GraphQLAdapter(GraphQLServer, SuiteRuntimeInterface):
+
+    @staticmethod
+    def expose(fcn):
+        return GraphQLServer.expose(fcn)
+
+    # ...

oliver-sanders on 4 Mar 2019

👍1

Something to keep in mind over the whole "commandQueued" thing.

This is legacy from our old REST interface where we were restricted to a simple REQ-REP model.

Now that we are looking at using sockets for the HTTP and TCP interfaces we can keep the socket open and trickle back data until the client looses interest i.e:

REQ - stop_suite, kill=True
REP - commandQueued
REP - commandSucceeded

I guess that's really more of SUB-PUB model but anyway this functionality would be really useful for the GUI (could display a waiting symbol). The lack of this at present trips a lot of users up.

It's especially useful when commands fail, at present users are not informed of command failure (from where they issued the command) and have to look in the suite log to find out why it failed.

oliver-sanders on 4 Mar 2019

Hi @dwsutherland , I'm trying your branch flask-gevent-graphql, and as always I'm trying to run my all-time favourite etc/examples/tutorial/cycling/five/suite.rc.

I am running it with cylc run --no-detach --verbose --debug five as I normally do, but it failed due to a local variable temp used before assignment.

@kinow - Ok, I see the issue (didn't fail for me), needed to only include satisfied (like how it currently is).. just put a fix in.. that "should" fix it..