Opentrons: RFC: json protocol format

[btmorr]

Overview

In order to improve cross-language compatibility of applications interacting with the OT2, we plan to add HTTP endpoints to the Opentrons API server that will support controlling the OT2 via protocols defined in a json protocol format. The server will continue to support existing Python protocols, but adding support for a json format will make it easier to create applications in other languages, such as JavaScript, JVM languages, Ruby, etc., that can generate protocols to control the OT2.

Proposal

The json protocol format would require the following:

• The top-level of the data packet must have the following fields:
  
  
  
  • "protocol-schema": a string with schema version that obeys semantic versioning
  
  
  • "metadata": an object with whatever information is useful to human readers, but is not meant to be read by any machine that is interpreting the protocol (other than being syntactically correct json)
  
  
  • "robot": an object with configuration data about the robot (to distinguish, for instance, between different named hardware configurations)
  
  
  • "instruments": pipettes or other instruments (active components) supported by the OT2 server
  
  
  • "labware": an object with a key for each tiprack, plate, tube rack, and any other component that is installed on the deck of the robot. Each labware object should have:
  
  
  • "slot": a number string for the slot number on the deck of the robot
  
  
  • "type": a model string for the labware, which must in the labware definitions
  
  
  • "commands" a list of objects, where each object is either an "annotations" object, a single command object, or a nested "commands" object
  
  

Each single command object will have:

• a "command" field with a valid server command as the value
• any other key-value pairs to pass to the command function as parameters

When interpreting a protocol, the server will discard metadata and any annotations objects, and flatten "commands" objects into a list of single command objects, sets up the robot state using the robot model string and data from the "instruments" and "labware" objects. The robot will then execute the flattened command sequence in order.

An example of the proposed format (originally posted here):

{
  "protocol-schema": "1.0.0",

  "metadata": {
    "protocol-name": "Example Protocol",
    "author": "Ian",
    "description": "An example of how JSON protocols will look. Does a multi-dispense then delays 1.5sec before dropping tip.",
    "created": 1519916981999,
    "last-modified": 1519917009476,
    "category": "basic pipetting",
    "subcategory": "sample prep",
    "tags": ["simple", "example"]
  },

  "designer-application": {
    "application-name": "opentrons/protocol-designer",
    "application-version": "1.0.0",
    "data": {}
  },

  "robot": {
    "model": "OT-2 Standard"
  },

  "instruments": {
    "pipette1Id": {
      "type": "pipette",
      "mount": "right",
      "model": "p300_single"
    }
  },

  "labware": {
    "tiprack1Id": {
      "slot": 7,
      "type": "GEB-tiprack-300uL",
      "name": "Tip rack"
    },
    "sourcePlateId": {
      "slot": 10,
      "type": "trough-12row",
      "name": "Source (Buffer)"
    },
    "destPlateId": {
      "slot": 11,
      "type": "96-flat",
      "name": "Destination Plate"
    },
    "trashId": {
      "slot": 12,
      "type": "fixed-trash",
      "name": "Trash"
    }
  },

  "commands": [
    {
      "annotation": {
        "name": "Distribute 1",
        "description": "Distribute source well to destination wells"
      },
      "commands": [
        {
          "command": "pick-up-tip",
          "pipette": "pipette1Id",
          "labware": "tiprack1Id",
          "well": "A1"
        },
        {
          "command": "aspirate",
          "pipette": "pipette1Id",
          "volume": 200,
          "labware": "sourcePlateId",
          "well": "A1"
        },
        {
          "command": "touch-tip",
          "pipette": "pipette1Id",
          "labware": "sourcePlateId",
          "well": "A1"
        },
        {
          "command": "dispense",
          "pipette": "pipette1Id",
          "volume": 50,
          "labware": "destPlateId",
          "well": "B1"
        },
        {
          "command": "dispense",
          "pipette": "pipette1Id",
          "volume": 50,
          "labware": "destPlateId",
          "well": "B2"
        },
        {
          "command": "dispense",
          "pipette": "pipette1Id",
          "volume": 50,
          "labware": "destPlateId",
          "well": "B3"
        },
        {
          "command": "blowout",
          "pipette": "pipette1Id",
          "labware": "trashId",
          "well": "A1"
        }
      ]
    },
    {
      "annotation": {
        "name": "Delay then drop tip",
        "description": ""
      },
      "commands": [
        {
          "command": "delay",
          "wait": 1.5,
          "message": "hello robot operator, this is an example of the delay command"
        },
        {
          "command": "drop-tip",
          "pipette": "pipette1Id",
          "labware": "trashId",
          "well": "A1"
        }
      ]
    }
  ]
}

Notes:

• All date fields are in milliseconds since epoch
• The "designer application" field is for protocol-generating applications put application metadata for record-keeping
• All "annotations" fields are for human-readability, and are expected to be discarded by the server
• Times (such as for "wait" commands) are in seconds, and fractional values are supported

Unknowns:

• How would active modules on the deck and/or software plugins for control commands would fit into this schema?
• What format would be used for defining custom labware data?

[theosanderson]

Hi, @willcanine asked if I would chime in here. It sounds a great idea and would have massively simplified some of the stuff I have implemented in which I built a queue system in a database (the queued commands could have simply been JSON strings and your parser could have been used). So that is one suggestion (not for the schema), that it is possible to have a hybrid setup where one can write in Python robot.execute(jsonCommandString) [may be an impossibility though].

Minor schema thoughts:

• I feel it might be slightly clearer to a novice if the top-level and lower-level commands blocks had different names. The lower level ones could be commandsets for instance? There needs to be exactly one top-level one, which doesn't apply to the lower levels. [Ed: procedure/subprocedure as below sounds great to me]
• From a personal point of view the more flexibility in terms of being able to position the pipette the better. But that adds more attributes. The most common thing that I can imagine people wanting to do is to dispense from the bottom of the well. For my part I have a lot of commands that aspirate/dispense all around a large well. Having attributes x,y,z,r and theta for any pipette-command would be great I think (these are offsets from the center of the well/labware as in the API).
• In terms of the conventions one is expected to use, naming things sourcePlateId probably doesn't make sense - it should be sourcePlate? Therefore maybe one might want to change the labware name attribute to description/title to avoid confusion?
• A great thing about this is that people can hack together really basic scripts to write these protocols. To facilitate this it would be good to allow "well": "A1" to also be legally written as "well": rowcol(0,0) or similar [but maybe this breaks json?]
• This is a more major feature request, beyond just the schema, but if you are able to prompt for user intervention it will dramatically increase the number of protocols that can be (semi-)automated.

[jamesscottbrown]

@IanLondon suggested that I have a look at this.

A JSON format like this seems like a good idea. Generating JSON directly would be more sensible than what I am doing currently (generating Python code).

However, whilst this proposal gets the basic idea across, and is enough to start a discussion, many aspects are currently unspecified.

Metadata

You say that the metadata "is not meant to be read by any machine that is interpreting the protocol"

As written this could be read as meaning that the data should not be used by a machine that is intepreting the protocol. However, depending on what exactly is meant by "machine" and whether it includes a computer attached to the robot there are many cases where it could be useful to read the metadata (e.g. for logging how often each protocol is run, populating an interface that lets the user select a protocol, warning a user if they are about to run a protocol that has a last-modified date that is earlier than the last-modified date of a previously run protocol with the same name, etc.)

I would instead say that the anything recorded in the metadata object should not be required to perform the protocol.

Robot

The specification should say what fields this object includes, and what values they can take.

Instruments

It is interesting that pipettes are identified by a model string, whereas in the Python API they are represented by separate min_volume, max_volume and channels fields.
If you adopt this approach, there should be a list of supported model names.

Also, what about the aspirate_speed and dispense_speed options?

Does mount here correspond to axis in the Python API?

Labware

In this example, slot is an integer, whereas in the Python API it is a string <letter><number>.

Commands

The name commands suggests that it is an array of command objects, but it can also contain annotations objects.
Procedure or method are possible alternative names (process has the disadvantage of being a verb, and steps/instructions are also plural nouns). Following @theosanderson's suggestion of distinct names for non-top-level blocks, procedure and subprocedure would work.

The example uses separate pick-up-tip/aspirate/touch-tip/dispense/blowout commands - will the JSON fomat also support the Transfer shortcutstransfer/distribute/consolidate?

Other

@theosanderson

A great thing about this is that people can hack together really basic scripts to write these protocols. To facilitate this it would be good to allow "well": "A1" to also be legally written as "well": rowcol(0,0) or similar [but maybe this breaks json?]

This isn't syntactically valid JSON, but you could could allow something very similar ("well": "rowcol(0,0)"): the well attribute would always be a string, and the parser would recognise what format it was in an convert appropriately. There are various formats that could be supported: row as digit, column as letter; row and column both as digits; single integer. Supporting more might make it easier to write protocols, but would potentially also make them harder to read.

[btmorr]

A couple of notes about this RFC:

Specification

This is not meant to be a formal specification yet. We're headed that direction, but this is the start of the conversation, rather than the end. We will be taking comments from this RFC and incorporating them into the plan, and then publishing a more firm specification along with implementing support for it in the API. We're considering using json-schema as part of firming this up.

Flexibility and complex actions

When most of the Python API was developed, Python was the only way for users to specify the robot's behavior, whcih required many users to learn to code for the first time. In order to make this as easy as possible, the API took on a lot of complexity in order to provide flexibility in how wells are addressed (there are at least 20 different ways I know of to address well "A1" in a plate), and to provide high-level commands so that users would not have to write loops or long chains of simple actions.

We expect that the primary creators of json protocols will not be programmers writing by hand--either experienced developers or novices. The creators will generally be software, such as the Protocol Developer (an Opentrons web app under current development) or 3rd party apps that we want to encourage by moving toward a language-agnostic format. Because of this, those applications are able to provide the concept of high-level actions to the user (such as "transfer", "distribute", "consolidate", etc), but then generate a json protocol made of atomic actions that make up those high-level actions (e.g.: "transfer" in the UI would be exported in the protocol as "pick-up-tip", "move-to", "aspirate", "move-to", "dispense", ...). We plan to provide enough support for annotations and metadata that those high-level concepts can be included as notes by the creating application and aid in readability.

In keeping with this, the first version of the specification will only include one way to address wells. Applications generating protocols can include as much row/column logic as desired, but should output the alphanumeric name of the well in the protocol. Choosing a single way to address wells and similar simplifications will reduce the number of points of unclarity in protocols (in what order will a consolidate happen?), and make it possible to dramatically simplify aspects of the API server, which will in turn result in improvements in the reliability and stability of the server. We do not plan on reducing options when this would make the robot actually un-able to perform a given action--we want it to be possible to fully perform any action with the OT2 using the json protocol--but how something is written may be different from the way an individual did so in Python (such as addressing a well as "A1" instead of index 0).

OT1 vs OT2

The json protocol is only being implemented for the OT2 currently. As such, it is designed around various refactors to the API server on the edge branch of this repository. We've done our best to make as many changes as possible non-breaking (for example, mount='right' is preferred over axis='b', but the axis parameter has not been removed), but the json protocol will use the preferred options in the new line of development rather than the legacy constructors and parameters. Other examples are using the slot numbers etched into the OT2 deck instead of the "" designations of slots, and using the P300_Single, P300_Multi, P10_Single, and P10_Multi constructors instead of Pipette with max_volume and channels. In keeping with this, we're going to take the definitions of those models (for instance, the number of channels for a given multi-channel pipette) out of Python source code and put it into json data in the repo with a "package.json" file, so that they can be loaded and used directly in other applications.

Specific comments

@theosanderson

• The robot.execute(jsonCommandString) is very close to exactly what we're trying to make possible, so we're definitely on the same page there.
• The general plan is at a minimum to expose whatever parameters are needed to have full control over what we call "atomic actions"--that is, actions that could not be broken down into smaller top-level API actions. As an example, by that definition, aspirate is an atomic action, and transfer is not. Options like x, y, z or r and theta will definitely be exposed for move actions.
• If you are using the Run App to send a protocol to the OT2, "prompt for user intervention" can be at least partially acheived by using the pause command without a complimentary resume, and then clicking the "resume" button in the App once the user intervention is complete. In a Python protocol or a third-party control app, it would be up to the developer to put in whatever prompt mechanism makes sense. I would definitely be in favor of a future feature allowing the protocol developer to specify a message that would be delayed by a "pause" command, though.
• Instead of name for a labware, how's about display-name? This is a more accurate description of how we expect to use it in the Run App.
• I'm in favor of procedure/subprocedure--will include this in the proposal in place of commands.

@jamesscottbrown

• I agree with your interpretation of the metadata field--I'll revise the language in the finished proposal.

[IanLondon]

Also in consideration of making it easier to infer types when parsing more shallowly, how do you all feel about changing

  "instruments": {
    "pipette1Id": {
      "type": "pipette",
      "mount": "right",
      "model": "p300_single"
    }
  }

to putting pipettes under a pipettes key and then removing the type:

  "instruments": {
    "pipettes": {
      "pipette1Id": {
        "mount": "right",
        "model": "p300_single"
      }
    },

    "modules": { ??? TBD ??? }
  }

this also gives us a place to put modules, once the schema supports them, modules will probably go under a modules key

[IanLondon]

Clarification note: as far as the spec goes, the ids for labware & pipettes just need to be unique strings. You can do "pipette1Id" or "q0-439jr20?39u2!309_309". These IDs are used to define labware & pipettes, then later to specify those labware or pipettes in the commands.

[jamesscottbrown]

@btmorr Thanks for this reply - it clarifies the design philosophy underlying this proposal.

I agree that it is a good idea to take the opportunity to abandon as much unnecessary baggage from previous implementations as possible, and that keeping things simple by providing only one way to address wells is sensible.

However, I'm not entirely convinced that implementing only the low-level commands is the best decision. Yes, it will simplify the server, but all of the eliminated complexity will be pushed onto the applications that are trying to generate the JSON. In particular, things like automatically dividing a transfer into multiple transfers small enough to be made using the selected pipette are very helpful. It seems preferable for the functions to convert high-level command to low-level commands to be written once, rather than being separately implemented by each application (with the potential for bugs each time).

[emernic]

A couple notes on doing complex job scheduling for OT2 robots:

• These protocols do not contain timing or precedence constraints. The OpenTron server is assumed to be "dumb" in the sense that it runs all the commands as quickly as possible (or does nothing for a specified time). This is a good thing because it allows complex scheduling to be done by upstream software if desired.

• It would be helpful if the OT server could quickly accept and run many small "protocols" that contain only a few commands. This would allow upstream software to break up protocols into small chunks and feed them to the robot "piecemeal". This is more of an OT server issue than a protocol schema issue, but it's relevant from the conceptual standpoint of what a "protocol" can represent. For example, we would not always want to home the robot and simulate the run at the start of each protocol.

[DhashS]

Some notes:

• Debating the JSON schema in a raw format would be preferrable, that way we as developers can see the whole layout at once. I can write one up if someone can send me a bunch of sample code, I don't have the familiarity with the sourcebase to yank it out. This should be done hand-in-hand with an AST, since then we can debate serialization separated from the AST.

  • Debating the AST as a whole would be great, allowing us a place to write operational semantics.

  • Partial protocol execution would be great, since then branching logic execution is simplified.

[btmorr]

@DhashS @emernic Partial protocol execution is part of the plan. We're planning on including session management in the server when we implement the json protocol endpoints, and I'll make sure it allows for partial protocol execution.

@jamesscottbrown For more complex commands, I'm not precisely opposed to including them, but for obvious reasons we'll be implementing the core "atomic" actions first, and then consider supporting more complex actions in the future. Also, some of those decisions are driven by the fact that we're a very small team and have to be strategic about what we're able to accomplish with very limited resources. Since it's primarily a resource constraint though, we're definitely open to PRs from the community to add support for commands. Once the core actions are in, anyone who wants to contribute can take a look at how the existing actions were implemented as a template for making other actions available. Once the system is more fully stable and feature-complete, our team will definitely look at what can be done to add features--including more complex commands--that will have the greatest impact on making it easier for both users and developers.

[IanLondon]

@jamesscottbrown regarding generating low-level commands (aspirate/dispense etc) from high-level actions (transfer/distribute etc), I am hoping that Opentrons and perhaps external contributors will be able to develop libraries in a variety of languages to do this.

The beginnings of what I hope will become a reusable JS library are in https://github.com/Opentrons/opentrons/tree/edge/protocol-designer/src/step-generation - though it doesn't have a very friendly API for 3rd parties at the moment as I currently don't have the bandwidth to make it a free-standing and nice-to-use library. (Sorry for lack of docs, I'm hoping to write more docs once I'm in a good place with the rest of our protocol designer app.)

To me, the strongest point of using low-level commands in a protocol generator is that if you're doing liquid/tip tracking, you need to know exactly what eg a distribute action is doing. So in your protocol generator, if you say "distribute 400uL from well A1 to wells B1, C1, D1 with a P300 single-channel, with fresh tips each time", you'd have to rebuild the logic of that command to determine how many tips are being used. If your logic ever mis-matches the logic of the Python server that would translate a higher-JSON distribute command down to atomic actions, then you get a mess. Low-level commands ensure that the protocol generator is able to have full information about the protocol, without needing to query an external server (eg, in the browser, even if offline).

Low-level commands also help with reproducibility. If there's a feature added to the theoretical JSON distribute command in the Opentrons Python server, then changes could make any protocol written before that change behave differently when executed. Versioning the JSON protocols (under protocol-schema) can help with that. However, through writing protocols for customers and writing protocol generator apps, I've found that the higher-level functions are never quite as flexible as I'd like. Like for distribute I'd like to be able to start taking from the next source well when it depletes the previous source wells without changing tips, and generally I'd like tip reuse options that are aware of the contents of wells and don't change tips between different source wells that have the same contents. If you don't just translate down to low-level commands, something like this requires a liquid tracking data model as part of the protocol execution, which adds a load of complexity to the JSON spec and the server.

All that being said, I wouldn't oppose with adding higher-level actions like distribute into the JSON for the convenience of people writing programs that generate protocols. Those actions would be more limited and more subject to having drifting behaviors than the nascent JS library I mentioned which only generates low-level atomic commands, but they're helpful to get up-and-running quickly, especially when we do not have ready-to-use libraries out there for you all yet (the first hurdle might be that tip tracking is required to be able to use the low-level commands in your app -- you need to specify a tip "well" location to pick-up-tip). Like Ben said, we unfortunately don't have much bandwidth to build this out right now, but (this goes to anybody) please reach out if you're interested in helping out. We'll have a better base to build on a few weeks from now once we have the low-level commands down in the Opentrons Python server.

Thoughts?

[IanLondon]

To help clarify the schema, I wrote up a JSON Schema (http://json-schema.org/) for the Opentrons JSON protocol format and an example:

• JSON Schema spec for protocols (paste in the left box)

• Example protocol (paste in the right box)

JSON Schema is hard to read unless you've worked with it a bit. The best way to use this is to go to this online JSON Schema validator, copy and paste the schema I wrote into the left box, then paste a JSON protocol in the right box. It will tell you if the protocol is valid or not. Also see the auto-generated documentation here. Hope this is useful!

Note: this JSON spec is still a proposal, it remains open to suggestions! I think we're gearing to close up on this within the next couple days/weeks, but there will always be the next schema version that we can incorporate changes into.

[emernic]

I am very excited to see this coming along. A few questions/comments I had after reading the schema:

• Volume should be explicitly noted to be in microliters.
• It would be good to see the sections for specifying custom labware and giving module commands soon. Ideally, there should be some support for using user-defined modules; however, I think this is a much lower priority.
• Having some way to resume the protocol programmatically after wait commands would be helpful for integrating the OT-2 with other hardware. I am not sure what the best implementation strategy will be for this.
• Is there a reason why both name and description are required if annotation is provided for a subprocedure?

[NathanYee]

Are there plans to support the TempDeck in Json protocols?

[IanLondon]

@NathanYee yes! Modules like TempDeck are not in the current JSON spec, but it's on the way.