Viewers: Support of RTSTRUCT visualization

[fedorov]

Request

What feature or change would you like to see made?

Basic visualization of RTSTRUCT structure sets in the image plane.

Why should we prioritize this feature?

As we discussed and planned for the IDC MVP functionality, we thought the following should be feasible:

Support of RTSTRUCT and SEG will be limited to in-plane visualization, not in MPR view. Visualization will be supported as semi-transparent overlay or contour. Interface will allow scrolling to the centroid of the individual structures, and toggle of the visibility for the individual structures/segments.

Submitting this to the tracker so it can be prioritized, as @swederik asked me to do.

Great example of similar capabilities implemented using OHIF: https://econtour.org/

[JamesAPetts]

Hey @fedorov, I'm going to start working on this, some questions:

Visualization will be supported as semi-transparent overlay or contour.

For the MVP, is a contour representation for RTSTRUCTs ok? In the long term it would be nice to have master formats and interoperability between different formats, but contours are the easiest route for an initial implementation.

Is there a set goal for how large an RTSTRUCT should be able to be parsed in a reasonable amount of time? The standard allows for entire studies of regions to be encoded, it'd be good to have an idea of what target is deemed reasonable.

Interface will allow scrolling to the centroid of the individual structures, and toggle of the visibility for the individual structures/segments.

This is all easily doable.

[JamesAPetts]

More thoughts:

Labels should ideally be read from the RT ROI Observations Sequence (3006,0080), however most of the useful information in here is type 3.

Do we also want RT Related ROI Sequence (3006,0030) information present in the UI on the first pass? Its tricky as this defines nice hierarchies with could be rendered in useful ways, but the sequence is type 3. Perhaps we should just show this information (if present) on mouse hover?

Additionally is visualisation other relevant type 3's such as:
Anatomic Region Sequence (0008,2218) and the Primary Anatomic Structure Sequence (0008,2228) high priority?

[fedorov]

@JamesAPetts thank you for the update!

For the MVP, is a contour representation for RTSTRUCTs ok?

Yes, absolutely.

Is there a set goal for how large an RTSTRUCT should be able to be parsed in a reasonable amount of time? The standard allows for entire studies of regions to be encoded, it'd be good to have an idea of what target is deemed reasonable.

Note that for IDC we are not targeting creation of RTSTRUCT. The idea is to support visualization of existing annotations, so the overall approach is to use existing TCIA content for defining the scope.

For the MVP, there are very few RTSTRUCT in the TCGA subset of data - just the tcga_hnsc collection has those. Overall, beyond TCGA, TCIA contains RTSTRUCT in the following collections:
cc_radiomics_phantom, mri_dir, soft_tissue_sarcoma, lctsc, hnscc, hnscc_3dct_rt, nsclc_radiomics, head_neck_pet_ct, tcga_hnsc, 4d_lung, head_neck_cetuximab. Below are some stats for those collections (if I did not mess up the query!):

Does it make sense to have the initial implementation that does not optimize performance, and is tested against tcga_hnsc?

Do we also want RT Related ROI Sequence (3006,0030) information present in the UI on the first pass? Its tricky as this defines nice hierarchies with could be rendered in useful ways, but the sequence is type 3. Perhaps we should just show this information (if present) on mouse hover?

Additionally is visualisation other relevant type 3's such as: Anatomic Region Sequence (0008,2218) and the Primary Anatomic Structure Sequence (0008,2228) high priority?

We should probably do some deeper analysis of the actual content of those attributes for TCGA collections before making that decision. How early in the process would you need this? We can work to help with this to better formulate the requirements.

cc: @dclunie

[JamesAPetts]

Note that for IDC we are not targeting creation of RTSTRUCT. The idea is to support visualization of existing annotations, so the overall approach is to use existing TCIA content for defining the scope.

👍

Does it make sense to have the initial implementation that does not optimize performance, and is tested against tcga_hnsc?

Yeah I think it does, just wanted to clarify 👍 .

We should probably do some deeper analysis of the actual content of those attributes for TCGA collections before making that decision. How early in the process would you need this? We can work to help with this to better formulate the requirements.

We don't need that immediately, more when we look at making the UI for struct navigation. Parsing and displaying of this information can be done at a later time if needbe.

[JamesAPetts]

Interface will allow scrolling to the centroid of the individual structures

Does the MVP want this to be the volumetric centroid or just the middle slice?

We should probably do some deeper analysis of the actual content of those attributes for TCGA collections before making that decision. How early in the process would you need this? We can work to help with this to better formulate the requirements.

Here is an example of one entry from one of the datasets:

{
  ObservationNumber: 17
  ReferencedROINumber: 17
  ROIObservationDescription: "Type:Soft, Range:*/*, Fill:0, Opacity:0.0, Thickness:1, LineThickness:2"
  RTROIInterpretedType: "CTV"
  ROIInterpreter: ""
}

[fedorov]

Several points from discussion with @JamesAPetts on Slack today:

[fedorov]

Hope the ^^^ makes sense ...

[JamesAPetts]

Here is a sketch of the UI (ignore colors, it'd look OHIF-ey):

https://www.figma.com/file/npC6xkCgrlsGcQIfR7xhFU/OHIF-RTSTRUCT-Panel?node-id=0%3A1

One thing that was said to be a requirement was comparing an ROIContour between two different structure sets, so I propose the following hierachal approach:

• On load of a series, the most recent RTSTRUCT is loaded (similar to SEG).
• Other RTSTRUCTs are visibile in the UI as expandable lists, as we have their metadata.
• When an RTSTRUCT is expanded, fetch its data and load it into memory.
• Have toggles for:
  
  
  
  • Hide whole RTSTRUCT.
  
  
  • Hide an individual ROIContour.
  
  

Any feedback @fedorov @pieper .

I'm going to rebase the current struct work on master and, once approved, I'll make a ticket for the UI.

[pieper]

@JamesAPetts LGTM

Question: we touched in the call briefly on the idea of having different structs visible in different viewports. Do you have a design in mind for this? In Slicer it's a pop up menu with a check box per viewport, but it might also make sense to have a drag and drop option.

[fedorov]

Design looks great for the first iteration @JamesAPetts!

we touched in the call briefly on the idea of having different structs visible in different viewports

Let's leave it out of the current initial version for now? I would rather leave a bit more breathing space to handle other issues and features that already came up and I am sure will come up before MVP.

[pieper]

we touched in the call briefly on the idea of having different structs visible in different viewports

Let's leave it out of the current initial version for now? I would rather leave a bit more breathing space to handle other issues and features that already came up and I am sure will come up before MVP.

Sounds good yes. We might need that functionality for other projects though (e.g. LNQ) so let's keep it on the overall OHIF horizon but not as part of the IDC:priority.

[fedorov]

Do we also want RT Related ROI Sequence (3006,0030) information present in the UI on the first pass? Its tricky as this defines nice hierarchies with could be rendered in useful ways, but the sequence is type 3. Perhaps we should just show this information (if present) on mouse hover?

FWIW, none of the TCIA collections has RT ROI Observations Sequence > RT Related ROI Sequence included, based on my queries.

Additionally is visualisation other relevant type 3's such as: Anatomic Region Sequence (0008,2218) and the Primary Anatomic Structure Sequence (0008,2228) high priority?

Can you tell me where that one is hiding within RTSS?

[dclunie]

Current "PS3.3 C.8.8.8 RT ROI Observations Module" as amended by CP 1864 (ftp://medical.nema.org/medical/dicom/final/cp1864_ft_AddAnatomicRegionSequenceToRTStructureSet.pdf)

Unlikely to be a priority because so recently added and not yet seen in the wild.

[JamesAPetts]

Since the initial discussion has been resolved I'm going to close this ticket.

different structs visible in different viewports
Has been given its own ticket:
https://github.com/OHIF/Viewers/issues/1522