Libertem: Improve dataset open forms

[uellue]

Since LiberTEM is getting better at guessing correct parameters for opening a dataset, users start to develop the expectation that default parameters are correct. For that reason we should remove all default parameters that are not verified against the data set. That includes the scan dimension of K2IS data sets.

[sk1p]

Currently, the default parameters serve a dual purpose, they are examples of the expected format. So when tackling this, we should make the parameter forms more user-friendly.

For example, the scan size should be split into a group of two <input type="number" min="1" /> fields, with separate labels (scan width / scan height). Then the user can't enter the data in an invalid format, and the meaning of both values is clear from the labels.

[uellue]

How do we handle n-dimensional data if we have separate input fields for each dimension and the number of dimensions is unknown? As an alternative to separate fields, we could use a longer description like in the Datatype field for raw data.

Would it be relevant that users can copy and paste a shape between scripting and GUI if it is in tuple format?

[sk1p]

How do we handle n-dimensional data if we have separate input fields for each dimension and the number of dimensions is unknown?

We don't, because we only handle 4D data in the GUI at the moment :grin: It could be solved by dynamically adding/removing form fields, like this example (although I'm not super happy with that):

As an alternative to separate fields, we could use a longer description like in the Datatype field for raw data.

Yes, combined with client-side validation, because it allows for many more invalid states than the 'separate inputs' implementation.

Would it be relevant that users can copy and paste a shape between scripting and GUI if it is in tuple format?

Hmm, that may be an issue. I guess we should then also allow full tuples as inputs, with parens and all that.

[uellue]

After discussions with @sk1p: We prefer separate fields. Reasons:

• Pasting a shape into the field is rarely used. The info dialogue still has the shapes as tuples to transfer them from the GUI to scripts, which is the common case. We can implement some "paste logic" to accept tuples if that is ever going to become a topic.
• The fields will probably only be used for raw data in the future after detection is completely implemented.
• Parameter caching means that one rarely has to enter new values anyway.
• Changing values will be easier by using tab to navigate between cells.
• Validation is MUCH easier.
• Adding and removing fields can be implemented as soon as we support n-dimensional data in the GUI.

[uellue]

We should review the opening dialogues as part of #706 since the tile shape parameter will be completely meaningless after that.

[AnandBaburajan]

After discussions with @sk1p: We prefer separate fields. Reasons:

• Pasting a shape into the field is rarely used. The info dialogue still has the shapes as tuples to transfer them from the GUI to scripts, which is the common case. We can implement some "paste logic" to accept tuples if that is ever going to become a topic.
• The fields will probably only be used for raw data in the future after detection is completely implemented.
• Parameter caching means that one rarely has to enter new values anyway.
• Changing values will be easier by using tab to navigate between cells.
• Validation is MUCH easier.
• Adding and removing fields can be implemented as soon as we support n-dimensional data in the GUI.

@sk1p, @uellue Is the plan still to replace scan & detector size fields with dynamic ones? If yes, can I work on this? Maybe I could work on the raw dataset's fields first.

[sk1p]

Sounds good, and is related to your GSoC project :+1: as @uellue mentioned, the tile shape parameter will be completely gone soon, so best to concentrate on scan size and detector size.

Maybe I could work on the raw dataset's fields first.

:+1:

I would suggest to keep keyboard users in mind: it should not be harder for a keyboard-only user to enter a tuple-style value than it currently is. For example, the focus could jump to the next field when hitting comma, possibly adding a new field if the user is on the last one, making sure tabbing between fields works well etc.

We should review the opening dialogues as part of #706 since the tile shape parameter will be completely meaningless after that.

Agreed. Instead, we will probably have a second form part "I/O backend", where the user could select a backend and possibly parameters. For example, instead of the MMAP backend, select a direct I/O backend. This will then replace the "direct I/O enabled" checkbox for the raw dataset and make direct I/O available for more data sets. It will also solve the direct I/O on windows issue, as the backend selection process will be dynamic and we could prevent the direct I/O backend even from appearing in the list of available backends on Windows/Mac OS X.

[AnandBaburajan]

@sk1p What should the max number of scan or detector size fields be, if it isn't 4? I meant, are there 4D STEM datasets with more than 4 dimensions?

[sk1p]

What should the max number of scan or detector size fields be, if it isn't 4? I meant, are there 4D STEM datasets with more than 4 dimensions?

4D-STEM datasets are always 4D, but we also support different nD-shapes via the Python API. In the future the GUI could possibly also support 1D and maybe also 3D visualizations.

Until then, I guess the scan_size parameters will be fixed to 2D. So there are a few possibilities:

1) Implement a fixed 2D scan_size input
2) Implement a flexible system, but fix it to 2D for now
3) like 2) but also implement simple 1D "visualization": could be done using a JS plotting library. As this also involves changes in how the data is sent over to the client, and at least also changes in the download handling, I think this should be spun out into a separate issue