Bids-specification: Clarify the definition of Event in Common Principles

[VisLab]

The current definition of Event in Common Principles seems restrictive:

Event - a stimulus or subject response recorded during a task. Each event has an onset time and duration. Note that not all tasks will have recorded events (e.g., “resting state”).

Many modalities such as EEG typically record and rely on many events that don't fall into a simple stimulus-response category. I assumed that the event category was more encompassing. This issue will come to the fore when people start writing more tools for BIDS data in these modalities. Here is a first cut at a clarification:

Event - something that happens or may be perceived as happening at a particular moment during the recording. Events are most commonly associated with sudden-onset sensory stimulus presentations or with the distinct onset of a subject motor action. Other events may include unplanned incidents, changes in task instructions or experiment control parameters, and noted data feature occurrences. In BIDS, every event has an onset time and duration.

[sappelhoff]

Thanks for this suggestion @VisLab I think your overhaul of the definition is in line with how we actually treat events already.

opinions @robertoostenveld @effigies @tsalo ?

[hoechenberger]

Hey, sorry I'm late to the party. I just wanted to double-check about this proposal:

Other events may include unplanned incidents, changes in task instructions or experiment control parameters, and noted data feature occurrences.

Does that mean that it would be totally okay to add e.g. a time period where data recording was disturbed or one or a few sensors produced faulty data? Because this is something that often occurs during EEG recordings, and until now I believed there is no way to mark those time periods in BIDS – only entire channels can be marked as bad for the entire recording. Would "two sensors produced noise for 10 seconds, and this time period should therefore probably be dropped during analysis" qualify as "noted data feature occurrences" or an "unplanned incident"? If so, how would I label these events?

[sappelhoff]

Does that mean that it would be totally okay to add e.g. a time period where data recording was disturbed or one or a few sensors produced faulty data?

I think it would be possible with the lenient definition of what constitutes an event.

I would however not recommend to use it that way --> your use case is more related to annotations, which we want to develop in BEP021

[hoechenberger]

I would however not recommend to use it that way --> your use case is more related to annotations, which we want to develop in BEP021

Thanks, @sappelhoff!

cc @agramfort

[hoechenberger]

@sappelhoff Oh I just realized, this BEP is related to derivates only? I'm talking about annotating the raw data, though…

[sappelhoff]

Oh I just realized, this BEP is related to derivates only?

you could annotate raw data and save that as an "annotation.tsv" (or whatever we will call it) ... and then the annotation.tsv would be a derivative? --> it's not fleshed out yet :-) cc @jasmainak

[VisLab]

Sorry, I missed the above. Thanks for pinging me. I am not sure I completely understand the issue that was raised and may need some additional clarification if I missed the point.

I think the suggested change applies both to raw data and to derived data. It is fundamental to the nature of events. Most of my experience is with EEG and biophysical signals. As I mentioned in my original post, there are many event markers that occur in the data (some directly off the recording devices) that appear in the data. Things like "begin block markers', instruction display markers, and different types of cues may be inserted as the data is recorded. Some recording software allows experimenter defined markers for other things. None of these fit into stimulus/response. There are also tools that write markers into the event stream without changing the raw data (for example marking blink positions in the data), etc. Clinical applications also may write markers in the data pertaining to other things. Increasingly acquisition software is being designed to put in many different types of event markers directly in the output stream to better document the context under which the data is taken. This is essential for data sharing.

There is no problem assigning HED tags for these different types of events. To take them out and put them in a separate annotations.tsv file in addition to the events file doesn't seem to make sense to me for usability. It seems to hobble the potential richness of what can be automatically recorded as the experiment takes place.

[effigies]

I think the suggested change applies both to raw data and to derived data. It is fundamental to the nature of events.
[...]
There is no problem assigning HED tags for these different types of events. To take them out and put them in a separate annotations.tsv file in addition to the events file doesn't seem to make sense to me for usability. It seems to hobble the potential richness of what can be automatically recorded as the experiment takes place.

I agree with all of this.

The distinction between derivatives and raw data is always somewhat artificial, and descriptive data that can reasonably fit into "raw" data structures has generally been welcome. There are some notable edge cases like lesion masks where we've fallen on the side of calling them derivatives, even though there's an argument that they're a description of the data, because treating mask images in derivatives allows for a cleaner spec. I don't see new classes of events as muddying the spec.

[robertoostenveld]

Agreed that the raw/derived distinction is hard to make. The interpretation of the results from "apply an algorithm and see whether the outcome exceeds a threshold" is more likely to be considered derived, whereas "visual inspection and writing down annotations" is more likely to be considered "raw". But whether the inspection was visual or not, would not conceptually change the result.

For me it would be just fine (and I believe in line with the BIDS philosophy) to store _metadata_ along with the data that it describes and enriches. So for example ICA- or bandpass-filtered data is _not_ metadata (and hence would go in derivatives), but clinical or behavioral observation annotations are (think video EEG). Same goes with bad channels, bad segments, or a combination of those (i.e. channels that are bad for a limited time). That is also why I would be fine to have sub-01_task-whatever_annotations.tsv sitting besides sub-01_task-whatever_eeg.edf. But I would pragmatically also be fine if the annotations and events were merged in a single tabular file (which I would call events.tsv).

[VisLab]

I'm not opposed to allowing an annotations.tsv file, but I think that this
is a distinct issue from defining what an event is.

The things like bad channel identification could be annotations. I would
like to, in fact, allow multiple annotation files. You might want to
produce one file for bad channel identification using one algorithm and
another file using another and then compare them. Analysts could then
decide which annotation files to merge. These types of annotation files
maybe should go into derivatives if the algorithms do some type of
transformation to the data. If they are just markers for things that an
algorithm detected, then they could possibly go in either place.

However, there are many markers that do not come from an analysis but from
the creation of the experiment. These are not produced by a choice of
algorithm but by the actual experiment. For analysts doing whole signal
analysis rather than calculated ERPs this type of information is needed to
document the experiment. They are naturally recorded as part of the
experiment. I think they should be part of the events.tsv.

Summary: events are things that happen. We should allow experimenters to
put their markers in the events.tsv or in annotations.tsv or in
x_annotations.tsv. However, we should suggest that information that is
calculated post hoc --- like identification of blinks and identification of
sections of bad channels be put into annotations files, while events that
are naturally recorded as part of the experiment should be in the events.tsv

On Tue, Sep 15, 2020 at 8:32 AM Robert Oostenveld [e-mail]
wrote:

Agreed that the raw/derived distinction is hard to make. The
interpretation of the results from "apply an algorithm and see whether the
outcome exceeds a threshold" is more likely to be considered derived,
whereas "visual inspection and writing down annotations" is more likely to
be considered "raw". But whether the inspection was visual or not would not
conceptually change the result.

For me it would be just fine (and I believe in line with the BIDS
philosophy) to store metadata along with the data that it describes and
enriches. So for example ICA- or bandpass-filtered data is not metadata
(and hence would go in derivatives), but clinical or behavioral observation
annotations are (think video EEG). Same goes with bad channels, bad
segments, or a combination of those (i.e. channels that are bad for a
limited time). That is also why I would be fine to have
sub-01_task-whatever_annotations.tsv sitting besides
sub-01_task-whatever_eeg.edf. But I would pragmatically also be fine if
the annotations and events were merged in a single tabular file (which I
would call events.tsv).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/564#issuecomment-692716906,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJCJOXRDZV6PZ7PQFG7TITSF5UHLANCNFSM4PXS7D2A
.

[VisLab]

In thinking about it, I think that I'd like to see just one events.tsv in
the original dataset and the multiple annotations.tsv in the derivatives.
They all contain "events". Let the data owner decide which markers are
properties of the experiment and which are layers added by computation.
Often additional experimental markers are written post-hoc --- such as when
the buttons are pressed if that isn't integrated into the experimental
setup. Other kinds of markers can be written post-hoc as well.

On Tue, Sep 15, 2020 at 8:57 AM Kay Robbins [e-mail] wrote:

I'm not opposed to allowing an annotations.tsv file, but I think that this
is a distinct issue from defining what an event is.

The things like bad channel identification could be annotations. I would
like to, in fact, allow multiple annotation files. You might want to
produce one file for bad channel identification using one algorithm and
another file using another and then compare them. Analysts could then
decide which annotation files to merge. These types of annotation files
maybe should go into derivatives if the algorithms do some type of
transformation to the data. If they are just markers for things that an
algorithm detected, then they could possibly go in either place.

However, there are many markers that do not come from an analysis but from
the creation of the experiment. These are not produced by a choice of
algorithm but by the actual experiment. For analysts doing whole signal
analysis rather than calculated ERPs this type of information is needed to
document the experiment. They are naturally recorded as part of the
experiment. I think they should be part of the events.tsv.

Summary: events are things that happen. We should allow experimenters
to put their markers in the events.tsv or in annotations.tsv or in
x_annotations.tsv. However, we should suggest that information that is
calculated post hoc --- like identification of blinks and identification of
sections of bad channels be put into annotations files, while events that
are naturally recorded as part of the experiment should be in the events.tsv

On Tue, Sep 15, 2020 at 8:32 AM Robert Oostenveld <
[email protected]> wrote:

Agreed that the raw/derived distinction is hard to make. The
interpretation of the results from "apply an algorithm and see whether the
outcome exceeds a threshold" is more likely to be considered derived,
whereas "visual inspection and writing down annotations" is more likely to
be considered "raw". But whether the inspection was visual or not would not
conceptually change the result.

For me it would be just fine (and I believe in line with the BIDS
philosophy) to store metadata along with the data that it describes
and enriches. So for example ICA- or bandpass-filtered data is not
metadata (and hence would go in derivatives), but clinical or behavioral
observation annotations are (think video EEG). Same goes with bad channels,
bad segments, or a combination of those (i.e. channels that are bad for a
limited time). That is also why I would be fine to have
sub-01_task-whatever_annotations.tsv sitting besides
sub-01_task-whatever_eeg.edf. But I would pragmatically also be fine if
the annotations and events were merged in a single tabular file (which I
would call events.tsv).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/564#issuecomment-692716906,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJCJOXRDZV6PZ7PQFG7TITSF5UHLANCNFSM4PXS7D2A
.

[effigies]

FWIW, I have a slight preference to continue calling any file that contains "events" events.tsv, rather than adding a synonym annotations.tsv.

[VisLab]

That would be okay too. Is there a good pointer to how events.tsv files
work in derivatives? Do derivative datasets inherit original events. Can
they have multiple additional event files?

On Tue, Sep 15, 2020 at 12:37 PM Chris Markiewicz [e-mail]
wrote:

FWIW, I have a slight preference to continue calling any file that
contains "events" events.tsv, rather than adding a synonym annotations.tsv
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/564#issuecomment-692866163,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJCJOXRTNEFQRFZB226HJ3SF6Q4DANCNFSM4PXS7D2A
.

[effigies]

There's an example in Preprocessed or cleaned data:

pipeline2/
    sub-001/
        eeg/
            sub-001_task-listening_run-1_desc-autoannotation_events.tsv
            sub-001_task-listening_run-1_desc-autoannotation_events.json
            sub-001_task-listening_run-1_desc-filtered_eeg.edf
            sub-001_task-listening_run-1_desc-filtered_eeg.json

Adding a desc-<label> entity is all that's really required to distinguish a derived file from raw. I'm not sure exactly what it means to "inherit" original events. The way I use them is to index both the original and derivative dataset with PyBIDS:

layout = BIDSLayout(root='/path/to/raw', derivatives=['/path/to/deriv1', '/path/to/deriv2'])

Then I have access to variables associated with entity sets, whether they show up in the raw data, one derivative or the other.

[robertoostenveld]

FWIW, I have a slight preference to continue calling any file that contains "events" events.tsv, rather than adding a synonym annotations.tsv.

agreed. If different file names are needed to allow the files to co-exist in the same directory, then I believe it is better (more scalable, and in line with BIDS conventions) to use an entity (key-value pair) to clarify which types of events are in which file. I think that desc-<label> would work fine. And we already have _events.json which could perhaps be used to elaborate (as metadata in additional JSON fields) about the different event types or methods used.

[VisLab]

Agreed.

On Wed, Sep 16, 2020 at 10:44 AM Robert Oostenveld [e-mail]
wrote:

FWIW, I have a slight preference to continue calling any file that
contains "events" events.tsv, rather than adding a synonym annotations.tsv
.

agreed. If different file names are needed to allow the files to co-exist
in the same directory, then I believe it is better (more scalable, and in
line with BIDS conventions) to use an entity (key-value pair) to clarify
which types of events are in which file. I think that desc- would
work fine. And we already have _events.json which could perhaps be used
to elaborate (as metadata in additional JSON fields) about the different
event types or methods used.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/564#issuecomment-693492819,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJCJOV4BUCWXBNSLPPRVKDSGDMOLANCNFSM4PXS7D2A
.

[effigies]

So this looks like a discussion of implications, and nobody's yet raised any objections. I think we can go ahead and open a PR with some confidence. @VisLab would you like to do it?

[VisLab]

It is already open as pull request #595 awaiting review. I wasn't sure how
to get this discussion moved over there. Thanks

On Thu, Sep 17, 2020 at 7:11 AM Chris Markiewicz [e-mail]
wrote:

So this looks like a discussion of implications, and nobody's yet raised
any objections. I think we can go ahead and open a PR with some confidence.
@VisLab https://github.com/VisLab would you like to do it?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/564#issuecomment-694188982,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJCJOU2A57P36IB6SO6CZLSGH4HVANCNFSM4PXS7D2A
.

[effigies]

Ah, my brain! I even approved it this week... Sorry!