Bids-specification: Same task across runs

[adelavega]

I have a question about how a task is defined in the spec, and how this applies to tasks that might span across runs (i.e. naturalistic tasks "movie-watching").

In the spec it states:

Task - a set of structured activities performed by the participant. Tasks are usually accompanied by stimuli and responses, and can greatly vary in complexity. For the purpose of this protocol we consider the so-called “resting state” a task. In the context of brain scanning, a task is always tied to one data acquisition. Therefore, even if during one acquisition the subject performed multiple conceptually different behaviours (with different sets of instructions) they will be considered one (combined) task

Note that it states that tasks are "always tied to one data acquisition", not "conceptually different behaviors".

I think this reading of the spec would suggest that a naturalistic study spanning three runs (e.g.: https://openneuro.org/datasets/ds001338), should be three tasks.

However, I think the section on Run clears this up:

Run - an uninterrupted repetition of data acquisition that has the same acquisition parameters and task (however events can change from run to run due to different subject response or randomized nature of the stimuli). Run is a synonym of a data acquisition.

I would suggest that the definition for Task is modified to avoid confusion:

a task is always tied to one or more data acquisitions

[chrisgorgo]

Also brought up in
https://openneuro.org/datasets/ds001338#comment-5c6314ba80ad0939ed28f806

On Tue, Feb 12, 2019 at 10:53 AM Alejandro de la Vega <
[email protected]> wrote:

I have a question about how a task is defined in the spec, and how this
applies to tasks that might span across runs (i.e. naturalistic tasks
"movie-watching").

In the spec it states:

Task - a set of structured activities performed by the participant. Tasks are usually accompanied by stimuli and responses, and can greatly vary in complexity. For the purpose of this protocol we consider the so-called “resting state” a task. In the context of brain scanning, a task is always tied to one data acquisition. Therefore, even if during one acquisition the subject performed multiple conceptually different behaviours (with different sets of instructions) they will be considered one (combined) task

Note that it states that tasks are "always tied to one data acquisition",
not "conceptually different behaviors".

I think this reading of the spec would suggest that a naturalistic study
spanning three runs (e.g.: https://openneuro.org/datasets/ds001338),
should be three tasks.

However, I think the section of runs clears this up:

Run - an uninterrupted repetition of data acquisition that has the same acquisition parameters and task (however events can change from run to run due to different subject response or randomized nature of the stimuli). Run is a synonym of a data acquisition.

I would suggest that the definition for Task is modified to avoid
confusion:
"a task is always tied to one or more data acquisitions"

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bids-standard/bids-specification/issues/146, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOkpz2I-ncgU6Pvw6iy3n-swtkl1VLIks5vMw2ggaJpZM4a30-5
.

[chrisgorgo]

In https://openneuro.org/datasets/ds001338 the stimulus (an audio story) was split into three parts creating three tasks. They are not three runs (repetitions) of the same task because each time a different part of the story was played.

I don't think that dataset needs to be corrected - as you suggested. But it might be good to provide this example as part of the specification to bring more clarity.

[adelavega]

Wouldn't the same argument apply to a dataset such as this one: https://openneuro.org/datasets/ds001545/. The stimulus is also split up into three tasks, yet treated as a single task.

This is how I've seen most naturalistic datasets implemented in BIDS (e.g. raiders, life, forrest). I could imagine other conceptual "tasks" that differ in specific stimuli across runs (different words used in Stroop).

In your reading, are both styles legal? (and BIDS has some flexibility?).

In any case, I've published a fork of ds001338 using single task "style": https://github.com/neuroscout-datasets/ds001338_raw

[chrisgorgo]

Wouldn't the same argument apply to a dataset such as this one: https://openneuro.org/datasets/ds001545/. The stimulus is also split up into three tasks, yet treated as a single task.

Indeed in this case I would recommend splitting into task-movie01, task-movie02, etc.

This allows distinguishing between watching the same segment of a movie twice (task-movie01_run-01, task-movie01_run-02) vs. watching two different segments of the same movie (task-movie01, task-movie02).

In my interpretation of the definition of task splitting a movie in a consistent manner is not randomizing stimuli and thus different from Stroop example. It's just my interpretation though.

[emdupre]

Sorry, just to confirm: Is that to suggest that the data could be represented both as task-movie_run01 and task-movie01 ? Or are you suggesting we should standardize towards task-movie01 ?

Like @adelavega I've also mostly seen the task-movie_run-01 labelling, so it'd be nice to clarify this !

[chrisgorgo]

My personal interpretation is that task-movie_run-01, task-movie_run-02 should be only used if the same movie was watched twice. If a movie was split into two section task-movie01, task-movie02 should be used. Both options are allowed, but they mean something else.

[adelavega]

@chrisfilo, I think you make a good point about about watching the same segment twice, but this could also be determined using the events.tsv file as well (i.e. different stimuli files are referred to across runs).

This allows for more complex designs. For example, https://openneuro.org/datasets/ds001246/versions/1.0.1 has three runs of the same task (e.g. "perception"), with different stimuli for each run.

[adelavega]

@tyarkoni -- pinging you as you had some thoughts on this you wanted to shae

[tyarkoni]

I agree with @adelavega here—I don't see a good reason to privilege a difference in stimuli over a difference in nominal task instructions. If the same movie is split into 3 runs, but it's passive viewing in all cases, calling the result task-movie01, task-movie02, etc. feels inconsistent to me with the way we handle standard cognitive tasks where the stimuli change. E.g., suppose you're doing a face vs. house recognition task that lasts for 2 runs, but the specific images are different in each run. We wouldn't call that task-recog1_run-1 and task-recog2_run-1 just because the stimuli are different, would we? Typically we'd say you did the same task for 2 runs, even if the stimuli are different. It feels much more natural to call it task-recog_run-1 and task-recog_run2. I don't see any difference in the movie situation.

@chrisfilo I don't really see why randomization should make a difference for purposes of determining labels. Say you have 20 clips you need to distribute over 2 runs. Why should randomization scheme change the naming convention? If they're randomized, then that makes task-movie_run-1 and task-movie_run-2 okay, but if they're not randomized, then it's task-movie1_run-1 and task-movie2_run-1? The only argument I can see for that is that order is a confound in the latter case, so maybe you shouldn't fit the same model to both runs. But that's a judgment call that I don't think we can impose on people at the spec level.

I also don't think this really helps distinguish cases where the same stimuli are repeated a second time. How would you know whether it's run-1 and run-2 because of randomization, vs. because exactly the same sequence of events is presented a second time?

In general I think we should try to stay away as much as possible from having design considerations bleed into the labeling. Unfortunately that means we're stuck with ambiguity in cases like this. If it were up to me, I would say that run indices should just sequentially increase within session, based on the chronological order they were acquired in, and task should be used to encode anything else the researcher wants to track. So then you could have things like task-movie1_run-1, task-movie1_run-2, task-movie1repeat_run-3, task-movie2repeat_run-4, etc. I realize this is probably incompatible with the spec at at this point, but it would at least have the benefit of being unambiguous—and would also avoid having to rely on task to uniquely distinguish any two files.

Absent that, I don't see any way to settle on a convention that won't be ambiguous and elicit disagreement on occasion, and I think we just have to live with that.

[sappelhoff]

Absent that, I don't see any way to settle on a convention that won't be ambiguous and elicit disagreement on occasion, and I think we just have to live with that.

+1

It seems to me that no action is warranted for now. Ambiguous cases such as these should be well documented (e.g., in the README)

Closing for now, feel free to reopen if new arguments or suggestions for changing the spec emerge