Fmriprep: FMRIPREP groups BOLD runs (ds000205/sub-07)

[oesteban]

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py", line 450, in build_workflow
    ignore_aroma_err=opts.ignore_aroma_denoising_errors,
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/workflows/base.py", line 193, in init_fmriprep_wf
    ignore_aroma_err=ignore_aroma_err)
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/workflows/base.py", line 444, in init_single_subject_wf
    ignore_aroma_err=ignore_aroma_err)
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/workflows/bold/base.py", line 255, in init_func_preproc_wf
    if 'fieldmaps' not in ignore else []
  File "/usr/local/miniconda/lib/python3.6/site-packages/bids/grabbids/bids_layout.py", line 125, in get_fieldmap
    fieldmaps = self._get_fieldmaps(path)
  File "/usr/local/miniconda/lib/python3.6/site-packages/bids/grabbids/bids_layout.py", line 143, in _get_fieldmaps
    sub = os.path.split(path)[1].split("_")[0].split("sub-")[1]
  File "/usr/local/miniconda/lib/python3.6/posixpath.py", line 105, in split
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not list

[oesteban]

Seems like runs are being grouped. At this point,

https://github.com/poldracklab/fmriprep/blob/121fb90df3b0988913852e273b70a9a33696fb89/fmriprep/workflows/base.py#L344

The subject_data dictionary contains:

{'fmap': [], 'bold': [['/data/sub-07/func/sub-07_task-functionallocalizer_run-01_bold.nii.gz', '/data/sub-07/func/sub-07_task-view_run-01_bold.nii.gz'], '/data/sub-07/func/sub-07_task-view_run-02_bold.nii.gz'], 'sbref': [], 't2w': [], 't1w': ['/data/sub-07/anat/sub-07_T1w.nii.gz']}

@chrisfilo, @tyarkoni: do you recall if this has been added as a feature or the behavior is unintended? Maybe we are passing a wrong layout?

[tyarkoni]

Can you elaborate? What's the pybids call that collect_data is wrapping? And what are you expecting to get back in subject_data?

[oesteban]

Expected:

{'fmap': [], 'bold': ['/data/sub-07/func/sub-07_task-functionallocalizer_run-01_bold.nii.gz', '/data/sub-07/func/sub-07_task-view_run-01_bold.nii.gz', '/data/sub-07/func/sub-07_task-view_run-02_bold.nii.gz'], 'sbref': [], 't2w': [], 't1w': ['/data/sub-07/anat/sub-07_T1w.nii.gz']}

And here the collect_data definition:

https://github.com/poldracklab/fmriprep/blob/121fb90df3b0988913852e273b70a9a33696fb89/fmriprep/utils/bids.py#L120-L186

[tyarkoni]

Thanks, will look into it shortly. That doesn't look like intended behavior, but some things have changed internally, so it's certainly possible it's a bug. Are you cloning from master, or is this the latest PyPI release?

[emdupre]

@oesteban this is from #875; specifically here. I added this because multiple echos needed to be grouped together from the same run, but clearly I did not think of this case!

[emdupre]

I can patch a fix where only runs within the same task are grouped-- is that the desired behavior? Or is the concern about grouping (within) runs at all?

[oesteban]

Thanks @emdupre, so you need the grouping to be done for the multi-echo scans, right? - yes, fmriprep expects to have files (not lists) as input.

[oesteban]

Fixed in #916.

@tyarkoni sorry for the false alarm and thanks a lot for your super fast answers 👍

[oesteban]

I'm afraid this is still happening. For ds000031 we still have grouped bold scans. I have the impression that task and run match across sessions and they are grouped.

We'll need to improve the regex in #916 to add the presence of the echo label (in this case, the value would not need to match).

WDYT?

[emdupre]

Sorry again for the bug, here.

Flipping my old logic, what if we hide the echo value, and sort on everything else? The code would look something like this:

    def _hide_echo(x):
        if 'echo' not in x:
            return x
        echo = re.search("_echo-\\d*", x).group(0)
        return x.replace(echo, "_echo-?")

    if subj_data["bold"] is not []:
        bold_sess = subj_data["bold"]
        if any(['_echo-' in bold for bold in bold_sess]):
            runs = [list(bold) for _, bold in groupby(bold_sess, key=_hide_echo)]
            runs = list(map(lambda x: x[0] if len(x) == 1 else x, runs))
        else:
            runs = bold_sess
        subj_data.update({"bold": runs})

This should be robust to any other key differences, and seems to work locally on ds000205 and ds000210.

EDIT: I patched the code snippet here.

[oesteban]

Yes, this would work except for multi echo and multi session datasets.
Right?

On Jan 3, 2018 12:15, "Elizabeth DuPre" [e-mail] wrote:

Sorry again for the bug, here.

Flipping my old logic, what if we hide the echo value, and sort on
everything else? The code would look something like this:

def _hide_echo(x):
    echo = re.search("_echo-\\d*", x).group(0)
    return x.replace(echo, "_echo-?")

if subj_data["bold"] is not []:
    bold_sess = subj_data["bold"]
    if any(['_echo-' in bold for bold in bold_sess]):
        runs = [list(bold) for _, bold in groupby(bold_sess, key=_hide_echo)]
        runs = list(map(lambda x: x[0] if len(x) == 1 else x, runs))
        subj_data.update({"bold": runs})

This should be robust to any other key differences, and seems to work
locally on ds000205 and ds000210.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/issues/914#issuecomment-355115184,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAkhxtMKiXJlCxfYg8rMBV1izICm9yZGks5tG9_kgaJpZM4RQTmL
.

[emdupre]

I think this should work on multi-echo and multi-session data-- do you have an example dataset in mind ? I can test and confirm. Right now, I can confirm it works on single echo only, multi-echo only, and mixed echo data.

[oesteban]

Sorry, I automatically closed the issue from another PR. Please don't get confused by that PR.

Yes, I meant the bids examples repo. I think adding multi-echo examples to https://github.com/INCF/BIDS-examples/wiki/MRI-datasets would be great, DYT @chrisfilo ?

Then we would use those for testing purposes :)

However, I don't think there is an openfmri dataset with multi-echo and multi-session. That one will need be generated by hand.

Anyways, all you mentioned in your last comment sounds very reasonable. Let's start with that PR asap and we'll see if it is worth testing multi-session.

[chrisgorgo]

That's a great idea!

Best,
Chris

On Wed, Jan 3, 2018 at 3:23 PM, Oscar Esteban [e-mail]
wrote:

Sorry, I automatically closed the issue from another PR. Please don't get
confused by that PR.

Yes, I meant the bids examples repo. I think adding multi-echo examples to
https://github.com/INCF/BIDS-examples/wiki/MRI-datasets would be great,
DYT @chrisfilo https://github.com/chrisfilo ?

Then we would use those for testing purposes :)

However, I don't think there is an openfmri dataset with multi-echo and
multi-session. That one will need be generated by hand.

Anyways, all you mentioned in your last comment sounds very reasonable.
Let's start with that PR asap and we'll see if it is worth testing
multi-session.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/issues/914#issuecomment-355156318,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp3C-kTmKsXzEd_vCGM-TyCgOdpy2ks5tHAwPgaJpZM4RQTmL
.