Skip to content

[ENH] Update BIDSDataGrabber for pybids 0.7 #2737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Jan 23, 2019
Merged

[ENH] Update BIDSDataGrabber for pybids 0.7 #2737

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
ffd0d8e
Lock travis Python 0.6.5
adelavega Oct 9, 2018
8db5111
Merge master
adelavega Oct 15, 2018
801b295
Update BIDSDataGrabber for pybids 0.7, including tests, and derivativ…
adelavega Oct 15, 2018
1d267c8
Fix test, outfields
adelavega Oct 15, 2018
b0121c0
Refer to proper input, index_derivatives
adelavega Oct 16, 2018
cc89c58
Datatype not modality, in pybids tests
adelavega Oct 19, 2018
b8aacce
Merge master
adelavega Jan 11, 2019
a0e92f1
Lint BIDSGrabbder
adelavega Jan 11, 2019
0f94d9d
Bump pybids version to 0.7.0
adelavega Jan 11, 2019
77a0510
use https for travis pybids requirement
adelavega Jan 11, 2019
26a57a6
Merge branch 'master' into update-pybids
adelavega Jan 14, 2019
2a84cab
Merge branch 'master' into update-pybids
adelavega Jan 14, 2019
5590d4d
Re-enable editable mode, and run tests in Python 2
adelavega Jan 14, 2019
3c21466
Merge branch 'master' of https://github.com/nipy/nipype into update-p…
adelavega Jan 16, 2019
d2e53fb
Merge master and fix merge conflict
adelavega Jan 22, 2019
b75bf7c
Update nipype/info.py
effigies Jan 22, 2019
7105cb5
Remove return type named tuple
adelavega Jan 22, 2019
09a1a43
Merge branch 'update-pybids' of github.com:adelavega/nipype into upda…
adelavega Jan 22, 2019
75a5a2b
Update nipype/interfaces/io.py
effigies Jan 22, 2019
139c163
Merge branch 'update-pybids' of github.com:adelavega/nipype into upda…
adelavega Jan 22, 2019
b623cdd
remake specs
adelavega Jan 22, 2019
aeda06a
Add FSL auto test
adelavega Jan 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ before_install:

- travis_retry pip install -r requirements.txt
- travis_retry pip install grabbit==0.1.2
- travis_retry git clone -b 0.6.5 https://github.com/INCF/pybids.git ${HOME}/pybids && pip install -e ${HOME}/pybids
- travis_retry git clone https://github.com/INCF/pybids.git ${HOME}/pybids && pip install -e ${HOME}/pybids

install:
- travis_retry pip install $EXTRA_PIP_FLAGS -e .[$NIPYPE_EXTRAS]
Expand Down
23 changes: 14 additions & 9 deletions nipype/interfaces/io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2761,8 +2761,10 @@ class BIDSDataGrabberInputSpec(DynamicTraitedSpec):
desc='Generate exception if list is empty '
'for a given field')
return_type = traits.Enum('file', 'namedtuple', usedefault=True)
strict = traits.Bool(desc='Return only BIDS "proper" files (e.g., '
'ignore derivatives/, sourcedata/, etc.)')
index_derivatives = traits.Bool(False, usedefault=True,
desc='Index derivatives/ sub-directory')
extra_derivatives = traits.List(Directory(exists=True),
desc='Additional derivative directories to index')


class BIDSDataGrabber(IOBase):
Expand All @@ -2788,7 +2790,7 @@ class BIDSDataGrabber(IOBase):
are filtered on common entities, which can be explicitly defined as
infields.

>>> bg = BIDSDataGrabber(infields = ['subject'], outfields = ['dwi'])
>>> bg = BIDSDataGrabber(infields = ['subject'])
>>> bg.inputs.base_dir = 'ds005/'
>>> bg.inputs.subject = '01'
>>> bg.inputs.output_query['dwi'] = dict(modality='dwi')
Expand All @@ -2810,8 +2812,10 @@ def __init__(self, infields=None, **kwargs):

if not isdefined(self.inputs.output_query):
self.inputs.output_query = {
"func": {"modality": "func", 'extensions': ['nii', '.nii.gz']},
"anat": {"modality": "anat", 'extensions': ['nii', '.nii.gz']},
"bold": {"datatype": "func", "suffix": "bold",
"extensions": ["nii", ".nii.gz"]},
"T1w": {"datatype": "anat", "suffix": "T1w",
"extensions": ["nii", ".nii.gz"]},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a problem for this PR, but we may want to think about making it a much more generally usable default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Such as? you mean returning more things?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Ideally something that will contain most, if not all, files, and can just be queried sensibly. I feel like specifying the query is more of a power-user move, so making sure there's a reasonable way to ask for the files most people will want would be good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me, although I worry about about performance if a bunch of unnecessary queries are made. Although the indexing itself is probably the slowest part, so it should probably be fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're at it, other fields you'd want? brain_mask is another one I can think of for fmri. I would like to hear from ppl doing preprocessing and and DWI as I'm unfamiliar with what is typically expected.

}

# If infields is empty, use all BIDS entities
Expand All @@ -2838,10 +2842,11 @@ def _run_interface(self, runtime):
return runtime

def _list_outputs(self):
exclude = None
if self.inputs.strict:
exclude = ['derivatives/', 'code/', 'sourcedata/']
layout = bidslayout.BIDSLayout(self.inputs.base_dir, exclude=exclude)
layout = bidslayout.BIDSLayout(self.inputs.base_dir,
derivatives=self.inputs.derivatives)

if isdefined(self.inputs.extra_derivatives):
layout.add_derivatives(self.inputs.extra_derivatives)

# If infield is not given nm input value, silently ignore
filters = {}
Expand Down
8 changes: 4 additions & 4 deletions nipype/interfaces/tests/test_io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -591,9 +591,9 @@ def test_bids_grabber(tmpdir):
bg.inputs.base_dir = os.path.join(datadir, 'ds005')
bg.inputs.subject = '01'
results = bg.run()
assert 'sub-01_T1w.nii.gz' in map(os.path.basename, results.outputs.anat)
assert 'sub-01_T1w.nii.gz' in map(os.path.basename, results.outputs.T1w)
assert 'sub-01_task-mixedgamblestask_run-01_bold.nii.gz' in \
map(os.path.basename, results.outputs.func)
map(os.path.basename, results.outputs.bold)


@pytest.mark.skipif(not have_pybids,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the @pytest.mark.skipif(sys.version_info < (3, 0), lines a couple below, and for the other tests, too? Pybids does still support Python 2, and we should make sure that it works, as long as it does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I guess...

Expand Down Expand Up @@ -633,9 +633,9 @@ def test_bids_infields_outfields(tmpdir):
for outfield in outfields:
assert(outfield in bg._outputs().traits())

# now try without defining outfields, we should get anat and func for free
# now try without defining outfields
bg = nio.BIDSDataGrabber()
for outfield in ['anat', 'func']:
for outfield in ['T1w', 'bold']:
assert outfield in bg._outputs().traits()


Expand Down