Skip to content

[FIX] Docker issues and documentation #1825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 17, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
Upcoming release 0.13
=====================

* FIX: Issues in Docker image permissions, and docker documentation (https://github.com/nipy/nipype/pull/1825)
* ENH: Revised all Dockerfiles and automated deployment to Docker Hub
from CircleCI (https://github.com/nipy/nipype/pull/1815)
* FIX: Semaphore capture using MultiProc plugin (https://github.com/nipy/nipype/pull/1689)
Expand Down
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,8 @@ RUN conda config --add channels conda-forge --add channels intel && \
libxslt=1.1.29 \
traits=4.6.0 \
psutil=5.0.1 \
icu=58.1
icu=58.1 && \
find /usr/local/miniconda/ -exec chmod 775 {} +

# matplotlib cleanups: set default backend, precaching fonts
RUN sed -i 's/\(backend *: \).*$/\1Agg/g' /usr/local/miniconda/lib/python3.5/site-packages/matplotlib/mpl-data/matplotlibrc && \
Expand Down
14 changes: 6 additions & 8 deletions circle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,17 @@ dependencies:
- if [[ ! -d ~/examples/nipype-tutorial ]]; then wget --retry-connrefused --waitretry=5 --read-timeout=20 --timeout=15 -t 0 -q -O nipype-tutorial.tar.bz2 "${DATA_NIPYPE_TUTORIAL_URL}" && tar xjf nipype-tutorial.tar.bz2 -C ~/examples/; fi
- if [[ ! -d ~/examples/nipype-fsl_course_data ]]; then wget --retry-connrefused --waitretry=5 --read-timeout=20 --timeout=15 -t 0 -q -O nipype-fsl_course_data.tar.gz "${DATA_NIPYPE_FSL_COURSE}" && tar xzf nipype-fsl_course_data.tar.gz -C ~/examples/; fi
- if [[ ! -d ~/examples/feeds ]]; then wget --retry-connrefused --waitretry=5 --read-timeout=20 --timeout=15 -t 0 -q -O fsl-5.0.9-feeds.tar.gz "${DATA_NIPYPE_FSL_FEEDS}" && tar xzf fsl-5.0.9-feeds.tar.gz -C ~/examples/; fi
- if [[ -e $HOME/docker/image.tar ]]; then docker load -i $HOME/docker/image.tar; fi
- if [[ -e $HOME/docker/image27.tar ]]; then docker load -i $HOME/docker/image27.tar; fi
- if [[ -e $HOME/docker/image35.tar ]]; then docker load -i $HOME/docker/image35.tar; fi
- if [[ -e $HOME/docker/image.tar ]]; then docker load --input $HOME/docker/image.tar; else echo 'No docker image found in cache'; fi
- docker images
- sed -i -E "s/(__version__ = )'[A-Za-z0-9.-]+'/\1'$CIRCLE_TAG'/" nipype/info.py
- e=1 && for i in {1..5}; do docker build -t nipype/nipype:latest --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` --build-arg VCS_REF=`git rev-parse --short HEAD` --build-arg VERSION=$CIRCLE_TAG . && e=0 && break || sleep 15; done && [ "$e" -eq "0" ] :
timeout: 21600
- docker build -f docker/Dockerfile_py27 -t nipype/nipype_test:py27 . :
- e=1 && for i in {1..5}; do docker build -f docker/Dockerfile_py27 -t nipype/nipype_test:py27 . && e=0 && break || sleep 15; done && [ "$e" -eq "0" ] :
timeout: 1600
- docker build -f docker/Dockerfile_py35 -t nipype/nipype_test:py35 . :
- e=1 && for i in {1..5}; do docker build -f docker/Dockerfile_py35 -t nipype/nipype_test:py35 . && e=0 && break || sleep 15; done && [ "$e" -eq "0" ] :
timeout: 1600
- docker save -o $HOME/docker/image.tar nipype/nipype:latest nipype/nipype_test:py27 nipype/nipype_test:py35 :
timeout: 1600
- docker save nipype/nipype:latest > $HOME/docker/image.tar
- docker save nipype/nipype_test:py27 > $HOME/docker/image27.tar
- docker save nipype/nipype_test:py35 > $HOME/docker/image35.tar

test:
override:
Expand Down
43 changes: 32 additions & 11 deletions doc/devel/testing_nipype.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,26 +107,47 @@ This will skip any tests that require matlab.
Testing Nipype using Docker
---------------------------

As of :code:`nipype-0.13`, Nipype is tested inside Docker containers. Once the developer
`has installed the Docker Engine <https://docs.docker.com/engine/installation/>`_, testing
Nipype is as easy as follows::
As of :code:`nipype-0.13`, Nipype is tested inside Docker containers. First, install the
`Docker Engine <https://docs.docker.com/engine/installation/>`_.
Nipype has one base docker image called nipype/nipype, and several additional test images
for various Python versions.

The base nipype/nipype image is built as follows::

cd path/to/nipype/
docker build -f docker/nipype_test/Dockerfile_py27 -t nipype/nipype_test:py27
docker run -it --rm -v /etc/localtime:/etc/localtime:ro \
-e FSL_COURSE_DATA="/root/examples/nipype-fsl_course_data" \
docker build -t nipype/nipype .

This base image contains several useful tools (FreeSurfer, AFNI, FSL, ANTs, etc.) and
a nipype installation, all in Python 3.5.
It is possible to fetch a built image from the latest master branch of nipype
using::

docker run -it --rm nipype/nipype:master


The docker run command will then open the container and offer a bash shell for the
developer.

The additional test images have several test scripts installed. For instance,
to build and run all tests on Python 2.7::

cd path/to/nipype/
docker build -f docker/Dockerfile_py27 -t nipype/nipype_test:py27 .
docker run -it --rm -e FSL_COURSE_DATA="/root/examples/nipype-fsl_course_data" \
-v ~/examples:/root/examples:ro \
-v ~/scratch:/scratch \
-w /root/src/nipype \
nipype/nipype_test:py27 /usr/bin/run_pytest.sh
nipype/nipype_test:py27 /usr/bin/run_pytests.sh

For running nipype in Python 3.5::

cd path/to/nipype/
docker build -f docker/nipype_test/Dockerfile_py35 -t nipype/nipype_test:py35
docker run -it --rm -v /etc/localtime:/etc/localtime:ro \
-e FSL_COURSE_DATA="/root/examples/nipype-fsl_course_data" \
docker build -f docker/Dockerfile_py35 -t nipype/nipype_test:py35 .
docker run -it --rm -e FSL_COURSE_DATA="/root/examples/nipype-fsl_course_data" \
-v ~/examples:/root/examples:ro \
-v ~/scratch:/scratch \
-w /root/src/nipype \
nipype/nipype_test:py35 /usr/bin/run_pytest.sh
nipype/nipype_test:py35 /usr/bin/run_pytests.sh

The last two examples assume that the example data is downladed into ~/examples and
the ~/scratch folder will be created if it does not exist previously.