CI: Test Python 3.8 #3154

effigies · 2020-01-15T16:55:33Z

Closes #3098.

effigies · 2020-01-15T17:57:06Z

Tests stall on nipype/pipeline/plugins/tests/test_legacymultiproc_nondaemon.py::test_run_multiproc_nondaemon_false.

codecov · 2020-02-23T01:58:44Z

Codecov Report

Merging #3154 into master will increase coverage by 0.11%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3154      +/-   ##
==========================================
+ Coverage   64.89%   65.01%   +0.11%     
==========================================
  Files         301      301              
  Lines       39717    39833     +116     
  Branches     5268     5269       +1     
==========================================
+ Hits        25775    25896     +121     
+ Misses      12884    12880       -4     
+ Partials     1058     1057       -1

Flag	Coverage Δ
#unittests	`65.01% <100.00%> (+0.11%)`	⬆️

Impacted Files	Coverage Δ
nipype/info.py	`92.30% <100.00%> (ø)`
nipype/interfaces/minc/minc.py	`85.18% <0.00%> (+0.01%)`	⬆️
nipype/interfaces/fsl/preprocess.py	`71.74% <0.00%> (+0.03%)`	⬆️
nipype/interfaces/ants/registration.py	`73.09% <0.00%> (+0.05%)`	⬆️
nipype/interfaces/base/specs.py	`89.28% <0.00%> (+0.05%)`	⬆️
nipype/interfaces/base/traits_extension.py	`88.29% <0.00%> (+0.06%)`	⬆️
nipype/utils/filemanip.py	`72.94% <0.00%> (+0.06%)`	⬆️
nipype/testing/fixtures.py	`98.70% <0.00%> (+0.09%)`	⬆️
nipype/interfaces/niftyreg/regutils.py	`80.76% <0.00%> (+0.09%)`	⬆️
nipype/interfaces/niftyreg/reg.py	`86.00% <0.00%> (+0.09%)`	⬆️
... and 26 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28ef1af...ab4298a. Read the comment docs.

effigies · 2020-02-23T03:16:04Z

It seems that that test actually breaks the multiprocessing pool. Locally it will pass, but the next test will fail to start up jobs. Might also be causing problems with pytest-xdist.

@oesteban Just in case you have ideas.

oesteban · 2020-02-25T17:35:51Z

Sorry, I was away for the weekend. I'm marking this unread and will come back to it later today.

yarikoptic · 2020-03-23T12:21:00Z

observing the stall of test_run_multiproc_nondaemon_false while building 1.4.2 on Debian (unstable/sid) as of yesterday Mar 22.

yarikoptic · 2020-03-23T17:28:38Z

FWIW test_run_multiproc_nondaemon_true hangs too ;-/

yarikoptic · 2020-03-23T17:44:35Z

and test_run_multiproc

effigies · 2020-03-23T17:51:50Z

To clarify: All for Python 3.8? Or is this a Python 3.7 issue as well?

@oesteban I'm considering marking these as skip if Python >= 3.8.

yarikoptic · 2020-03-23T18:44:12Z

I believe it is 3.8 according to process trace:

                               /bin/sh -c export PYTHONPATH=/build/nipype-1.4.2/.pybuild/cpython3_3.8_nipype/build; \ cd build && \          MPLCONFIGDIR=/build/nipype-1.4.2/build LC_ALL=C.UTF-8 py.test-3 -v --doctest-modules ../nipype
                                 /usr/bin/python3 /usr/bin/py.test-3 -v --doctest-modules ../nipype
                                   /usr/bin/python3 /usr/bin/py.test-3 -v --doctest-modules ../nipype
                                   /usr/bin/python3 /usr/bin/py.test-3 -v --doctest-modules ../nipype

also test_hold_job_until_procs_available stalls.

My current diff with skips now

# quilt diff
Index: nipype-1.4.2/nipype/pipeline/plugins/tests/test_legacymultiproc_nondaemon.py
===================================================================
--- nipype-1.4.2.orig/nipype/pipeline/plugins/tests/test_legacymultiproc_nondaemon.py
+++ nipype-1.4.2/nipype/pipeline/plugins/tests/test_legacymultiproc_nondaemon.py
@@ -134,6 +134,7 @@ def run_multiproc_nondaemon_with_flag(no
     return result
 
 
+@pytest.mark.skipif(True, reason="hangs, known upstream")
 def test_run_multiproc_nondaemon_false():
     """
     This is the entry point for the test. Two times a pipe of several
@@ -152,6 +153,7 @@ def test_run_multiproc_nondaemon_false()
     assert shouldHaveFailed
 
 
+@pytest.mark.skipif(True, reason="hangs, known upstream")
 def test_run_multiproc_nondaemon_true():
     # with nondaemon_flag = True, the execution should succeed
     result = run_multiproc_nondaemon_with_flag(True)
Index: nipype-1.4.2/nipype/pipeline/plugins/tests/test_multiproc.py
===================================================================
--- nipype-1.4.2.orig/nipype/pipeline/plugins/tests/test_multiproc.py
+++ nipype-1.4.2/nipype/pipeline/plugins/tests/test_multiproc.py
@@ -33,6 +33,7 @@ class MultiprocTestInterface(nib.BaseInt
         return outputs
 
 
+@pytest.mark.skipif(True, reason="hangs, known upstream")
 def test_run_multiproc(tmpdir):
     tmpdir.chdir()
 
@@ -114,6 +115,7 @@ def test_no_more_threads_than_specified(
         pipe.run(plugin="MultiProc", plugin_args={"n_procs": max_threads})
 
 
+@pytest.mark.skipif(True, reason="hangs, known upstream")
 def test_hold_job_until_procs_available(tmpdir):
     tmpdir.chdir()

effigies · 2020-03-23T20:05:42Z

This seems concerning, as I think this means the multiprocessing plugins can't be tested with Python 3.8. The best case scenario is that this is an interaction with pytest, but that would be very bad...

yarikoptic · 2020-03-23T20:15:21Z

eh, I am just an unlucky in general, even nipype/tests/test_nipype.py::test_no_et seems to stall (no etelemetry client is installed if that is relevant), although I believe that on previous run it went through.

yarikoptic · 2020-03-24T00:50:34Z

and now they hang on test_provenance... and if I skip that one test_provenance_exists hangs.
given that it is at 99% I wonder if that is just not a side effect of some previous test ...

edit: if I drop my patch altogether and use this PR diff, I am back at hanging at test_run_pbsgraph
edit 2: if I get back to my skips, and also skip test_provenance_exists, I get to the end but the whole process seems to not quit.

effigies · 2020-03-24T16:18:09Z

Do you have qsub installed on your test rig?

yarikoptic · 2020-03-24T17:11:12Z

qsub

~~I hope (checked to be true)~~ not - there is no any pbs/slurm in the build env. Shouldn't tests requiring it skip if no qsub?

effigies · 2020-03-24T18:36:13Z

It's currently being marked xfail. I'll propose a PR to make it skipif.

yarikoptic · 2020-03-24T20:41:50Z

ok, if I take that patch too, I still get stalling on no_et - if I skip it -- on test_provenance... didn't try yet again to extend skips. I will check first if it is etelemetry altogether disabling/removing it.

effigies · 2020-03-24T20:45:52Z

I can't replicate the provenance one. What versions of prov and rdflib are you using? Are you aware of the weird hack of needing to install neurdflib after prov?

satra · 2020-03-24T20:47:15Z

hopefully that hack will be resolved after rdflib 5.0.0 is released.

effigies · 2020-03-24T20:48:11Z

Agreed. Does installing from rdflib@master have any issues?

satra · 2020-03-24T20:56:12Z

yes, at this point the neurdflib PRs are now in master. so rdflib@master can be used instead of neurdflib.

effigies · 2020-03-29T13:18:55Z

Rebased on #3195 to consolidate. @yarikoptic How are things now?

effigies · 2020-03-31T14:30:20Z

I guess we should go ahead with this, as some tests are better than none, but it's probably not a good thing that we can't test (Legacy)MultiProc on Python 3.8+. Do we know if it works at all? Has anybody tried using nipype with Python 3.8 yet?

effigies · 2020-04-01T14:46:22Z

The prov issues seem to be stochastic... I'm going to try dropping the rdflib/neurdflib dependency and switch to master. That will postpone any inclusion for after the 5.0.0 release, but at this point it seems worth it.

effigies · 2020-04-01T16:52:31Z

Not resolved by rdflib@master.

effigies · 2020-04-01T17:54:37Z

Now that I've put a timeout that might give us some debugging output if it does hang again, it won't hang... Rerunning again...

nipype/info.py

yarikoptic · 2020-04-17T17:38:40Z

FWIW, I thought that I might just avoid "stall" by simply using older nipype -- nope, elderly nipype 1.1.9 also gets stuck there, so should add skips.

yarikoptic · 2020-04-17T20:58:10Z

re test_run_pbsgraph:

It's currently being marked xfail. I'll propose a PR to make it skipif.

could/should it also get a timeout then, since xfail is no good if it just hangs there.

effigies · 2020-04-18T14:04:02Z

@yarikoptic Timeouts added to OAR/PBS tests.

doc/requirements.txt

yarikoptic · 2020-04-19T16:52:41Z

FWIW, I have not tried this specific version, but I think I have all the skips - but then good old 1.1.9 still hangs at some later tests (at about 98% completness of the tests). If someone gets down to the underlying issue which causes the stalls -- would be very appreciated and would open possibility to upload recent nipype to debian proper.

effigies · 2020-04-20T18:20:36Z

This gets things going, so I'm going to merge. I'll create an issue for Python 3.8 + multiprocessing, and feel free to open more outstanding issues.

@effigies

> we had to skip some tests in Python 3.8 due to changes in how multiprocessing works: nipy/nipype#3154 > I'm not sure whether Nipype will work in general after Python 3.7. ― @effigies, https://mattermost.brainhack.org/brainhack/pl/kxjyek8wk3nwjmcrdyw969kfde

Pointing to nipy#3154

effigies added this to the 1.4.1 milestone Jan 15, 2020

effigies modified the milestones: 1.4.1, 1.5.0 Jan 26, 2020

effigies mentioned this pull request Feb 23, 2020

REL: 1.5.0 #3174

Merged

17 tasks

effigies force-pushed the ci/py38 branch from 52d4dd1 to 499bb44 Compare February 23, 2020 01:58

effigies changed the base branch from maint/1.4.x to master February 23, 2020 03:11

effigies removed this from the 1.5.0 milestone Feb 24, 2020

effigies force-pushed the ci/py38 branch 2 times, most recently from fa21602 to c17de57 Compare March 23, 2020 20:01

effigies mentioned this pull request Mar 24, 2020

TEST: Clean up tests #3195

Merged

1 task

effigies force-pushed the ci/py38 branch from c17de57 to 250021c Compare March 29, 2020 13:16

effigies force-pushed the ci/py38 branch 2 times, most recently from b554ee7 to c33f0e6 Compare March 31, 2020 14:29

yarikoptic reviewed Apr 17, 2020

View reviewed changes

nipype/info.py Outdated Show resolved Hide resolved

effigies force-pushed the ci/py38 branch 2 times, most recently from a982342 to 5579985 Compare April 18, 2020 14:00

satra reviewed Apr 18, 2020

View reviewed changes

doc/requirements.txt Outdated Show resolved Hide resolved

effigies added 5 commits April 18, 2020 12:50

CI: Test Python 3.8

04fc7b0

CI: Skip some multiproc tests in Py38

7ebe6ec

TEST: Timeout provenance tests after 1min

828cd92

MNT: Pin rdflib >=5

58dde80

TEST: Set timeouts for PBS and OAR tests

ab4298a

effigies force-pushed the ci/py38 branch from c74ed38 to ab4298a Compare April 18, 2020 16:52

effigies merged commit 9a7db4d into nipy:master Apr 20, 2020

effigies deleted the ci/py38 branch April 20, 2020 18:20

effigies mentioned this pull request Apr 20, 2020

Python 3.8 + Multiprocessing #3209

Open

effigies added this to the 1.5.0 milestone May 19, 2020

effigies mentioned this pull request Dec 1, 2020

Restore multiproc tests in Python 3.8+ #3279

Open

yarikoptic added a commit to yarikoptic/nipype that referenced this pull request Sep 13, 2024

Adjust changelog and reflect current state of problems

16a9002

Pointing to nipy#3154

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Test Python 3.8 #3154

CI: Test Python 3.8 #3154

effigies commented Jan 15, 2020 •

edited

Loading

effigies commented Jan 15, 2020

codecov bot commented Feb 23, 2020 •

edited

Loading

effigies commented Feb 23, 2020

oesteban commented Feb 25, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

effigies commented Mar 23, 2020 •

edited

Loading

yarikoptic commented Mar 23, 2020

effigies commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 24, 2020 •

edited

Loading

effigies commented Mar 24, 2020

yarikoptic commented Mar 24, 2020 •

edited

Loading

effigies commented Mar 24, 2020

yarikoptic commented Mar 24, 2020

effigies commented Mar 24, 2020

satra commented Mar 24, 2020

effigies commented Mar 24, 2020

satra commented Mar 24, 2020

effigies commented Mar 29, 2020

effigies commented Mar 31, 2020

effigies commented Apr 1, 2020

effigies commented Apr 1, 2020

effigies commented Apr 1, 2020

yarikoptic commented Apr 17, 2020

yarikoptic commented Apr 17, 2020

effigies commented Apr 18, 2020

yarikoptic commented Apr 19, 2020

effigies commented Apr 20, 2020

CI: Test Python 3.8 #3154

CI: Test Python 3.8 #3154

Conversation

effigies commented Jan 15, 2020 • edited Loading

effigies commented Jan 15, 2020

codecov bot commented Feb 23, 2020 • edited Loading

Codecov Report

effigies commented Feb 23, 2020

oesteban commented Feb 25, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

effigies commented Mar 23, 2020 • edited Loading

yarikoptic commented Mar 23, 2020

effigies commented Mar 23, 2020

yarikoptic commented Mar 23, 2020

yarikoptic commented Mar 24, 2020 • edited Loading

effigies commented Mar 24, 2020

yarikoptic commented Mar 24, 2020 • edited Loading

effigies commented Mar 24, 2020

yarikoptic commented Mar 24, 2020

effigies commented Mar 24, 2020

satra commented Mar 24, 2020

effigies commented Mar 24, 2020

satra commented Mar 24, 2020

effigies commented Mar 29, 2020

effigies commented Mar 31, 2020

effigies commented Apr 1, 2020

effigies commented Apr 1, 2020

effigies commented Apr 1, 2020

yarikoptic commented Apr 17, 2020

yarikoptic commented Apr 17, 2020

effigies commented Apr 18, 2020

yarikoptic commented Apr 19, 2020

effigies commented Apr 20, 2020

effigies commented Jan 15, 2020 •

edited

Loading

codecov bot commented Feb 23, 2020 •

edited

Loading

effigies commented Mar 23, 2020 •

edited

Loading

yarikoptic commented Mar 24, 2020 •

edited

Loading

yarikoptic commented Mar 24, 2020 •

edited

Loading