Catch misleading ImportError #5098
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If pip cache data get out of sync with the Python version used to create them, it can cause a misleading pickling error. When serialize.py tries to deserialize pickled cache data, packages with an out-of-date path will generate an ImportError that may refer to any number of imports, some of which can be paths that look like external packages, such as urllib3 or requests. Example: ``` File "/Users/higgins/Projects/cfgov-refresh/.tox/unittest-fast/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py",line 168, in _loads_v1 cached = pickle.loads(data) ImportError: No module named urllib3._collections ``` It can be hard to understand and get out of this state, because attempting to load a cached package in any local virtual environment will hit an import error that doesn't seem related. Also, tox tests that generate a new environment will also bomb if they try to install a cached package. Adding ImportError as an exception allows a user to get out of this state by skipping the cache, the way a ValueError does, and allows the cache to get back in sync for users who have upgraded Python. It appears [this commenter](#5079 (comment)) on #5079 encountered the same pickling error. Users who realize it's a cache problem can use `--no-cache-dir` on all their installs, or clear the system pip cache, but a self-healing solution would help toilers like me.
|
Hey @higs4281! Sorry for the delay in responding to this PR. This is a fix in an vendored dependency |
|
Ah -- thanks for the explanation. I'll close. |
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
lock
bot
added
the
auto-locked
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If pip cache gets out of sync with the Python version used to create
it, it can cause a misleading pickling error.
When serialize.py tries to deserialize pickled cache data, packages
with an out-of-date path will generate an ImportError that may refer to
any number of imports that can look like missing external packages,
such as urllib3 or requests. Example:
In this state, attempting to load a cached package in any local virtual environment
will hit an import error that doesn't seem related. Also, tox tests that generate
a new environment will also bomb if they try to install a cached package.
Adding ImportError as an exception lets a user escape this state
by skipping the cache, the way a ValueError does.
It appears this commenter on #5079 encountered the same pickling error.
Users who realize it's a cache problem can use
--no-cache-diron alltheir installs, or clear the system pip cache, but a self-healing solution
would help toilers like me.