Skip to content

Fix: #13420 - Add cache for nodes._check_initialpaths_for_relpath #13422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2025

Conversation

sashko1988
Copy link
Contributor

@sashko1988 sashko1988 commented May 13, 2025

  • add lru_cache to nodes._check_initialpaths_for_relpath
  • update its test

fixes #13420

@psf-chronographer psf-chronographer bot added the bot:chronographer:provided (automation) changelog entry is part of PR label May 13, 2025
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good as a workaround - i recon the path types doo too much for their own good

the proposal in #10824 would be a more invasive solution to the intended problem as it would "root" node-ids in the collection root

@@ -594,7 +598,7 @@ def __init__(
try:
nodeid = str(self.path.relative_to(session.config.rootpath))
except ValueError:
nodeid = _check_initialpaths_for_relpath(session, path)
nodeid = _check_initialpaths_for_relpath(session._initialpaths, path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bluetech @nicoddemus seeing this right in place i'm thinking - what if they cooperated with the parent

with the introduction of directory nodes i believe the case is a lot more clear cut and possibly could avoid the common path calls all-together

also it seems like going for directory names more times could potentially safe a lot of time as commonly there are many files in the same directory

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if they cooperated with the parent with the introduction of directory nodes i believe the case is a lot more clear cut and possibly could avoid the common path calls all-together

Not sure I follow, we already have Directory nodes:

class Directory(FSCollector, abc.ABC):

@sashko1988 sashko1988 changed the title Bugfix 13420: Add cache for nodes._check_initialpaths_for_relpath Bugfix #13420: Add cache for nodes._check_initialpaths_for_relpath May 13, 2025
@sashko1988 sashko1988 changed the title Bugfix #13420: Add cache for nodes._check_initialpaths_for_relpath Fix: #13420 - Add cache for nodes._check_initialpaths_for_relpath May 13, 2025
@sashko1988
Copy link
Contributor Author

this looks good as a workaround - i recon the path types doo too much for their own good

the proposal in #10824 would be a more invasive solution to the intended problem as it would "root" node-ids in the collection root

I agree. This is rather a quick workaround that shouldn't do any harm. Don't know how long #10824 will take. But it would be good to see my fix in the next patch release.

@sashko1988
Copy link
Contributor Author

@RonnyPfannschmidt do I need to add anything else so Changelog entry check starts running?

@RonnyPfannschmidt
Copy link
Member

@webknjaz any idea why its down?

@webknjaz
Copy link
Member

Dunno. If it's a one-time occasion, might be a network error or something that prevented GH from sending or receiving data. We can retrigger it to see.

@webknjaz webknjaz added skip news used on prs to opt out of the changelog requirement and removed skip news used on prs to opt out of the changelog requirement labels May 14, 2025
@webknjaz
Copy link
Member

Looks like it was a fluke 🤷‍♂️

@sashko1988
Copy link
Contributor Author

@RonnyPfannschmidt @webknjaz folks, I might be blind, but I don't see the Merge button. Is there anything else missing?

@webknjaz
Copy link
Member

@sashko1988 only people with a commit bit see it. GH doesn't allow arbitrary users put code into arbitrary repos they don't own.

@sashko1988
Copy link
Contributor Author

@webknjaz, fair enough. Would you happen to know when this PR will be merged - if it will be merged at all?

@webknjaz
Copy link
Member

You got an approval from Ronny. Unless someone objects, he'll probably merge it soon. But it may hang for some while.

@RonnyPfannschmidt RonnyPfannschmidt merged commit cfbe319 into pytest-dev:main May 15, 2025
28 checks passed
@RonnyPfannschmidt
Copy link
Member

Should this be backported?

@sashko1988
Copy link
Contributor Author

@RonnyPfannschmidt, yes, please! It would be good to have that fix in 8.3.x!

@RonnyPfannschmidt RonnyPfannschmidt added the backport 8.3.x apply to PRs at any point; backports the changes to the 8.3.x branch label May 16, 2025
Copy link

patchback bot commented May 16, 2025

Backport to 8.3.x: 💚 backport PR created

✅ Backport PR branch: patchback/backports/8.3.x/cfbe319d0c98302eae507622da8908be351fed2e/pr-13422

Backported as #13425

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback bot pushed a commit that referenced this pull request May 16, 2025
…3422)

* add lru_cache to nodes._check_initialpaths_for_relpath
update tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Oleksandr Zavertniev <oleksandr.zavertniev@yellowbrick.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
(cherry picked from commit cfbe319)
nicoddemus pushed a commit that referenced this pull request May 16, 2025
…3422) (#13425)

* add lru_cache to nodes._check_initialpaths_for_relpath
update tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------



(cherry picked from commit cfbe319)

Co-authored-by: Sashko <20253875+sashko1988@users.noreply.github.com>
Co-authored-by: Oleksandr Zavertniev <oleksandr.zavertniev@yellowbrick.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 8.3.x apply to PRs at any point; backports the changes to the 8.3.x branch bot:chronographer:provided (automation) changelog entry is part of PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Slow collection time when tests are not in a relative folder to the current working folder
4 participants