Skip to content

BUG: Progressbar does not update on streamed stdout #7757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tvwenger opened this issue Apr 15, 2025 · 1 comment
Open

BUG: Progressbar does not update on streamed stdout #7757

tvwenger opened this issue Apr 15, 2025 · 1 comment
Labels

Comments

@tvwenger
Copy link
Contributor

Describe the issue:

I'm running pymc in a condor HTC environment, where my models are sampled on compute nodes and the output is streamed or copied back to the host.

Without any options to checkpoint sampling with pymc, I want a way to check the status of the sampling even though I don't have access to stdout on the compute nodes.

Currently, we only pass refresh=True on the last iteration, so the only time condor streams stdout back to the host machine is after the last iteration. This makes me sad because no updates are streamed back to the host during sampling, so I have no way to satisfy my curiosity on the expected runtime.

In summary, the issue is that progressbar=True doesn't actually print the progress bar to stdout in such a way that condor can stream it until sampling is completed.

I want refresh=True here for all iterations, not just the last:

pymc/pymc/util.py

Lines 893 to 908 in 2842401

self._progress.update(
self.tasks[chain_idx],
completed=draw,
draws=draw,
sampling_speed=speed,
speed_unit=unit,
**more_updates,
)
if is_last:
self._progress.update(
self.tasks[chain_idx],
draws=draw + 1 if not self.combined_progress else draw,
**more_updates,
refresh=True,
)

Reproduceable code example:

pm.sample() # on HTC

Error message:

PyMC version information:

Latest

Context for the issue:

No response

@tvwenger tvwenger added the bug label Apr 15, 2025
@Hashcode-Ankit
Copy link

@tvwenger @ricardoV94 is this need to be fixed?, Let me know will raise a pr for it.
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants