Skip to content

Update benchmark plotting scripts #1648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 23, 2022
Merged

Update benchmark plotting scripts #1648

merged 5 commits into from
Mar 23, 2022

Conversation

acxz
Copy link
Contributor

@acxz acxz commented Mar 21, 2022

I tried to run the benchmark plotting script to recreate the plot however ran into some difficulties with older syntax.
This PR updates the benchmark plotting syntax to be up to date with the ipynb used for generating the plot and runs successfully for me under Julia v1.7.

I also came across a duplicate benchmarks svg file that is not being used. The svg file being used in the benchmarks.md page is located under _assets/benchmarks/ and not under _assets/images/

@MilesCranmer
Copy link
Member

Awesome!

Since you are updating the plot anyways, do you think you could slightly offset the points along the x-direction as a function of the specific benchmark (e.g., mandelbrot gets x+=0.01, pi_sum gets x+=0.02, etc.)? It would be great to see the overlapped benchmarks.

@acxz
Copy link
Contributor Author

acxz commented Mar 21, 2022

I see, that's smart, yeah let me try to see how that can be done.

Edit: I think for this PR, I just want to update the scripts syntactically so that it produces the exact same graph. I think changing how the plot looks should be another PR. Also as per discussion here maybe we want to move the plotting over to the benchmarks repo anyway.

@ViralBShah
Copy link
Member

ViralBShah commented Mar 23, 2022

@acxz Welcome! Is this ready to merge?

@acxz
Copy link
Contributor Author

acxz commented Mar 23, 2022

@ViralBShah yep!

@ViralBShah ViralBShah merged commit 7569da6 into JuliaLang:main Mar 23, 2022
@MilesCranmer
Copy link
Member

@acxz Any idea why the plot looks like it hasn't changed yet?

https://web.archive.org/web/diff/20220118113840/20220508160242/https://julialang.org/benchmarks/

@acxz
Copy link
Contributor Author

acxz commented May 8, 2022

This PR doesn't update the plot.

@MilesCranmer
Copy link
Member

Ah, I see. What is left to actually upload the plot image itself?

@MilesCranmer
Copy link
Member

MilesCranmer commented May 8, 2022

Maybe we want to first manually update the csv/svg? (since there are some large performance changes since the last update)

Then, an idea would be to set up a GitHub action post step on the Microbenchmarks repo to automatically submit a PR to this repo if the benchmarks are significantly different than the currently saved ones?

@acxz
Copy link
Contributor Author

acxz commented May 8, 2022

We should move this convo to JuliaLang/Microbenchmarks#48

but basically we need to fix: JuliaLang/Microbenchmarks#29 first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants