Skip to content

Should unit tolerance be a % rather than fixed at 1e-5? #450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tmilnthorp opened this issue Jun 20, 2018 · 2 comments
Closed

Should unit tolerance be a % rather than fixed at 1e-5? #450

tmilnthorp opened this issue Jun 20, 2018 · 2 comments

Comments

@tmilnthorp
Copy link
Collaborator

I noticed that when adding the Femto and Peta prefixes to PowerDensity that the tests fail.

[1 W/m³].As( PowerDensityUnit.FemtowattPerCubicMeter ) is calculating 1 / 1e-15 = 999999999999999.87

Obviously this isn't within 1e-5 of 1e15 (actual difference is 0.125).

For a number this large that difference is 0.00000000000001.25% :)

I know that I could override the tolerance in the tests, but perhaps we should think about changing the tolerance across the board? For very small numbers, the opposite is true: 1e-5 may be too large.

@gojanpaolo
Copy link
Contributor

See Relative tolerance instead of absolute tolerance #391

@tmilnthorp
Copy link
Collaborator Author

Ah thank you - I looked through the list but must have missed this. I will close this and reference that one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants