Set self-adjusting deadline for connection writing #16068
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In go-gitea#16055 it appears that the simple 5s deadline doesn't work for large file writes. Now we can't - or at least shouldn't just set no deadline as go will happily let these connections block indefinitely. However, what seems reasonable is to set some minimum rate we expect for writing. This PR suggests the following algorithm: * Every write has a minimum timeout of 5s (adjustable at compile time.) * If there has been a previous write - then consider its previous deadline, add half of the minimum timeout + 2s per kb about to written. * If that new deadline is after the minimum timeout use that. Fix go-gitea#16055 Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Codecov Report
@@ Coverage Diff @@
## main #16068 +/- ##
==========================================
- Coverage 44.06% 44.06% -0.01%
==========================================
Files 682 682
Lines 82460 82465 +5
==========================================
+ Hits 36338 36340 +2
- Misses 40205 40209 +4
+ Partials 5917 5916 -1
Continue to review full report at Codecov.
|
GiteaBot
added
the
lgtm/need 2
fnetX
reviewed
Jun 4, 2021
zeripath
commented
Jun 4, 2021
zeripath
commented
Jun 4, 2021
|
It would also help if it actually compiled! |
Signed-off-by: Andrew Thornton <art27@cantab.net>
zeripath
added a commit
to zeripath/gitea
that referenced
this pull request
Jun 9, 2021
In go-gitea#16055 it appears that the simple 5s deadline doesn't work for large file writes. Now we can't - or at least shouldn't just set no deadline as go will happily let these connections block indefinitely. However, what seems reasonable is to set some minimum rate we expect for writing. This PR suggests the following algorithm: * Every write has a minimum timeout of 5s (adjustable at compile time.) * If there has been a previous write - then consider its previous deadline, add half of the minimum timeout + 2s per kb about to written. * If that new deadline is after the minimum timeout use that. Fix go-gitea#16055 Signed-off-by: Andrew Thornton <art27@cantab.net>
GiteaBot
added
lgtm/need 1
Merged
lafriks
approved these changes
Jun 10, 2021
GiteaBot
added
lgtm/done
lafriks
pushed a commit
that referenced
this pull request
Jun 10, 2021
In #16055 it appears that the simple 5s deadline doesn't work for large file writes. Now we can't - or at least shouldn't just set no deadline as go will happily let these connections block indefinitely. However, what seems reasonable is to set some minimum rate we expect for writing. This PR suggests the following algorithm: * Every write has a minimum timeout of 5s (adjustable at compile time.) * If there has been a previous write - then consider its previous deadline, add half of the minimum timeout + 2s per kb about to written. * If that new deadline is after the minimum timeout use that. Fix #16055 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
AbdulrhmnGhanem
pushed a commit
to kitspace/gitea
that referenced
this pull request
Aug 10, 2021
* Set self-adjusting deadline for connection writing In go-gitea#16055 it appears that the simple 5s deadline doesn't work for large file writes. Now we can't - or at least shouldn't just set no deadline as go will happily let these connections block indefinitely. However, what seems reasonable is to set some minimum rate we expect for writing. This PR suggests the following algorithm: * Every write has a minimum timeout of 5s (adjustable at compile time.) * If there has been a previous write - then consider its previous deadline, add half of the minimum timeout + 2s per kb about to written. * If that new deadline is after the minimum timeout use that. Fix go-gitea#16055 * Linearly increase timeout * Make PerWriteTimeout, PerWritePerKbTimeouts configurable Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In #16055 it appears that the simple 5s deadline doesn't work for large
file writes. Now we can't - or at least shouldn't just set no deadline
as go will happily let these connections block indefinitely. However,
what seems reasonable is to set some minimum rate we expect for writing.
This PR suggests the following algorithm:
deadline, add half of the minimum timeout + 2s per kb about to written.
Fix #16055
Signed-off-by: Andrew Thornton art27@cantab.net