Skip to content

fix: accept invalid headers with a space #1953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 19, 2025

Conversation

ksw2000
Copy link
Contributor

@ksw2000 ksw2000 commented Feb 12, 2025

Fix #1917

Make behavior consistent with net/http by allowing header keys and trailers containing spaces without canonicalizing them.

Change the behavior of headerScanner.next(). When parsing headers, we do not normalize the header key.
In addition, before normalizing the key, check whether the key contains a space. If it does, disable normalization forcibly.

Make behavior consistent with net/http by allowing header keys and trailers containing spaces without canonicalizing them
@ksw2000
Copy link
Contributor Author

ksw2000 commented Feb 12, 2025

I think the fuzzy test run failure is independent of my changes since the previous commit passed. The latest commit is a refactor to fix linter problems. In addition, the failed test - fuzzResponseReadLimitBody is irrelevant to the header parser.

@ksw2000 ksw2000 marked this pull request as ready for review February 12, 2025 17:08
@erikdubbelboer
Copy link
Collaborator

No, what the fuzzer found is caused by your changes. Here is the test case from the fuzzer that passes on master but panics on your branch:

func TestReadLimitBodyExample(t *testing.T) {
	body := []byte{0xa1, 0xfd, 0xdf, 0x77, 0x20, 0x35, 0xa, 0x54, 0x72, 0x61, 0x6e, 0x73, 0x66, 0x65, 0x72, 0x2d, 0x45, 0x6e, 0x63, 0x6f, 0x64, 0x69, 0x6e, 0x67, 0x3a, 0xb0, 0xa, 0xa, 0x30, 0x52, 0xd, 0xa, 0x3a, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x89, 0x3a, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x89, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0xa, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0xa, 0x20, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0x20, 0x3a, 0x3a, 0xa, 0xa, 0x20, 0x20, 0xa, 0x20, 0xa, 0x20, 0x3a, 0x20, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x20, 0x3a, 0x20, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x2e}

	res := AcquireResponse()
	defer ReleaseResponse(res)

	_ = res.ReadLimitBody(bufio.NewReader(bytes.NewReader(body)), 24922)
}

@ksw2000
Copy link
Contributor Author

ksw2000 commented Feb 18, 2025

This bug is caused by two issues:

  1. When refactoring parseTrailer, the usage of continue jumped to the wrong loop, and there was also a logical error in using disableNormalizing. This is not the main problem.
  2. In the original implementation, if the key in the trailer was " \n", the program would return an error because of the space. However, after the modification, spaces were allowed, but since the initial logic did not block "\n", the modified program would mistakenly allow this error. In the new implementation, parseTrailer follows the approach of parseHeaders by calling validHeaderFieldByte to check for unexpected characters.

I am trying to minimize the test you provided and write it back as a unit test.

@erikdubbelboer erikdubbelboer merged commit 086a114 into valyala:master Feb 19, 2025
15 checks passed
@erikdubbelboer
Copy link
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ignoring parsing errors of parseHeaders in client.go
2 participants