cmd/go: module project cannot update go.sum file in readonly mode

[davecheney]

What version of Go are you using (go version)?

$ go version
go version go1.13 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

In attempting to upgrade my project from Go 1.12 to Go 1.13 our project will not build because -mod=readonly wants to update the go.sum file but is prevented from doing so by its own logic.

Go 1.12 happily made whatever changes were necessary to the sum file,

% rm go.sum
% env GO111MODULE=on go1.12  install -mod=readonly -v -tags "oidc gcp" github.com/heptio/contour/cmd/contour
% wc -l go.sum
     275 go.sum

What did you expect to see?

Build succeeds

What did you see instead?

Project, https://github.com/heptio/contour

$ make check
go install -mod=readonly -v -tags "oidc gcp" github.com/heptio/contour/cmd/contour
go: updates to go.sum needed, disabled by -mod=readonly
Makefile:35: recipe for target 'install' failed

[bcmills]

This change was an intentional bug-fix: see #25530 (comment) and #30667.

-mod=readonly is typically used in CI systems, and the expectation is that users will build the program locally (or run go mod tidy) before pushing to CI, either of which will populate the relevant entries in go.sum.

[bcmills]

Could you share more detail about your use-case for using -mod=readonly with an incomplete go.mod go.sum file?

[davecheney]

I don’t believe our go.mod file is incomplete, go mod tidy makes no changes. How can I help answer this question?

…

On 4 Sep 2019, at 11:49, Bryan C. Mills ***@***.***> wrote: Could you share more detail about your use-case for using -mod=readonly with an incomplete go.mod file? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[zikaeroh]

If the behavior difference is purely that 1.12 would create a go.sum if not present with -mod=readonly specified, and 1.13 doesn't, then it might be noteworthy that the repo linked in the issue has go.sum in its .gitignore, so it wouldn't exist in a fresh checkout or CI and would now fail with -mod=readonly regardless of go.mod's completeness.

[davecheney]

That is correct. When we moved to modules we found the sum file was constantly dirty, this broke our ci caching and caused lots of turds in pull requests. #31978 suggested that go.sum was not required, perhaps that advice should be revised for projects building in readonly mode (again to stop unintended go.mod changes in pull requests)

…

On 4 Sep 2019, at 13:16, zikaeroh ***@***.***> wrote: If the behavior difference is purely that 1.12 would create a go.sum if not present with -mod=readonly specified, and 1.13 doesn't, then it might be noteworthy that the repo linked in the issue has go.sum in its .gitignore, so it wouldn't exist in a fresh checkout or CI and would now fail with -mod=readonly regardless of go.mod's completeness. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[bcmills]

(Sorry, I typo'd go.mod in place of go.sum in my earlier reply.)

[bcmills]

The purpose of the go.sum file is to ensure that all developers resolve the same versions to the same code. It only fulfills that purpose if it is checked in.

If the developers have run tests in all applicable configurations, or have run go mod tidy, then the go.sum file should only change if there is a corresponding code change that adds an import of a previously-unused package. If you are seeing otherwise, please file an issue with steps to reproduce.

[bcmills]

A go.sum file is not strictly required, just as a complete go.mod file is not strictly required — in both cases, the go command will fill in the missing information as it runs. However, -mod=readonly explicitly requests that the go command not fill in missing information.

[jayschwa]

I have encountered a similar issue. go mod tidy from Go 1.13 places fewer lines in go.mod and go.sum than Go 1.12.9. As a result, our CI that runs Go 1.12.9 fails with cannot update go.sum in readonly mode. For now, we will use the Go 1.12.9 variant of go.sum as our source of truth in version control (until we switch our CI and prod stack to 1.13).

I'm not sure why the different Go versions produce different go.mod and go.sum files. Perhaps one of our dependencies is doing different things via version-specific build tags.

[bcmills]

@jayschwa, the difference in go mod tidy behavior is likely due to the fix for #31248.

[bcmills]

#31978 suggested that go.sum was not required, perhaps that advice should be revised for projects building in readonly mode

There are now at least two prominent locations where we advise to check in the go.sum file:

https://blog.golang.org/using-go-modules:

Both go.mod and go.sum should be checked into version control.

http://golang.org/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file:

Typically your module's go.sum file should be committed along with your go.mod file.

[bcmills]

Given the above documentation, and the fact that the current behavior seems to align with its intent (per #30667), I don't think there is anything more to be done here.

If you find further documentation that needs to be updated, or if you find that the go.sum file becomes dirty when you don't expect (especially in a way that go mod tidy does not correct), please do file a new issue for that.

(At the moment #33008 is the only such issue I am aware of.)

[bcmills]

#33784 is another known source of spurious go.sum updates.