runtime_link.go:8: could not import runtime (open : no such file or directory)

[xhd2015]

This is an issue from github workflow:

It can be replicated in local with: go run ./script/run-test --include go1.19.13 --with-setup-only --log-debug -v ./runtime/test/build/modules_with_trap_all/service

It seems caused by -overlay, after removing the overlay content, the build passes.

[xhd2015]

After adding some debug, found that GOCACHE are different:

• without --with-setup-only
  • GOCACHE=/tmp/xgo/go-instrument/go1.19.13_Us_xh_Pr_xh_xg_go_go_7ddb57fe/build-cache
• with --with-setup-only
  • GOCACHE=/tmp/xgo/go-instrument/go1.19.13_Us_xh_.x_go_go_go_539cb0a6/build-cache

If force --with-setup-only to use the upper GOCACHE, the test passes.

[xhd2015]

fixed in 2688f96

[xhd2015]

The fix significantly reduces the --with-setup overhead with cache, because GOCACHE now becomes consistent, there is no useless extra tests running.

[xhd2015]

However, this seems likely not only related to inconsistent build cache, but a bug related cache+overlay combination:

The follow image shows:

• first build, --reset-instrument without --trap-all, this clears the build cache
• second build, --trap-all, the reuses the build cache with all files having overlay
  

[xhd2015]

The bug could be go's mishandling of GOCACHE and -overlay.

Workaround could be making xgo's instrument to be idempotent:

• same file set + flags -> same build cache

flags affecting build cache:

• --trap pkg
• --mock-rule
• --trap-all
• -cover
• run or test
• the main module
• -tags

[xhd2015]

The go's build action computes cache:

https://github.com/golang/go/blob/a9d9b5570964c2c2ef07992150174bd3d1b2d891/src/cmd/go/internal/work/exec.go#L385

	inputFiles := str.StringList(
		p.GoFiles,
		p.CgoFiles,
		p.CFiles,
		p.CXXFiles,
		p.FFiles,
		p.MFiles,
		p.HFiles,
		p.SFiles,
		p.SysoFiles,
		p.SwigFiles,
		p.SwigCXXFiles,
		p.EmbedFiles,
	)
	for _, file := range inputFiles {
		fmt.Fprintf(h, "file %s %s\n", file, b.fileHash(filepath.Join(p.Dir, file)))
	}

[xhd2015]

After per-flag cache:

[xhd2015]

We can't fully suppress this failure. Currently if user encounter such error, add a --reset-instrument would solve it.

It already shows that in some cases with some specific building sequence, this bug can be replicated.

[xhd2015]

It's probably due to go compile's missing Deps in import cfg.

A possible solution would be: always import runtime.

[xhd2015]

Always import runtime, this fixed everything:

[xhd2015]

fixed in
09acbf9#diff-1455ecf704192d385801798bf96fbe01b3efeca6aa92d6ff200cf78626c5e879

The issue occurs after go1.19, tests show go1.17 and go1.18 are ok:

xgo/runtime/test/build/overlay_build_cache_error_with_go/gomod_test.go

Lines 39 to 45 in 26f967c

	minor, err := getGo1MinorVersion()
	if err != nil {
	t.Fatalf("failed to get go minor version: %v", err)
	}
	if minor >= 19 {
	if afterErr == nil {
	t.Errorf("expect cache+overlay combination error, actual nil")

• Report the bug to go team
  • build: could not import runtime (open : no such file or directory) golang/go#73368

[xhd2015]

go:vet also fails:

[xhd2015]

fixed in #323 5e9b1fb

see golang/go#73368 (comment)