rustc panics due to stale incremental compilation info #120582
Comments
stinodego
added
C-bug
stinodego
changed the title
|
Hi thanks for the report! At a first glance this doesn't seem like a clippy issue but a compiler internals issue. @rustbot claim |
Maybe some stale incremental compilation state, that caused |
|
I pulled that specific commit, and couldn't reproduce with the It seems that the issue may have happened during compilation as @flip1995 suggested, and not during clippy run. I run this in the repo dir:
I will try going through the commit history. |
That's what I am thinking. At some point I ran into the panic, then I was reverting changes but it kept happening, and then after running Unfortunately that made it impossible for me to reliably reproduce, given my limited knowledge of this subject. |
|
What I am thinking is that somehow cargo build had access to previous files that resulted in a successful compilation, and running clippy caused a rebuild with a stale state. From the stacktrace it appears that the issue is related to the query engine ie type checking, and the compilation error I am getting is related to that as well 🤔. It looks unlikely, but if you run something like Accessing deleted files by having references to them is a thing that can occur in linux systems. This example comes to mind https://lkml.iu.edu/hypermail/linux/kernel/2401.3/05626.html |
|
Should we close this issue or move it to |
|
I think it's best to simply report to Til you can move issues around. |
rustbot
added
the
needs-triage
flip1995
changed the title
saethlin
removed
the
needs-triage
|
@partiallytyped Do you still want to be assigned to this issue? |
|
@saethlin This is unfortunately beyond my skills at the moment. Please take it away if you want. |
|
The crash here is happening when loading the incremental compilation dependency graph. I've worked on that code a fair bit. What jumps out at me is that we blindly assume that the last 16 bytes are the number of edges and number of nodes. Usually when decoders crash with a capacity overflow or allocation error it's because they're trying to pre-allocate for an array, and the decoder read some random bytes instead of a length. The encoding format tries to adapt itself to minimize the number of zero bytes it writes, so the odds are actually quite good that if you grab a random 8 bytes of the dep graph file and then try to convert it from an element count to a number of bytes for an array's whole allocation (as the code in RawVec that crashes does), that multiplication will overflow. So, it seems likely to me that somehow your dep graph file was incomplete. I've been trying to fix a situation that looks a whole lot like this, where if you ran out of disk space during a compile, rustc would fail to report an error correctly and just tell Cargo it finished compilation. Then a new rustc process would start up and try to load a truncated file and crash. I'm afraid this looks exactly like that. @stinodego is there any chance you ran out of disk space recently? From the rustc version you reported, it looks like you're on a nightly before #119510 landed. If there's no way you ran out of disk space recently, I have a patch that would at least let us detect that there's a truncated file and crash more helpfully. |
|
I guess it's possible - I run on WSL and the virtual disk space gets filled up sometimes. |
|
w/r/t the Serenity example - disk space isn't an issue in my case, I can go check rn but my disk should be like 90-95% unused |
…etrochenkov Detect truncated DepGraph files I suspect that the following issues are caused by truncated incr comp files: * rust-lang#120582 * rust-lang#121499 * rust-lang#122210 We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`. Now theoretically since rust-lang#119510 merged I have fixed the out-of-disk issues and yet in rust-lang#120894 (comment) I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
Rollup merge of rust-lang#122245 - saethlin:check-dep-graph-size, r=petrochenkov Detect truncated DepGraph files I suspect that the following issues are caused by truncated incr comp files: * rust-lang#120582 * rust-lang#121499 * rust-lang#122210 We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`. Now theoretically since rust-lang#119510 merged I have fixed the out-of-disk issues and yet in rust-lang#120894 (comment) I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
|
I think the root cause of this issue was rustc incorrectly handling/reporting I/O errors. I've landed a number of PRs to address the problem; the most recent of those is #119510 which is now part of the latest stable toolchain, 1.77. Therefore, even though I have not reproduced exactly what you've reported here, I'm going to close this because I am reasonably confident that this bug is now fixed on stable. Of course it is possible that I am wrong and the bug you've reported here is not fixed. If you happen to run into problems like this in 1.77 or later please do not hesitate to open a new issue. |
|
All right, thanks a lot! |
Summary
Unfortunately I cannot reproduce this after running
cargo clean.I was implementing the
Nottrait rather than implementing.not()directly and at some point I got this panick when running clippy.cargo buildstill worked fine.Here's the branch I was working on:
https://github.com/pola-rs/polars/tree/72aa9515f385f1a2d30802a50ef851fcf92c2e6a
I hope this may still be useful somehow.
Version
Error output
Backtrace
The text was updated successfully, but these errors were encountered: