WASI libraries keep leaking memory

[CryZe]

I tried this code:

#[no_mangle]
pub extern "C" fn update() {
    static FOO: std::sync::Once = std::sync::Once::new();
    FOO.call_once(|| {
        let _ = std::fs::metadata("/mnt/c/foo.txt");
    });
}

I expected to see this happen:

If you compile this to a WASI library and call the update function from a WASM runtime, then it will query the metadata of the file only during the first call to update. Every other invocation the update function should do nothing.

Instead, this happened:

The update function seems to refresh the list of preopened directories on every call... and it leaks them. The WASM memory keeps growing, looking like this:

This likely is because of the way Rust lazily initializes the list of preopened directories through wasi-libc. It probably does it unconditionally, which wasi-libc probably can't handle, so it never frees any memory.

The problem also occurs if you have any file system code at all, and it never even gets called:

#[no_mangle]
pub extern "C" fn other() {
    let _ = std::fs::metadata("/mnt/c/foo.txt");
}

#[no_mangle]
pub extern "C" fn update() { }

If you compile this code, but never call other and instead only update, it also keeps leaking the list of preopened directories.

The WASM seems to come down to this:

 (export "update" (func $update.command_export))
 ...
 (func $update.command_export
  (call $__wasm_call_ctors)
  (call $update)
  (call $__wasm_call_dtors)
 )

with

 (func $__wasm_call_ctors
  (call $__wasilibc_populate_preopens)
 )
 (func $__wasm_call_dtors
  (call $dummy)
  (call $dummy)
 )

__wasilibc_populate_preopens looks too complex. The original C code probably needs to be checked.

Meta

rustc --version --verbose:

rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-pc-windows-msvc
release: 1.70.0
LLVM version: 16.0.2

[workingjubilee]

cc @sunfishcode ?

[CryZe]

It seems like this has already been fixed in the upstream wasi-libc: WebAssembly/wasi-libc@a6f8713#diff-c7b46e06335bd882bf33d42159d0f1f9a7b20fa8b43e36a570420c16b02868a9R236

But Rust still uses an old version of wasi-libc:

rust/src/ci/docker/host-x86_64/dist-various-2/build-wasi-toolchain.sh

Line 13 in 76fb0e3

git reset --hard 4362b1885fd369e042a7c0ecd8df3b6cd47fb4e8