Memory not correctly freed after an SSL request on ESP32_S3

[tbibre]

Library/API/IoT binding

nanoFramework.System.Net, nanoFramework.System.Net.Http

Visual Studio version

VS2022 (17.13.57)

.NET nanoFramework extension version

2022.12.1.36

Target name(s)

ESP32_S3

Firmware version

Customized to add Ethernet, up to date with main NF_interpreter

Device capabilities

No response

Description

Hi,

As discussed on Discord, we have an issue with an HTTPS WebServer where at some point (usually, after ~32 requests), the SSL connection is no longer working and the only fix is a reboot. On the last request, the socket is successfully created, then SslNative.SecureServerInit() seems to works correctly and the certificate is sent, but an exception occurs in SslNative.SecureAccept() and the keys are not exchanged.

++++ Exception System.Net.Sockets.SocketException - CLR_E_FAIL (4) ++++
++++ Message: 
++++ System.Net.Security.SslNative::SecureAccept [IP: 0000] ++++
++++ System.Net.Security.SslStream::Authenticate [IP: 0032] ++++
++++ System.Net.Security.SslStream::AuthenticateAsServer [IP: 0016] ++++
++++ System.Net.HttpListener::AcceptThreadFunc [IP: 009a] ++++

Without the debuger, it seems we can exchange a few more requests (~2).

When looking at the memory, it seems each request leaks ~2300 bytes, the crash seems to be happening when the device is low on memory.

According to @Ellerbach who could reproduce the leak with a standard image on Wifi, the HTTP is also leaking a bit.

How to reproduce

1. Instanciate an HTTPListener with TLS 1.2 or 1.3
2. Display the current memory
3. Wait for an HTTPS request
4. Process the request and answer it
5. Go back to 2. The free memory decreased by ~2300 bytes

Expected behaviour

The memory should be freed once the socket is closed and the SSL context is cleared.

Screenshots

(Memory displayed before listening for a new request)

(Memory status around related parts)

Sample project or code

https://bitbucket.org/NovaLynx/nf-https/src/main/

Short NF program + test application

Aditional information

No response

[tbibre]

I did some tests in the HttpListener, closing things after each step and looping back before the HttpListenerContext is created:

• Instanciates a lot of X509Certificate2 : no change on memory
• Get socket and close it : no change on memory
• Get socket, open SSLStream, close stream, dispose stream, close socket : no change on memory
• Get socket, open SSLStream, dispose stream, close socket : no change on memory
• Get socket, open SSLStream, SecureServerInit, dispose stream, close socket : leaks ~1700 bytes
• Get socket, open SSLStream, SecureServerInit, SecureAccept, dispose stream, close socket : leaks ~2000 bytes (sometimes 2300 ?)

[tbibre]

So, it looks like the source is with ownCertificate here : https://github.com/nanoframework/nf-interpreter/blob/1ba3a866e4b6559eaebae3d3b73f7f6bb2305dcc/src/PAL/COM/sockets/ssl/MbedTLS/ssl_generic_init_internal.cpp#L263

I don't totally understand what is done with it, but it is the second certificate allocated, and this one is not freed.
I tried to free it but it breaks the SSL handshake. I guess some parts are still referenced somewhere.

I removed the allocation and replaced all uses with context->x509_crt : it works and the memory seems stable.
I probably broke some use case while doing this, so this needs someone who knows better that part of the codebase.