You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In production environment, especially desktop apps, it's common to have a button (or any other way) to allow users to abort the model loading. Fortunately, llama.cpp has already added support for it ggml-org/llama.cpp#4462. I think we should introduce this feature in LLamaSharp.
Similarly, async model loading is also important for applications based on LLamaSharp, which avoids blocking the main thread for a long time when loading a large model. I've found a similar work of it in the node.js binding of llama.cpp withcatai/node-llama-cpp#178. We could also implement it by polling the progress callback.
The text was updated successfully, but these errors were encountered:
In production environment, especially desktop apps, it's common to have a button (or any other way) to allow users to abort the model loading. Fortunately, llama.cpp has already added support for it ggml-org/llama.cpp#4462. I think we should introduce this feature in LLamaSharp.
Similarly, async model loading is also important for applications based on LLamaSharp, which avoids blocking the main thread for a long time when loading a large model. I've found a similar work of it in the node.js binding of llama.cpp withcatai/node-llama-cpp#178. We could also implement it by polling the progress callback.
The text was updated successfully, but these errors were encountered: