server: rename legacy --ctx-size to --kv-size option

[phymbert]

Context
--ctx-size is a legacy name before introduction of parallelism slots and creates confusion (see discussion #4130).

Proposed changes
Introduce --kv-size option and deprecate --ctx-size one.

@ggerganov Thanks for the amazing job you are doing here, hope this small contribution will help.

[pugzly]

Gatekeeping intensifies..
At this rate llama.cpp will become near-unreadable in another year or so for all the new people who weren't closely watching its development from the first line of code.
At least documentation of that flag should clearly state it's "former/renamed --ctx-size"

[phymbert]

Gatekeeping intensifies.. At this rate llama.cpp will become near-unreadable in another year or so for all the new people who weren't closely watching its development from the first line of code. At least documentation of that flag should clearly state it's "former/renamed --ctx-size"

@pugzly Agreed , server README.md updated with deprecation note.

[phymbert]

Taking into account @pugzly feedback, @ggerganov I am wondering if this change should also be aplied to the whole code base.

For example, n_ctx in llama_kv_cache_init might also be renamed to kv_size:
https://github.com/ggerganov/llama.cpp/blob/8f1be0d42f23016cb6819dbae01126699c4bd9bc/llama.cpp#L1941-L1948

[snajpa]

I'm new to llama.cpp but the behavior of this option was immediately obvious from the first-run output, that the "context size" gets divided by the number of slots and thus that it's not exactly a context size, but rather the space allocated for context. I'm just a N=1 datapoint, but I think the confusion could be corrected simply by updating the docs of the server example to deal with parallel slots and the need for raising the ctx_size to slots*ctx_size -- or the code could multiply it itself, while treating the ctx_size as if it's only for a single slot.

[ggerganov]

I am wondering if this change should also be aplied to the whole code base.

Yes, eventually n_ctx / ctx_size should be changed to kv_size in the entire codebase

[phymbert]

I am wondering if this change should also be aplied to the whole code base.

Yes, eventually n_ctx / ctx_size should be changed to kv_size in the entire codebase

OK reverted to draft PR, I will give it a try.

[phymbert]

@ggerganov I have tried, but I have the feeling that it's a bigbang change and I am not confident to be the one to bring it to master. Even if I spent some time on it, please feel free to simply close the PR, otherwise I will add necessary changes you request.

[ggerganov]

No worries, I've moved the changes to #5568 in order to run ggml-ci on it. Will think about if it is worth merging or not. Thanks

[pugzly]

I'm new to llama.cpp but the behavior of this option was immediately obvious from the first-run output, that the "context size" gets divided by the number of slots and thus that it's not exactly a context size, but rather the space allocated for context. I'm just a N=1 datapoint, but I think the confusion could be corrected simply by updating the docs of the server example to deal with parallel slots and the need for raising the ctx_size to slots*ctx_size -- or the code could multiply it itself, while treating the ctx_size as if it's only for a single slot.

That is great, but there's is 1 year worth of guides, tutorials, and applications built around and on top of llama.cpp, many of which may or may not be rendered obsolete, due to this, to the most part, "aesthetic" change.
I'm not saying it should not be done, but if done, ideally it should be well documented and with plenty of warning time.

[ggerganov]

Don't worry - when and if the change is applied, there will be deprecation notices. Plus it's actually a tiny API change (see llama.h), so it's very easy to adapt.