server: exit failure if --embedding is set with an incoherent --ubatch-size

[phymbert]

Context

there is no advantage to increase n_batch above n_ubatch with embeddings models with pooling, because the entire batch must fit in a physical batch (ie. n_ubatch). n_batch is always >= n_ubatch.

• See @slaren comment in: server: docs: --threads and --threads, --ubatch-size, --log-disable #6254 (comment)

Proposition

Exit failure if --embedding is set and --ubatch-size != --batch-size in the server example. Probably also in the retrieval example in #6193.

Aldo probably KV bert.context_size must be taken into account.

[ngxson]

Exit failure if --embedding is set and --ubatch-size != --batch-size in the server example

Yeah I think it's ok to do so.

Aldo probably KV bert.context_size must be taken into account.

I think we should only warn the user if n_ctx_train > n_batch (just like in embedding.cpp).

If n_ctx_train < n_batch then we don't care, because users with low RAM may want to use a smaller n_batch. Only when n_tokens > n_batch then we throw an error.

[mscheong01]

For retrieval example, we assigned the n_batch value to n_ubatch.

Exit failure if --embedding is set and --ubatch-size != --batch-size in the server example.

It might be a little confusing to users bc the default value of n_batch and n_ubatch are different(users who don't specify --batch-size or --ubatch-size will 100% encounter this failure).

[phymbert]

That's the idea of this ticket, user must configure good values for the embedding model.

[mscheong01]

That sounds right 👍 , we just need to ensure that the error message properly guides the users to provide the same values for both --batch-size and --ubatch-size options.

[mscheong01]

Adding this info in the README under the --embeddings parameter would be nice too.