Skip to content

Commit 2b3a25c

Browse files
ochafikggerganov
andauthored
sampler: fixes trigger tokens + lazy grammars (fix typo cast from token to string) (#12291)
* Fix typo in lazy grammar handling (fixes trigger tokens) Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
1 parent 8352cdc commit 2b3a25c

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

examples/server/server.cpp

+3-2
Original file line numberDiff line numberDiff line change
@@ -384,8 +384,9 @@ struct server_task {
384384
SRV_DBG("Grammar trigger token: %d (`%s`)\n", token, word.c_str());
385385
common_grammar_trigger trigger;
386386
trigger.type = COMMON_GRAMMAR_TRIGGER_TYPE_TOKEN;
387-
trigger.value = (llama_token) token;
388-
params.sampling.grammar_triggers.push_back(trigger);
387+
trigger.value = word;
388+
trigger.token = token;
389+
params.sampling.grammar_triggers.push_back(std::move(trigger));
389390
} else {
390391
SRV_DBG("Grammar trigger word: `%s`\n", word.c_str());
391392
params.sampling.grammar_triggers.push_back({COMMON_GRAMMAR_TRIGGER_TYPE_WORD, word});

0 commit comments

Comments
 (0)