Skip to content

Commit af4c6f2

Browse files
committed
[Bugfix] fix referencing uninitialized tokenizer when using multiple step scheduling with skip tokenizer init option
Signed-off-by: changjun.lee <pord7457@gmail.com>
1 parent 98d01d3 commit af4c6f2

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/engine/output_processor/multi_step.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ def _process_seq_outputs(self, seq: Sequence,
178178
# generates a fixed number of tokens without evaluating stopping
179179
# conditions within the block. This can cause an eos token to be
180180
# unintentionally ignored.
181-
if not sampling_params.ignore_eos:
181+
if not sampling_params.ignore_eos and self.detokenizer:
182182
eos_token_id = self.get_tokenizer_for_seq(seq).eos_token_id
183183
# Avoiding .index calls as exception throwing in the happy path
184184
# is expensive.

0 commit comments

Comments
 (0)