Skip to content

Commit 86a1f67

Browse files
authored
[Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285)
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com>
1 parent a257d9b commit 86a1f67

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

benchmarks/backend_request_func.py

+1
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,7 @@ async def async_request_deepspeed_mii(
201201
timeout=AIOHTTP_TIMEOUT) as session:
202202

203203
payload = {
204+
"model": request_func_input.model,
204205
"prompt": request_func_input.prompt,
205206
"max_tokens": request_func_input.output_len,
206207
"temperature": 0.01, # deepspeed-mii does not accept 0.0 temp.

0 commit comments

Comments
 (0)