[Model] Add GraniteMoeHybrid 4.0 model #17461
Conversation
|
This pull request has merge conflicts that must be resolved before it can be |
| def sample( | ||
| self, | ||
| logits: Optional[torch.Tensor], | ||
| sampling_metadata: SamplingMetadata, | ||
| ) -> Optional[SamplerOutput]: | ||
| next_tokens = self.sampler(logits, sampling_metadata) | ||
| return next_tokens |
There was a problem hiding this comment.
Sampler isn't defined inside model anymore (see #17084 )
| @@ -0,0 +1,39 @@ | |||
| # SPDX-License-Identifier: Apache-2.0 | |||
There was a problem hiding this comment.
Heads up that we will move the model tests in #17459
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
s3woz
force-pushed
the
granitemoehybrid_clean
branch
2 times, most recently
from
a365669 to
d433d2d
Compare
mergify
bot
added
documentation
s3woz
force-pushed
the
granitemoehybrid_clean
branch
from
d433d2d to
e1ba9e8
Compare
s3woz
force-pushed
the
granitemoehybrid_clean
branch
from
c45809d to
e1ba9e8
Compare
s3woz
force-pushed
the
granitemoehybrid_clean
branch
from
c45809d to
d803786
Compare
The PR adds support for upcoming Granite4.0 models. It is a companion PR to huggingface/transformers#37658 for adding the same model to HF.
Please note that model paths are local for now, but I will provide HF URLs asap.
@tdoublep