Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

does ipex.llm support speculative sampling? #664

Open
zhangnju opened this issue Jun 24, 2024 · 1 comment
Open

does ipex.llm support speculative sampling? #664

zhangnju opened this issue Jun 24, 2024 · 1 comment
Assignees
@ZailiWang
Labels
LLM Query

Comments

@zhangnju
Copy link

Describe the issue

Hi IPEX guys
does ipex.llm support speculative sampling? if support, where are the codes? and how do we test it?
Thanks
@ZailiWang ZailiWang self-assigned this Jun 25, 2024
@ZailiWang
Copy link
Contributor

Hi, for IPEX CPU the support for speculative sampling is working in progress, please keep tuned.
For IPEX XPU, this support has not been taken into the scope for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ZailiWang ZailiWang

Labels
LLM Query
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zhangnju @ZailiWang