Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement GPTQ for RWKV #88

Closed
3outeille opened this issue Apr 17, 2023 · 2 comments
Closed

Implement GPTQ for RWKV #88

3outeille opened this issue Apr 17, 2023 · 2 comments

Comments

@3outeille
Copy link

@BlinkDL Hi, I am willing to dedicate some time to implement GPTQ for RWKV, is that okay ?
@BlinkDL
Copy link
Owner

BlinkDL commented Apr 17, 2023 •
edited
Loading

This is exactly what we need :) Please work on ChatRWKV

And please take a look at https://github.com/hahnyuan/RPTQ4LLM

And you only need quantization for matrix*vec (ignore all time_xxx stuff - they have to be in fp32. tiny amt of computation).

@3outeille
Copy link
Author

Referencing BlinkDL/ChatRWKV#98 for any questions related to this topic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@BlinkDL @3outeille