Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] DFT/STFT WebGPU op support (web/js) #20997

Open
xenova opened this issue Jun 11, 2024 · 2 comments
Open

[Feature Request] DFT/STFT WebGPU op support (web/js) #20997

xenova opened this issue Jun 11, 2024 · 2 comments
Labels
ep:WebGPU ort-web webgpu provider feature request request for unsupported feature or enhancement platform:web issues related to ONNX Runtime web; typically submitted using template

Comments

@xenova
Copy link

xenova commented Jun 11, 2024

Describe the feature request

To add WebGPU support for DFT and STFT ops:

Currently, neither are supported yet (as stated here).

Describe scenario use case

This will be used to optimize preprocessing performance for audio models like Whisper, specifically to be used in Transformers.js.
@xenova xenova added the feature request request for unsupported feature or enhancement label Jun 11, 2024
@github-actions github-actions bot added the platform:web issues related to ONNX Runtime web; typically submitted using template label Jun 11, 2024
@xenova xenova changed the title [Feature Request] DFT/STFT WebGPU op support [Feature Request] DFT/STFT WebGPU op support (web/js) Jun 11, 2024
@xenova
Copy link
Author

xenova commented Jun 11, 2024

cc @guschmue for this. Another thing I've noticed is that the WASM version is much slower than a pure JS implementation (which I currently use). So, might be something to look into too.

@guschmue guschmue added the ep:WebGPU ort-web webgpu provider label Jun 11, 2024
@guschmue
Copy link
Contributor

makes sense. Kind surprising that wasm is slower then javascript, maybe it's an algo choice. We'll look at it.

xenova added a commit to xenova/transformers.js that referenced this issue Jun 14, 2024
@xenova
Add rfft/stft tensor ops via ORT
490a4b8 
Will be much more useful when WebGPU support is ready for these ops (see microsoft/onnxruntime#20997)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ep:WebGPU ort-web webgpu provider feature request request for unsupported feature or enhancement platform:web issues related to ONNX Runtime web; typically submitted using template
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@guschmue @xenova