[Web] Latest version does nonstandard imports

[KTibow]

Describe the issue

The latest version of the web runtime does a lot of nonstandard importing of WASM that doesn't work with Vite.

If I don't configure it at all, it tries to fetch it from the shared directory of bundled JS. This does not work:

http://localhost:5173/@fs/home/kendell/Documents/Projects/locality/node_modules/.vite/deps/ort-wasm-simd-threaded.jsep.mjs net::ERR_ABORTED 404 (Not Found)

If I configure it to use my own static directory, it tries to import JS from it, which does not work:

Failed to load url /wasm/ort-wasm-simd-threaded.jsep.mjs (resolved id: /wasm/ort-wasm-simd-threaded.jsep.mjs). This file is in ../public and will be copied as-is during build without going through the plugin transforms, and therefore should not be imported from source code. It can only be referenced via HTML tags.

Instead, the runtime should simply import the files so that they can be bundled by all bundlers.

To reproduce

1. pnpm install [email protected]
2. import { InferenceSession } from "onnxruntime-web/webgpu";

const session = await InferenceSession.create(buffer, {
  executionProviders: ["webgpu"],
});

4. Notice errors log in console

Urgency

This is a regression; this worked when you didn't also have to host JS. This blocks me from using the latest version.

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.19.0-dev.20240604-3dd6fcc089

Execution Provider

'wasm'/'cpu' (WebAssembly CPU), 'webgpu' (WebGPU)

[fs-eire]

@KTibow thank you for reporting this issue. I am not very familiar with Vite. I want to understand more details about the problem.

The new version of ORT Web uses dynamic import to load file ort-wasm-simd-threaded.jsep.mjs. Do you mean that Vite does not support dynamic import, or it is not recommended to use dynamic import in Vite?

[KTibow]

The import function is supported in Vite to some degree.
If a static argument is passed, or a directory with a templated file, Vite can bundle the code. However, if the argument is dynamic, Vite won't know which code to bundle and won't update the function to work.
There are other ways that can help with dynamic file importing, but regardless, the directory of the file has to be known at build time in order for bundling to work.

* technicality: Vite isn't a bundler in itself, it uses Rollup, esbuild, and soon a rewrite of Rollup to bundle the code
* just writing this down has made me realize that this might not be a simple solution as I wrote in the original post, it'll probably require some consideration

[fs-eire]

The fundamental reason of why I uses dynamic import is becaues Emscripten does not support to generate both UMD/ESM glue for a Wasm target - you have to choose either. This means I have the following options:

1. build both ort-wasm-simd-threaded-umd.wasm+ort-wasm-simd-threaded-umd.js and ort-wasm-simd-threaded-esm.wasm+ort-wasm-simd-threaded-esm.mjs
2. only build one target, and use it in both ESM/UMD.

The option (1) not only doubled the package size for technically unnecessary reason, but also makes the deployment more complicated. A mismatch will not work and adds quite steps into troubleshooting.

I am currently using (2) to build ESM and import it in both UMD/ESM bundle.

[fs-eire]

If a static argument is passed, or a directory with a templated file, Vite can bundle the code. However, if the argument is dynamic, Vite won't know which code to bundle and won't update the function to work.

Based on the reason above, I expect the bundler to leave the dynamic import as-is, to keep the dynamic import behavior. rollup ( the underlying bundler) seems to support it correctly, but it seems Vite does not expect any .js/.mjs files to serve directly as assets.

[KTibow]

rollup seems to support it correctly

I'm a bit confused by how this would be possible. I have a suspicion, but I'd like to see the file structure of the bundled files so I can confirm/deny it.

[fs-eire]

https://github.com/microsoft/onnxruntime/tree/main/js/web/test/e2e

The E2E folder contains bundler test for webpack, rollup and percel

[fs-eire]

@KTibow you can alternatively import ORT Web in ESM. In latest main branch, dynamic import is disabled by default when you consume ORT Web as ESModule.

[KTibow]

Works in my tests 👍