Several optimizations #737
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I've been profiling orama, since it's taking 4s to ingest around 4k documents on my project and I would like to lower this down. I've noticed several things: It's ingesting a large amount of empty strings which is useless CPU time and also it's recalculating ids redundantly. This commit tries to address this two issues. Additionally, I've noticed that due to the async APIs, the code is spending most of it's time waiting for "run microtasks". I have no idea if it would be possible to compile those away, because right now it makes the default implementation much worse (performance-wise) in order for people to be able to provide their storage solution. Lastly, I've also noticed that providing the ID of the document, makes the ID be stored as part of the document properties. I thought it would only be to replace the default orama ID. I will fix this myself in userland by using `getDocumentProperties`, but perhaps is good to either change this default or document it.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Hi there! I fear you'll need to regenerate the test snapshots. The PR looks good then! Thank you so much |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've been profiling orama, since it's taking 4s to ingest around 4k documents on my project and I would like to lower this down. I've noticed several things: It's ingesting a large amount of empty strings which is useless CPU time and also it's recalculating ids redundantly. This commit tries to address this two issues.
Additionally, I've noticed that due to the async APIs, the code is spending most of it's time waiting for "run microtasks". I have no idea if it would be possible to compile those away, because right now it makes the default implementation much worse (performance-wise) in order for people to be able to provide their storage solution.
Lastly, I've also noticed that providing the ID of the document, makes the ID be stored as part of the document properties. I thought it would only be to replace the default orama ID. I will fix this myself in userland by using
getDocumentProperties, but perhaps is good to either change this default or document it.