Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Implement zstd based compressor for type metadata #1664

Open
chaokunyang opened this issue May 30, 2024 · 2 comments
Open

[Java] Implement zstd based compressor for type metadata #1664

chaokunyang opened this issue May 30, 2024 · 2 comments
Labels
enhancement New feature or request good first issue Good for newcomers java

Comments

@chaokunyang
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

In #1663, we add a MetaCompressor interface and deflater based implementation.

Zstd has better compression rate and can learn from train corpu, we should support using zstd as meta compressor

Describe the solution you'd like

Add a new java maven module, impement zstd based meta compressor.

Additional context

#1663

@chaokunyang chaokunyang added enhancement New feature or request java good first issue Good for newcomers labels May 30, 2024
@wangjie-fourth
Copy link
Contributor

@chaokunyang
Hi, I would like to try this. But i have question first, do i need create new maven module for this? It looks like just add new Class in fury-core?

@chaokunyang
Copy link
Collaborator Author

If we add a new class directly into fury-core, we must aovid introducing a new jar dependency. MethodHandle or reflection is feasible. If we don't use reflection, we may need to add a new maven module

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers java
Projects
None yet
Development

No branches or pull requests

2 participants