Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Metadata UUID Support #1211

Merged
merged 8 commits into from
Jun 19, 2024
Merged

[FEATURE] Metadata UUID Support #1211

merged 8 commits into from
Jun 19, 2024

Conversation

humcqc
Copy link
Contributor

@humcqc humcqc commented May 31, 2024

[FEATURE] Metadata UUID Support #1164

@langchain4j langchain4j added the P3 Medium priority label Jun 3, 2024
Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@humcqc thank you! Do tests pass for all EmbeddingStores that support storing metadata?

@humcqc
Copy link
Contributor Author

humcqc commented Jun 3, 2024

@humcqc thank you! Do tests pass for all EmbeddingStores that support storing metadata?

I've just tested the ones having the filter test : EmbeddingStoreWithFilteringIT
-> ElasticSearch, Milvus, InMemory, PG Vector ( JSON, JSONB, Columns)

@humcqc humcqc marked this pull request as ready for review June 4, 2024 15:58
@sjivan
Copy link

sjivan commented Jun 16, 2024

@langchain4j @humcqc Any chance we can get this approved and merged? With PgVectorEmbeddingStore I'd like to add a metadata column of type UUID which points to the parent table PK of the embedding table which has a type UUID.

@humcqc
Copy link
Contributor Author

humcqc commented Jun 16, 2024

@langchain4j @humcqc Any chance we can get this approved and merged? With PgVectorEmbeddingStore I'd like to add a metadata column of type UUID which points to the parent table PK of the embedding table which has a type UUID.

@langchain4j It should be ok to merge.

@sjivan by that time you can still use deprecated method :

    @Deprecated
    public Metadata add(String key, Object value) {
        return put(key, value.toString());
    }

@sjivan
Copy link

sjivan commented Jun 16, 2024

@langchain4j @humcqc Any chance we can get this approved and merged? With PgVectorEmbeddingStore I'd like to add a metadata column of type UUID which points to the parent table PK of the embedding table which has a type UUID.

@langchain4j It should be ok to merge.

@sjivan by that time you can still use deprecated method :

    @Deprecated
    public Metadata add(String key, Object value) {
        return put(key, value.toString());
    }

@humcqc thanks for getting the PR ready to merge. The workaround of the deprecated method does not work with PgVectorEmbeddingStore metadata because if the field in question is declared as "foo uuid", when the retreiver reads the column "foo" it introspects the type via JDBC and when it sees its of type UUID, it fails with an unssupported type exception. Hopefully this is no longer the case after change.

Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@humcqc thanks a lot!

Please update Weaviate ITs, "uuid" field needs to be added to metadataKeys, otherwise it fails

@humcqc
Copy link
Contributor Author

humcqc commented Jun 17, 2024

@langchain4j Done.

@humcqc
Copy link
Contributor Author

humcqc commented Jun 19, 2024

@langchain4j I've seen that some modules are in java 17
Is it possible to do the same thing for langchain4j-ollama ?

@langchain4j
Copy link
Owner

@humcqc yes, if there is a good reason for that

Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@humcqc good job, thank you!

@langchain4j langchain4j merged commit dc0effe into langchain4j:main Jun 19, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 Medium priority
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants