Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add dimension() in EmbeddingModel #1250

Merged
merged 28 commits into from
Jun 21, 2024

Conversation

Martin7-1
Copy link
Contributor

@Martin7-1 Martin7-1 commented Jun 8, 2024

Issue

#1080

Change

Add dimension() method in EmbeddingModel. Using an AbstractEmbeddingModel to implement EmbeddingModel and a common dimension() method like below:

    @Override
    public int dimension() {
        if (dimension != null) {
            return dimension;
        }

        return dimensionMap().compute(modelName(), (key, value) -> {
            this.dimension = Optional.ofNullable(value).orElse(embed("test").content().dimension());
            return dimension;
        });
    }

Now, existing {NameOfIntegration}EmbeddingModel just need to override dimensionMap() method to contains known embedding model dimension and modelName() method to get embedding model name.

General checklist

  • There are no breaking changes
  • I have added unit and integration tests for my change
  • I have manually run all the unit and integration tests in the module I have added/changed, and they are all green
  • I have manually run all the unit and integration tests in the core and main modules, and they are all green

Checklist for adding new model integration

  • I have added my new module in the BOM

Checklist for adding new embedding store integration

  • I have added a {NameOfIntegration}EmbeddingStoreIT that extends from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT
  • I have added my new module in the BOM

Checklist for changing existing embedding store integration

  • I have manually verified that the {NameOfIntegration}EmbeddingStore works correctly with the data persisted using the latest released version of LangChain4j

@Martin7-1 Martin7-1 changed the title [Feature] Add dimension() in EmbeddingModel [Draft] Add dimension() in EmbeddingModel Jun 8, 2024
@langchain4j langchain4j added the P2 High priority label Jun 10, 2024
Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Martin7-1 thank you!

@Martin7-1 Martin7-1 marked this pull request as ready for review June 20, 2024 13:07
@Martin7-1 Martin7-1 changed the title [Draft] Add dimension() in EmbeddingModel [Feature] Add dimension() in EmbeddingModel Jun 20, 2024
Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Martin7-1 I like this new solution much more, thank you!

Just a few minor things and we can merge!

@Martin7-1
Copy link
Contributor Author

The CI failed maybe because the main branch's commit. @langchain4j

Copy link
Owner

@langchain4j langchain4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Martin7-1 good job, thank you!

@langchain4j langchain4j merged commit 982577b into langchain4j:main Jun 21, 2024
6 checks passed
@langchain4j
Copy link
Owner

@Martin7-1 could you please also update embedding models in https://github.com/langchain4j/langchain4j-embeddings ?

@Martin7-1
Copy link
Contributor Author

@Martin7-1 could you please also update embedding models in https://github.com/langchain4j/langchain4j-embeddings ?

Sure!

langchain4j pushed a commit to langchain4j/langchain4j-embeddings that referenced this pull request Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants