Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] ChatLanguageModel: allow setting parameters dynamically #1261

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

langchain4j
Copy link
Owner

Issue

#837

Change

Drafted flexible (Streaming)ChatLanguageModel API that allows specifying common model parameters (e.g. model name, temparature, etc) for each request.

General checklist

  • There are no breaking changes
  • I have added unit and integration tests for my change
  • I have manually run all the unit and integration tests in the module I have added/changed, and they are all green
  • I have manually run all the unit and integration tests in the core and main modules, and they are all green
  • I have added/updated the documentation
  • I have added an example in the examples repo (only for "big" features)

@fmatar
Copy link

fmatar commented Jun 24, 2024

Would this also support passing of apiKeys / baseUrl / organizationId?

@langchain4j
Copy link
Owner Author

@fmatar I will check if/how that can be supported

@fmatar
Copy link

fmatar commented Jun 24, 2024

@langchain4j one of the approaches I implemented was to use the raw rest client for that purpose.

Then I pass all my chatCompletion request along with the security keys to the request and execute.

One major blocking factor was the passing of the model objects by ai4j since they're not serializable and I ended up writing the required serialized for them.

Happy to assist in anyway since this is a major roadblock we'd like to resolve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants