支持embeddings使用类似openai api的server #3892
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
the
size:M
sydnash
force-pushed
the
master
branch
2 times, most recently
from
6e8b750
to
4bf683a
Compare
|
请问如何将本地embedding模型封装成openai api格式,以便其他机器上的langchain chatchat调用。老哥有没有推荐的方法或代码库 |
|
速度真快，刚好需要用到，赞 |
|
请问如果我用fastchat启动bge-large-zh作embedding的话，可以用这个api的访问方式吗？langchain本地启动了一个bge-large-zh用来做tokenizer会影响embedding的效果吗？我实际测试langcain的embedding跟fastchat出来的embedding数值不一样 |
|
0.3.x 版本里使用 LocalAIEmbeddings 访问本地部署的模型，请确认这个实现是否重复了？ |
只要是兼容openai的embedding api就可以的。 tokenizer和embedding是两步，应该没有影响。 |
看了下LocalAiEmbeddings也是使用openai client访问openai api兼容服务，应该是一样的功能。
vllm里面有opneai 兼容api封装的例子。 |
|
你好，在2.0的贡献我们没有计划维护，目前主分支已经是3.0的代码，感谢你对本贡献，期待你的下次参与 3.0使用了第三方平台加载本地模型，对于本地模型的支持，请参考readme |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
目前只有embedding模型为text-embedding-ada-002的时候才会使用OpenAIEmbeddings
现在通过增加EMBEDDING_MODEL_USE_OPENAI和EMBEDDING_MODEL_OPENAI配置，可以使任意模型都可以通过OpenAIEmbeddings来调用自定义的类openai api的server来实现embedding。