Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended llm support (e.g. Llama 3, M8x22b) and synthetic test generation #936

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ciekawy
Copy link

@ciekawy ciekawy commented May 6, 2024

Enhancements for new opensource models with focus on Llama 3 70b instruct

This is a draft PR - it introduces several improvements to RAGAs to enhance its compatibility with Llama 3 70b and improve the quality of generated outputs, especially for synthetic data generation.

Prompt Adjustments

  • Modified prompts in prompt.py and prompts.py to increase the quality of generated outputs, which is particularly important for synthetic data generation.
  • The updated prompts aim to provide clearer instructions and more relevant examples to guide the language model in generating higher-quality outputs.

Support for Non-Typical LangChain LLM Configurations

  • Added support for LLM configurations that require additional parameters and dynamic calculations when invoking LLM generation.
  • Specifically, the Together.ai Llama 3 70b instruct model requires extra parameters, including dynamically calculated values in a callback.
  • Introduced a new LLMConfig class to encapsulate the custom configuration options.
  • Implemented a together_prompt_callback function to handle the dynamic prompt generation for the Together.ai Llama 3 70b instruct model.

Example Configuration

def together_prompt_callback(
    prompt: PromptValue
) -> t.Tuple[t.List[PromptValue], t.Dict[str, t.Any]]:
    empty_prompt = StringPromptValue(text='')
    prompt_str = prompt.prompt_str
    messages = [{"content": prompt_str, "role": "user"}]
    return [empty_prompt], {"messages": messages}

together_llama3instruct_config = LLMConfig(
    stop=["<|eot_id|>"],
    prompt_callback=together_prompt_callback,
    type="chat",
    prompt_format_string="<human>: {prompt}\n<bot>:",
    request_type="language-model-inference",
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant