Improve JSON format prompt for large chunks & Handle ZeroDivisionError #982
+15
−14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces error handling for ZeroDivisionError in the filter method of the NodeFilter class. This change ensures that the application gracefully handles cases where division by zero occurs, setting the score to 0 by default.
Modify the JSON_FORMAT_INSTRUCTIONS in output_parser.py to ensure better JSON output handling by LLMs, particularly for larger chunk sizes. This change helps maintain the structure of the output without newlines, which optimizes parsing by PydanticOutputParser and reduces failures due to formatting issues in long prompts.
dosubot
bot
added
the
size:XS
|
@shahules786 could you check if we can merge this in? |
mspronesti
suggested changes
May 24, 2024
dosubot
bot
added
size:S
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In this PR there are three different changes
Changes
AssestoAssessIt seems like the LLM sometimes loses track of the imposed instruction for the output, especially for long prompts. So although the output generated for a chunk_size of 512 is perfect, the output for a chunk_size of 1024 has an extra newline. Here parsing fails when using PydanticOutputParser even though the llm has generated an output and there is only an extra
'\n'as shown in the instances below.But by tuning the prompt and adding
Please output your response in the demanded json format.at the end of the instruction we get output without'\n'. This output can then be parsed and the context can be considered.