Fixes #874 #1292
Open
Fixes #874 #1292
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I stumbled across #874 and dug in.
The issue seems to be caused by the fact readMessageV2 expects a message to be present on the wire, but a compacted topic may return a "RecordBatch" with no records (when an offset gets compacted out).
The proposed fix loops reading additional headers when both RecordBatch is used ( i.e. magic number 2 ) and when the returned RecordBatch has zero records.
If the maintainers request any additional work, I'm happy to let someone else complete the work as I don't know when I'd be able to come back to this.