-
Notifications
You must be signed in to change notification settings - Fork 237
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Skipped event with concurrency enabled #556
Comments
Are you able to reproduce this easily? Could you share an example or test case? You are sure the effect of running TestHandler wasn't run? Maybe it was run by one of the other 4 instances of your handler? |
Here is a an example project that replicates what you are seeing: https://github.com/drteeth/commanded_issue_556 Run the tests to see the failure. My question is how should this work? Should the remaining handlers stop? Should the instance that failed start from 5 when it starts back up? |
I'm thinking the behavior for the partition of the failed handler should be the same as for a single thread -e.g. it does not process any other events. Other handlers can continue processing the events corresponding to their partitions. When fixing the code issue and restarting the failed partition handler, it should resume from where it left off. |
Can you confirm for me that the failure in the handler is permanent? Meaning this is no a temporary failure, no amount of retries is going to fix it? Am I assuming correctly here?
What would this mean though? If I had 2 concurrent handlers, the first processing odd events, the second even ones, when the first one encounters an error and ultimately dies, should the second one only process even events still? Should it take over from the failed on? Presumably not as it would also die. |
My example uses the in-memory store. I think it should use the PG adapter to be worth anything. I'll try to update it to see if that changes things. |
Updated my example to use PG adapter and to expect the subscription to not pass the failed event. |
@slashdotdash here's a failing test for the issue: drteeth@e43c899 |
Yes, I confirm the failure is permanent, meaning it would need a code change to process the event successfully.
It is probably a nice to have to continue processing the partitions where it can. Sorry for the late reply, you're moving too fast. |
I have the following setup:
Use Postgresql as an event store.
Have 150 events in the events table.
Created a TestHandler with concurrency = 5
Make TestHandler crash at event 40
The TestHandler is registered in the superviser with restart: :temporary.
Observed behavior:
What is the course of action to avoid skipping events? What happens if i delpoy while the handlers are processing events. It may also kill pids and events are loaded based on the last_seen from the event store subscriptions table.
The text was updated successfully, but these errors were encountered: