Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modified visibility store to support custom visibility store #6168

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

hazelkim052
Copy link
Contributor

What changed?

  • changed visibilityStore struct and some of its functions to public
  • added namespaceRegistry, searchAttributesProvider, searchAttributesMapperProvider arguments to custom visibility store factory interface

Why?

How did you test it?

Potential risks

Documentation

Is hotfix candidate?

common/persistence/visibility/factory.go Outdated Show resolved Hide resolved
Comment on lines 76 to 78
CreateIndex(ctx context.Context, index string, body map[string]any) (bool, error)
DeleteIndex(ctx context.Context, indexName string) (bool, error)
IndexPutTemplate(ctx context.Context, templateName string, bodyString string) (bool, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undo changes here. Actually, remove the ones you're adding to Client interface.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed them

Comment on lines 332 to 335
wfCloseTime, err := mutableState.GetWorkflowCloseTime(ctx)
if err != nil {
return err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What this function returns if the workflow is still running? Don't need to check it first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe then the closeTime is nil, and it panics when it tries to convert nil to time. I added a check to only get closeTime when workflow is not running. Also, in GetWorkflowCloseTime, I added error handling for when closeTime is nil, just in case when processDeleteExecution is called for workflow with the status "zombie" or "corrupted" and closeTime is nil.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed this code. We are getting workflow closetime from task as mutable state may be deleted when processing delete execution.

@@ -73,6 +75,7 @@ func NewManager(
metricsHandler metrics.Handler,
logger log.Logger,
) (manager.VisibilityManager, error) {
fmt.Println("saas-temporal: test: new manager called")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the prints.

Comment on lines +340 to +342
if body == nil {
body = make(map[string]interface{})
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really necessary? I think it's fine to call BodyJson(nil).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image I was getting this error, and I believe `BodyJson(nil).Do()` causes this issue.

triggered here at setupIndex for test cluster

@@ -115,7 +115,7 @@ type (
GetWorkflowExecution(ctx context.Context, request *persistence.GetWorkflowExecutionRequest) (*persistence.GetWorkflowExecutionResponse, error)
// DeleteWorkflowExecution add task to delete visibility, current workflow execution, and deletes workflow execution.
// If branchToken != nil, then delete history also, otherwise leave history.
DeleteWorkflowExecution(ctx context.Context, workflowKey definition.WorkflowKey, branchToken []byte, closeExecutionVisibilityTaskID int64, stage *tasks.DeleteWorkflowExecutionStage) error
DeleteWorkflowExecution(ctx context.Context, workflowKey definition.WorkflowKey, branchToken []byte, closeExecutionVisibilityTaskID int64, workflowCloseTime time.Time, stage *tasks.DeleteWorkflowExecutionStage) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think DeleteWorkflowExecution can be called when the workflow is still running. You need to add the close time parameter as *time.Time so it can be nil.
Or did you add like this because ms.GetExecutionInfo().GetCloseTime().AsTime() always return a time.Time object?
cc: @yiminc @yycptt

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea I added as time.Time because when we are creating visibility close task from proto, deleteVisibilityTask.CloseTime.AsTime() always return time.Time.

I also noticed that we are passing closeExecutionVisibilityTaskID as non-pointer object and we identify if it is "nil" by checking if that parameter equals to 0. So, I thought maybe we could do the similar thing for the closetime as well (i.e., identity if the closetime exists by checking if task.WorkflowCloseTime.After(time.Unix(0, 0).

service/history/visibility_queue_task_executor.go Outdated Show resolved Hide resolved
Comment on lines 1352 to 1354
if ms.executionInfo.CloseTime == nil {
return time.Time{}, ErrMissingCloseTimeInfo
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change. Let's not do this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah that is right. It will return error if I have this code block. I removed them

@hazelkim052 hazelkim052 marked this pull request as ready for review June 27, 2024 20:08
@hazelkim052 hazelkim052 requested a review from a team as a code owner June 27, 2024 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants