Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core Scheduler Logic Port to HSM framework #6140

Open
wants to merge 46 commits into
base: main
Choose a base branch
from

Conversation

tli-temporal
Copy link
Contributor

What changed?

Building on #6065, this PR introduces the bulk of scheduler logic to the HSM-based prototype. This makes the prototype able to correctly start workflows according to a schedule, but does not yet port the original long-polling logic. The plan is to replace long-polling with HSM-based callbacks in the subsequent PR. HSM also makes most of the side-effects or activities based logic in the original scheduler obsolete.

Why?

#6065

How did you test it?

Unit tests & Functional Tests

Potential risks

#6065

Documentation

None

Is hotfix candidate?

No

@tli-temporal tli-temporal requested a review from a team as a code owner June 14, 2024 19:00
@tli-temporal tli-temporal marked this pull request as draft June 14, 2024 19:00
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/fx.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
Copy link
Member

@bergundy bergundy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review the entire PR but there's some significant restructuring you should do before proceeding.
I'll also want @dnr to approve before we merge this.

service/history/workflow/mutable_state_impl.go Outdated Show resolved Hide resolved
proto/internal/temporal/server/api/enums/v1/common.proto Outdated Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/fx.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/scheduler.go Outdated Show resolved Hide resolved
components/scheduler/statemachine.go Show resolved Hide resolved
@tli-temporal tli-temporal marked this pull request as ready for review June 21, 2024 19:52
components/scheduler/config.go Outdated Show resolved Hide resolved
components/scheduler/config.go Outdated Show resolved Hide resolved
components/scheduler/statemachine.go Outdated Show resolved Hide resolved
components/scheduler/statemachine.go Show resolved Hide resolved
components/scheduler/statemachine.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
return err
}
// Copy outside scheduler, which has been updated, to the state machine
*sPtr = s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there's concurrent access to this machine? You'll lose that progress.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does env not check this automatically?

Either way, I added a simple check for whether the state has changed since last updated. We should probably chat about whether this is something we want to build into the framework instead of relying on users .

components/scheduler/config.go Outdated Show resolved Hide resolved
components/scheduler/config.go Outdated Show resolved Hide resolved
components/scheduler/config.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Outdated Show resolved Hide resolved
tag.NewAnyTag("overlap-policy", overlapPolicy), tag.NewBoolTag("manual", manual))
tweakables := e.config.Tweakables()

if s.cspec == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You assign the spec or error out so it should never reach this point without a valid spec.

components/scheduler/executors.go Outdated Show resolved Hide resolved
components/scheduler/executors.go Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants