-
Notifications
You must be signed in to change notification settings - Fork 38.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix httplog not logging watch duration #125626
base: master
Are you sure you want to change the base?
Conversation
Skipping CI for Draft Pull Request. |
cfb907a
to
1c96aee
Compare
1c96aee
to
15a8c5f
Compare
c20be53
to
5e6d4f5
Compare
/triage accepted |
cmd/kube-scheduler/app/server.go
Outdated
@@ -283,7 +283,7 @@ func buildHandlerChain(handler http.Handler, authn authenticator.Request, authz | |||
handler = genericapifilters.WithAuthentication(handler, authn, failedHandler, nil, nil) | |||
handler = genericapifilters.WithRequestInfo(handler, requestInfoResolver) | |||
handler = genericapifilters.WithCacheControl(handler) | |||
handler = genericfilters.WithHTTPLogging(handler) | |||
handler = genericfilters.WithHTTPLogging(handler, func(_ context.Context, f func()) { f() }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is very hard to think about what's happening.
Why can't we just do the following:
- change
withLogging
to do more-or-less exactly what'sListResource
is doing [check if TaskFrom is non-nil and if so append to it] - not change anything else...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
change withLogging to do more-or-less exactly what's ListResource is doing [check if TaskFrom is non-nil and if so append to it]
Actually my first attempt was like that. But it caused cycle import if we call TaskFrom in httplog
. Let me try if I can work this around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if it's an overkill to move routine filter in a separate directory to tackle the cycle import issue. But it should be what you suggested.
5e6d4f5
to
93731d2
Compare
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: linxiulei The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
|
||
w = responsewriter.WrapForHTTP1Or2(rl) | ||
handler.ServeHTTP(w, req) | ||
|
||
if routine.AppendTask(ctx, &routine.Task{Func: rl.Log}) { | ||
logInRoutine = true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you explain a bit what would happen when logInRoutine = true
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We added a feature to run the watch serving in a separate goroutine to free stack memory allocated by Golang runtime (#120902). This code determines if it is the case and run the rl.log()
at the end of the aforementioned goroutine.
I also added comment to explain that a bit. Thanks!
Signed-off-by: Eric Lin <[email protected]>
93731d2
to
525bdb1
Compare
/retest |
@@ -125,10 +126,21 @@ func withLogging(handler http.Handler, stackTracePred StacktracePred, shouldLogR | |||
rl := newLoggedWithStartTime(req, w, startTime) | |||
rl.StacktraceWhen(stackTracePred) | |||
req = req.WithContext(context.WithValue(ctx, respLoggerContextKey, rl)) | |||
defer rl.Log() | |||
logInRoutine := false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I would switch to sth like;
var logFunc func()
logFunc = rl.Log
defer func() {
if logFunc != nil {
logFunc()
}
}()
|
||
w = responsewriter.WrapForHTTP1Or2(rl) | ||
handler.ServeHTTP(w, req) | ||
|
||
// log after the task is executed in the separate routine | ||
// there is a task created in handler.ServeHTTP() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's put more context into this comment - I don't think it clarifies enough for people not familiar with it.
// We need to ensure that the request is logged after it is processed.
// In case the request is executed in a separate goroutine (created via
// WithRoutine handler in the handler chain), we want the logging to
// happen in that goroutine too, so we append it to the task.
|
||
// log after the task is executed in the separate routine | ||
// there is a task created in handler.ServeHTTP() | ||
if routine.AppendTask(ctx, &routine.Task{Func: rl.Log}) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm - when calling AppendTask, the main function is not yet set, right?
So won't it be overwritten later anyway?
I would like to see test output to see we actually fixed the problem.
@@ -35,6 +35,20 @@ func WithTask(parent context.Context, t *Task) context.Context { | |||
return request.WithValue(parent, taskKey, t) | |||
} | |||
|
|||
// AppendTask appends a task executed after completion of existing task. | |||
// It is a no-op if there is no existing task. | |||
func AppendTask(ctx context.Context, t *Task) bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This deserves some unit test.
What type of PR is this?
/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #125614
Special notes for your reviewer:
Does this PR introduce a user-facing change?