Add hook points support in conformance test suite

[shawnh2]

What would you like to be added:

Add the hook point support for current conformance test suite.

For example, like AfterTestFail hook point, developers can attach various commands to this hook point, and programs that attach to this hook point wil only be executed if one conformance test is failed.

Why this is needed:

This will be super helpful for developers to debug the failed conformance test.

Sometimes the error trace reported by conformance test is not very useful. For example, we'd like to see the status/logs of specific resources (by attaching kubectl cmd to the hook which I described above could ease the pain), even some other configs that can be retrieved by other commands.

[k8s-ci-robot]

@mlavacca: The label(s) /label needs-triage cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label needs-triage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

@mlavacca: The label(s) triage/needs cannot be applied, because the repository doesn't have them.

In response to this:

/triage needs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mlavacca]

/triage needs-triage

[k8s-ci-robot]

@mlavacca: The label(s) triage/needs-triage cannot be applied, because the repository doesn't have them.

In response to this:

/triage needs-triage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[kflynn]

This sounds like a good idea, yeah. Making it easier for developers to get value from the conformance tests, whether or not they're "part of the Gateway API", sounds like a Good Thing™. 🙂

[robscott]

Discussed in today's community meeting, this sounds like a great idea, thanks @shawnh2!

/triage accepted

[shawnh2]

Assign myself, since I've been working on something that is quite similar to this issue as well. I will post the initial design of this here once it's done.

/assign