Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): include parallel all e2e tests on PR check #7458

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

odockal
Copy link
Contributor

@odockal odockal commented Jun 4, 2024

What does this PR do?

Runs all e2e tests on PR check. This workflow should be non-blocking.

Screenshot / video of UI

What issues does this PR fix or reference?

#6494

How to test this PR?

On your own fork only.

  • Tests are covering the bug fix or the new feature

@odockal odockal requested review from benoitf and a team as code owners June 4, 2024 13:50
@odockal odockal requested review from cdrage and feloy and removed request for a team June 4, 2024 13:50
@odockal
Copy link
Contributor Author

odockal commented Jun 4, 2024

I am thinking about adding needs keyword into e2e-tests to depend on smoke job since it would run only if smoke has passed.

@odockal odockal requested a review from a team June 6, 2024 07:21
@cbr7
Copy link
Contributor

cbr7 commented Jun 6, 2024

I'm not seeing the step being run in the pr-check on here.

@odockal
Copy link
Contributor Author

odockal commented Jun 6, 2024

@benoitf @cbr7 after discussion I am reverting the need for smoke-e2e for e2e tests. This way it will be faster as it can run in parallel and it would also show us possible flakiness of the tests if one passes and other not.

@benoitf
Copy link
Collaborator

benoitf commented Jun 12, 2024

how long does it take ?
also what's the value in running the smoke tests if we run all the tests ?

@benoitf
Copy link
Collaborator

benoitf commented Jun 12, 2024

I see 5mn for smoke and 7mn for all the tests so should we drop smoke tests ?

@odockal
Copy link
Contributor Author

odockal commented Jun 12, 2024

@benoitf Initial idea was to know if there is something broken besides the core functionality.

In this particular case, e2e tests differ in a way that they also test extensions installation (dev sandbox and openshift local) and some basic registry handling (where we do not have secrets available so no real adding of the registries).

Now, what is core functionality that needs to be always passing was denoted as smoke tests. The rest is like all-tests. If we want to distinguish or not, is good question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants