Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed baseline compare issues #5605

Merged

Conversation

mildm8nnered
Copy link
Collaborator

@mildm8nnered mildm8nnered commented May 25, 2024

Fixes #5606 where baseline compare can produce some false alarms, because we throw away the text information from one of the baselines, but the files on disk may be absent or have changed, so the new text values read during filtering may (likely) not be appropriate.

In this PR, the logic and public API for filter is completely unchanged, but the public function converts the StyleViolations to BaselineViolations, and retrieves any existing BaselineViolations for the file.

These BaselineViolations are then passed to a private filter function, which implements the core filtering logic.

The compare function now passes the BaselineViolations from each Baseline to this private function.

We now also sort the violations reported by baseline compare, by location and rule, so the output should now be wholly deterministic which is was not quite before.

@mildm8nnered mildm8nnered marked this pull request as ready for review May 26, 2024 14:26
@mildm8nnered
Copy link
Collaborator Author

I think CI is failing here for unrelated reasons ...

Copy link
Collaborator

@SimplyDanny SimplyDanny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have another test that checks if outputs are sorted?

Source/SwiftLintCore/Models/Baseline.swift Outdated Show resolved Hide resolved
@mildm8nnered mildm8nnered force-pushed the mildm8nnered-fix-baseline-compare branch from dd70917 to c90993c Compare June 24, 2024 08:43
@SwiftLintBot
Copy link

SwiftLintBot commented Jun 24, 2024

17 Messages
📖 Linting Aerial with this PR took 0.68s vs 0.65s on main (4% slower)
📖 Linting Alamofire with this PR took 0.89s vs 0.88s on main (1% slower)
📖 Linting Brave with this PR took 5.26s vs 5.51s on main (4% faster)
📖 Linting DuckDuckGo with this PR took 2.8s vs 2.94s on main (4% faster)
📖 Linting Firefox with this PR took 8.09s vs 8.0s on main (1% slower)
📖 Linting Kickstarter with this PR took 7.22s vs 7.05s on main (2% slower)
📖 Linting Moya with this PR took 0.4s vs 0.41s on main (2% faster)
📖 Linting NetNewsWire with this PR took 1.84s vs 1.84s on main (0% slower)
📖 Linting Nimble with this PR took 0.56s vs 0.57s on main (1% faster)
📖 Linting PocketCasts with this PR took 6.06s vs 6.26s on main (3% faster)
📖 Linting Quick with this PR took 0.33s vs 0.34s on main (2% faster)
📖 Linting Realm with this PR took 3.42s vs 3.53s on main (3% faster)
📖 Linting Sourcery with this PR took 1.7s vs 1.73s on main (1% faster)
📖 Linting Swift with this PR took 3.16s vs 3.11s on main (1% slower)
📖 Linting VLC with this PR took 0.93s vs 0.89s on main (4% slower)
📖 Linting Wire with this PR took 11.85s vs 12.52s on main (5% faster)
📖 Linting WordPress with this PR took 8.89s vs 9.03s on main (1% faster)

Generated by 🚫 Danger

@mildm8nnered
Copy link
Collaborator Author

Should we have another test that checks if outputs are sorted?

I've updated the test so that there are multiple violations in the output of compare, and I'm passing the initial violations in in reverse order, to verify the sorting of the output.

@mildm8nnered mildm8nnered force-pushed the mildm8nnered-fix-baseline-compare branch from c90993c to ae39cec Compare June 26, 2024 21:59
@mildm8nnered mildm8nnered force-pushed the mildm8nnered-fix-baseline-compare branch from ae39cec to bf56623 Compare June 27, 2024 21:49
@mildm8nnered mildm8nnered merged commit 5c195a4 into realm:main Jun 28, 2024
12 checks passed
@mildm8nnered mildm8nnered deleted the mildm8nnered-fix-baseline-compare branch June 28, 2024 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

baseline compare produces some false alarms
3 participants