Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximize overall coverage by showing overlapping benchmarks #356

Open
DonggeLiu opened this issue Jun 19, 2024 · 3 comments
Open

Maximize overall coverage by showing overlapping benchmarks #356

DonggeLiu opened this issue Jun 19, 2024 · 3 comments

Comments

@DonggeLiu
Copy link
Collaborator

Extended from #354.
Context:
Currently, OSS-Fuzz-Gen selects functions with low coverage. However, some selected functions may execute the same code blocks of the project, leading to minimal project-level coverage improvement.

Goal:
Displaying the "per-project coverage increment" alongside the "per-benchmark" coverage increment will help us identify instances where overlapping code blocks are being tested. This insight will highlight cases where we should re-select benchmarks to enhance overall efficiency.

@DavidKorczynski
Copy link
Collaborator

Related: Fuzz Introspector has an analysis feature for "Optimal targets" which does a recursive analysis on which functions to target. Basically, pick the first best function to target, and then FI analyses the impact of this before proceeding to analyse which is the next best target to analyse. It's been around for some time and is in the UI e.g.: https://storage.googleapis.com/oss-fuzz-introspector/htslib/inspector-report/20240617/fuzz_report.html#Optimal-target-analysis

I'll expose this as an oracle API so we can use it for benchmark generation.

@DonggeLiu
Copy link
Collaborator Author

ACK, I will close this then.

@DavidKorczynski
Copy link
Collaborator

I still think this is a good thing to pursue though! Introspector's "Optimal targets" is just one heuristic trying to solve a similar problem, and the feature suggested in this issue is also relevant now that we can extract target functions from a diverse set of oracles.

@DonggeLiu DonggeLiu reopened this Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants