Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor darwin build job #186

Open
straight-shoota opened this issue Apr 7, 2022 · 6 comments
Open

Refactor darwin build job #186

straight-shoota opened this issue Apr 7, 2022 · 6 comments

Comments

@straight-shoota
Copy link
Member

darwin release builds take a long time to finish. The dist_darwin job on circle ci usually takes ~50 minutes for releaes builds. The comparable dist_linux job finishes in about 15 minutes (example: 1.4.0 release workflow).

We're using omnibus for dist_darwin. It appears to be building all libraries from scratch. The first 10+ minutes are entirely spend just downloading stuff.
This seems to be very inefficient. omnibus itself draws in some dependencies and then the entire process is very slow.

We actually just need to install a couple of libraries and then build the compiler. That's a pretty straighforward task and IMO shouldn't require omnibus. I think we could achieve this with a couple of make recipes for building the dependencies (and ideally cache them to avoid rebuilding every time).
Maybe an even better alternative would be to build in the nix environment which we already use for the test jobs on GitHub actions.

/cc @bcardiff @matiasgarciaisaia @beta-ziliani WDYT?

@beta-ziliani
Copy link
Member

IIUC omnibus is used for (potential?) darwin specific stuff. But I guess we can switch to nix and switch back when we see a real need for it.

@matiasgarciaisaia
Copy link
Member

Replacing omnibus with another thing may work just fine - the only caveat would be to pay close attention to compatibility issues with different macOS versions (I guess we don't support OpenDarwin or distros like that).

@straight-shoota
Copy link
Member Author

FTR: The latest build times on macOS are more like 36 minutes, so that's an improvement of the initial reporting. I have no idea why, though.
I guess that isn't too bad actually, considering the multiarch process means the compiler is built twice.

Still, our omnibus setup has shown to be outdated and hard to work with. I'd appreciate migrating to a better solution.

@straight-shoota
Copy link
Member Author

straight-shoota commented Mar 17, 2023

Maybe a "better solution" could also mean updating omnibus to a more recent version and partially improving the build configuration.

One thing I noticed is that omnibus actually provides a caching mechanism but I don't think we're using it. At least not in CI. It might just work locally if you keep the cache directory. @bcardiff Do you remember why we're not synchronizing the build cache across CI runs and instead jerry-rig a cache with the llvm_bin artifact on S3?
https://github.com/chef/omnibus/blob/main/docs/Build%20Cache.md

@bcardiff
Copy link
Member

I don't recall anything regarding omnibus caching. So it's worth a try.

I think there was some reason why we didn't update to omnibus, but I don't remember why. I'll if I find something, ultimately it was a long time ago and hopefully things changed in the right direction.

@straight-shoota
Copy link
Member Author

straight-shoota commented Mar 20, 2023

This might also be relevant: #134

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants