cluster: move archival into cluster library

[dotnwat]

This PR relocates archival into the cluster library to reflect the reality that it was built as part of v::cluster. While build systems like CMake are fine with this chaos, others are more strict.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v24.1.x
• v23.3.x
• v23.2.x

Release Notes

• none

[vbotbuildovich]

new failures in https://buildkite.com/redpanda/redpanda/builds/50542#01903e51-ee46-4d6b-b222-a24791f0b076:

"rptest.tests.crl_test.CertificateRevocationTest.test_pp_api"

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/50552#01903edb-9d5d-4b09-a7b2-91a07e0ea58b

[dotnwat]

Force pushed to fix merge conflict.

[Lazin]

I have a lot of code for the archival subsystem in the feature branch/PRs. This will get in the way in a big way. I'm also don't think archival should be a part of the cluster. We should probably fix dependencies and start building it separately and not as part of the cluster.

…

On Sat, Jun 22, 2024, 21:08 Noah Watkins ***@***.***> wrote: Force pushed to fix merge conflict. — Reply to this email directly, view it on GitHub <#19955 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWMNTNXMUNCXHR5OKUZBLZIXDUPAVCNFSM6AAAAABJW67VAKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGE2TQNJRHE> . You are receiving this because your review was requested.Message ID: ***@***.***>

[Lazin]

The archival STM was moved to archival module as part of the effort to fix this.

…

On Sat, Jun 22, 2024, 21:46 Evgeny Lazin ***@***.***> wrote: I have a lot of code for the archival subsystem in the feature branch/PRs. This will get in the way in a big way. I'm also don't think archival should be a part of the cluster. We should probably fix dependencies and start building it separately and not as part of the cluster. On Sat, Jun 22, 2024, 21:08 Noah Watkins ***@***.***> wrote: > Force pushed to fix merge conflict. > > — > Reply to this email directly, view it on GitHub > <#19955 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAWMNTNXMUNCXHR5OKUZBLZIXDUPAVCNFSM6AAAAABJW67VAKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGE2TQNJRHE> > . > You are receiving this because your review was requested.Message ID: > ***@***.***> >

[dotnwat]

I have a lot of code for the archival subsystem in the feature branch/PRs.

I'm happy to hold off on this PR and wait for your stuff to merge.

I'm also don't think archival should be a part of the cluster.

Unless that's going to happen very soon, let's move it. It's been sitting there a long time, and it's going to get in the way of Bazel. It's a trivial matter to move it back when someone dedicates time to pulling things apart.

[oleiman]

seems legit, though I'm probably biased toward bazel enablement

[Lazin]

It's a trivial matter to move it back when someone dedicates time to pulling things apart.

I'd really love to use bazel (actually anything which is not cmake) but once we move all the code into one library the code will gradually get more and more entangled.

[dotnwat]

I'd really love to use bazel (actually anything which is not cmake) but once we move all the code into one library the code will gradually get more and more entangled.

i don't think it is realistic to hold up bazel because we'd like archival to be separated from cluster. it's compiled into the cluster library today, but the files are physically separated, and that physical separation isn't preventing the situation from getting worse. someone needs to take the time to separate them if that is the outcome we are shooting for.

[dotnwat]

Converting to draft until @Lazin's PRs are merged.