Deprecate ScalaThenJava, and improve the docs

[eed3si9n]

Ref #235

[typesafe-tools]

To validate this pull request against other sbt modules, please visit this link.

[dwijnand]

My reading of #235 is that there's a small but not insignificant difference. If there isn't a strong reason to drop this I'd say let's avoid the needless breakage.

[eed3si9n]

My reading of #235 is that there's a small but not insignificant difference.

Insignificant difference in the promised meaning between ScalaThenJava and Mixed?
We don't implement the semantics that are promised so we should not keep it.

[dwijnand]

You can't make the Scala source depend on the Java source because the Java source is never passed to scalac. The fact that it's not handled in incremental is an incremental bug. But people might still be relying on it for regular, full compilation (e.g. in CI), so I don't think it should be just deleted.

[eed3si9n]

But people might still be relying on it for regular, full compilation (e.g. in CI), so I don't think it should be just deleted.

I'm happy to update the documentation to reflect the reality as much as possible, but I think this is a good candidate to deprecate it as an option since it's a pointless ordering that would only cause confusion.

[eed3si9n]

To clarify the situation, do you agree with my assessment that all these CompileOrders are there to squeeze "scalac doesn't have to parse Java sources" performance, and the fact that Scala-doesn't-see-Java or vice versa is an inconvenient side effect? If ppl wanted to enforce some invisibility, they should use subprojects, not compilation order.

[dwijnand]

That's not my assessment, no. My guess is that people use it for either of those reasons and I wouldn't characterise it as a "side effect" or "downside" - it's a strategy.

[eed3si9n]

My guess is that people use it

Does anyone use ScalaThenJava? I don't think anyone actually uses this flag, or if they are using it the person is confused about what it achieves. The notion of CompileOrder is an implementation detail / weird hack at best. The more I think about it, I think it would be even better if we deprecated JavaThenScala as well, and do what Scala compiler is naturally designed to do.

[dwijnand]

I don't think anyone actually uses this flag, or if they are using it the person is confused about what it achieves

Those are just guesses, right?

Are these all instances of confusion? https://github.com/search?p=4&q=ScalaThenJava+extension%3Asbt&type=Code

The notion of CompileOrder is an implementation detail / weird hack at best.

What are you basing that on? Looks like it's even documented: https://www.scala-sbt.org/1.x/docs/Java-Sources.html

[eed3si9n]

re: I don't think anyone actually uses this flag

Are these all instances of confusion? https://github.com/search?p=4&q=ScalaThenJava+extension%3Asbt&type=Code

I did search GitHub before commenting the above, and yes, compared to 200k builds out there one effectively no active project is using CompileOrder.ScalaThenJava.

re: CompileOrders are there to squeeze out scalac-java-parse performance

Nice find on the documentation. I think the documentation is consistent with what I was saying, which is the the purpose of this flag is to eke out performance:

If you do not have circular dependencies, you can use one of the other two options to speed up your build by not passing the Java sources to scalac.

I think it would be good to test this claim too. Maybe it's still relevant when you have 100s of generated Java sources.

re: implementation detail / weird hack at best.

I expect Zinc to act as a non-opinionated wrapper around Scala compiler, Java compiler, and we should refrain from adding semantic changes to how the code is compiled. CompileOrder introduces ad hoc visibility restriction to the compilation (i.e. Scala source can't see classes defined in Java sources) that's not even consistent during incremental cycles.

[dwijnand]

Btw, scala/scala depends on JavaThenScala: https://github.com/scala/scala/blob/8329196f299cc16946adc3f9d03a408441df6a96/build.sbt#L442-L444

[SethTisue]

I have mixed, mostly-negative feelings about deprecating JavaThenScala. It can't hurt performance and seems almost certain to help, at least a little; it's safer in the sense that you don't have to worry about running into weird bugs in scalac's Java parser (and I have found many such over the years, some of which remain unfixed); and perhaps most importantly, it imposes an architectural constraint on the code that is desired on some projects. In scala/scala, for example, we don't want any Java code to depend on any Scala code. Things that are in Java are the 0-layer architecturally, it's a mentally simpler model to know that, since JavaThenScala wouldn't work otherwise.

Finally — and this argument only occurred to me as I was writing the above paragraph — Scala's Java parser hasn't been updated in donkey's years. There are post-Java-8 constructs that it doesn't understand. If my Java code uses those constructs, I'm going to need JavaThenScala (or be forced to move the Java code to a different subproject, anyway, which I might not want to do for a variety of reasons).

(I wouldn't miss ScalaThenJava; I doubt anyone would.)

[eed3si9n]

ok. Let's keep JavaThenScala with renewed understanding of what it's for, and deprecate ScalaThenJava.

[SethTisue]

@dwijnand you never weighed back in about deprecating ScalaThenJava. I agree with Eugene here that the deprecation should go forward, and I'll just make one further argument that is perhaps obvious but hasn't been explicitly brought forward: at present, at this stage, we're only deprecating, not outright removing. if someone notices the deprecation warning and comes forward with a convincing argument against it, there will still be time to un-deprecate. (I don't think that's actually a likely outcome, but I've been wrong before, or so I'm told.)

[SethTisue]

note the introduction of @nowarn into the codebase, which Eugene confirmed (over at #961) is okay to do. There is a lot of warnings noise in the build it would good to start cutting down on.

[dwijnand]

@dwijnand you never weighed back in about deprecating ScalaThenJava.

I'm just as against deprecating is as JavaThenScala. The costs outweighs the benefits, in my opinion: it's going to introduce noise, thus more work, possibly fatal warning breakages, or possible real breakages when the deprecation cycle is done and its removed, and what have we gained? It might not be the most popular feature , but removing it isn't worth it, as I see it.

[SethTisue]

I'm not super gung ho about merging this, so whatever @eed3si9n decides is fine w/ me. I worry about the maintenance footprint (for us) and the mental bandwidth footprint (for users) of little-used, under-tested features, but I don't have much background in how such decisions tend to play out in this repo or in sbt generally, and Dale's position makes sense too, so... 🤷

[eed3si9n]

Setting aside the deprecation debate, we should document:

1. what those options actually do in clean compile
2. what those options do in an incremental compile
3. why build users might be interested in using either of the options

[SethTisue]

re: 1 and 2: Okay — I could review further documentation improvements, but I'm not a position to make them myself.

why build users might be interested in using either of the options

re: 3, the current state of the PR does that to the best of my ability, except that I spent less time on ScalaThenJava since I feel like the discussion (here and on the previous two tickets) seems to show that we truly understand what it's for.

[dwijnand]

I've no qualms at all in recording the behaviour details.