Bump protoc-bridge to 0.9.8 for Scala >= 2.12
#1688
Merged
+209
−142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This required updates to `scripts/create_repository.py` to accommodate
comparisons between `_2.13` and `_3` versions of ScalaPB artifacts. The
`_3` versions of the most recent ScalaPB artifacts are only available
for Scala 3.3 and later, and `compilerplugin_3` currently still depends
on `protoc-gen_2.13`.
The `MavenCoordinates` dataclass in `scripts/create_repository.py` has
new `unversioned_artifact`, `scala_version`, and `artifact_name`
members. Updates to `MavenCoordinates.is_newer_than()` now compare
objects with matching `artifact_name` values, and take the
`scala_version` into account. These changes then precipitated:
- Replacing the `artifact_name()` call with `artifact_name` accesses.
- Replacing `ArtifactLabelMaker._remove_scala_version_suffix` with
`MavenCoordinates.unversioned_artifact`.
- Extracting `__compare_versions` from `is_newer_than` to reuse the same
code for comparing Scala versions and artifact versions. (Properly
reversed the use of `lhs` and `rhs` in the process.)
This change also removes an unnecessary `try`/`catch` block from
`scripts.ScalaPbCodeGenerator.process()` for Scala 2.12 and above.
The block is still required for the Scala 2.11 implementation of
`scripts.ScalaPbCodeGenerator.run()` and in
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run()`.
Finally, this change adds notes in a few places to indicate where
support remains for Scala 2.11. This supporting code can ultimately be
removed if we ever decide to drop Scala 2.11 support.
---
Tested by setting the `protobuf` dependencies to a versions I know will
break `ScalaPB` 0.11.17:
- `abseil-cpp`: 20220623.1 => 20240722.0
- `protobuf`: v21.7 => v26.1
I temporarily removed the `try`/`catch` block for `Throwable` from
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
I then ran `bazel shutdown` to stop persistent `ProtoScalaPBRule`
workers.
After that, building with Bazel 7.4.1 (since Bazel 6.5.0 can't build
`abseil-cpp` 20240722.0 without setting C++14 compiler flags) crashed as
expected, instead of hanging. Undoing the `abseil-cpp` and `protobuf`
changes and building then produced a working build.
```txt
$ USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1):
scalapb_worker failed: error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--jvm_extra_protobuf_generator_out: java.lang.VerifyError: Cannot inherit from final class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:711)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:706)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at scalapb.options.Scalapb.<clinit>(Scalapb.java:24835)
at scalapb.options.compiler.Scalapb$.registerAllExtensions(Scalapb.scala:8)
at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:52)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$2(SocketBasedPluginFrontend.scala:31)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
at scala.concurrent.package$.blocking(package.scala:146)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$1(SocketBasedPluginFrontend.scala:23)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:30)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: third_party/test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
---
The initial motivation for this change was to try to eliminate the need
for the custom `scripts.ScalaPbCodeGenerator` implementation by
upgrading to `protoc-bridge` 0.9.8. This version contains a change to
`catch` any `Throwable` objects raised by a generator implementation in
`protocbridge.frontend.PluginFrontend.runWithInputStream()`.
- https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
This change originally came from:
- https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
- scalapb/protoc-bridge#367
I closed scalapb/ScalaPB#1771 as a result, and reexamined the
`try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the
`scripts.ScalaPBCodeGenerator` implementations added in bazelbuild#1648.
However, the last `protoc-bridge` version to support Scala 2.11 is
0.7.14. Scala 2.11 won't be able to use `protoc-bridge` 0.9.8, so we
still need to `catch` any `Throwable`s ourselves. This includes the
`catch` blocks from the Scala 2.11 `scripts.ScalaPbCodeGenerator` and
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
---
In `protoc-bridge` 0.9.7 and 0.9.8, `protocbridge.ProtocCodeGenerator`
declares a `run()` method with no implementation (and no `try`/`catch`
block).
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
In `protoc-gen` 0.9.7 and 0.9.8, `protogen.CodeGenApp` implements
`protocbridge.ProtocCodeGenerator`, and `CodeGenApp.run()` wraps a call
to `CodeGenApp.process()` inside a `try`/`catch` block.
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L41-L56
`scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17 extends
`CodeGenApp` and implements `process()`. For this version of
`ScalaPbCodeGenerator`, available in Scala 2.12 and later, there's no
need for a `try`/`catch` wrapper on our end.
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
However, the last available `compilerplugin` version for Scala 2.11 is
0.9.8. `scalapb.ScalaPbCodeGenerator` from that version implements
`protocbridge.ProtocCodeGenerator` and has a `try`/`catch` in its
`run()` method. However, the `Scalapb.registerAllExtensions(registry)`
call on line 14 lies outside this block, producing the crash described
in bazelbuild#1648 (commit 23ae356). This is why
we need the Scala 2.11 implementation of `scripts.ScalaPbCodeGenerator`.
- https://github.com/scalapb/ScalaPB/blob/v0.9.8/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L14
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator`,
also extends `ProtocCodeGenerator` directly, instead of implementing
`CodeGenApp` (so it also builds with Scala 2.11). This is why
`ExtraProtobufGenerator.run()` hangs when we remove the `try`/`catch`
block and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5, even
with later Scala versions.
---
Since `scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17
implements `CodeGenApp`, `scripts.ScalaPbCodeGenerator` for Scala 2.12
should be unnecessary. The only reason we need it is to maintain the
same interface as the Scala 2.11 implementation.
---
Before the `MavenCoordinates` changes, `scripts/create_repository.py`
would ScalaPB flip ScalaPB artifacts between their `_2.13` and `_3`
versions on subsequent runs (e.g., `protoc-bridge_2.13` vs.
`protoc-bridge_3`). See the comments added within
`MavenCoordinates.new()` for details.
---
Building Scala 3.3 and later versions with `compilerplugin_3` and
`protoc-bridge_3` artifacts (instead of `protoc-bridge_2.13` artifacts)
produces the following build failure:
```txt
$ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \
./test_thirdparty_version.sh
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:41)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:32)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar
third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
simuons
approved these changes
Jan 31, 2025
liucijus
approved these changes
Feb 3, 2025
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Feb 28, 2025
Bumps dependencies to versions that are compatible with both Bazel 7.5.0 and 8.0.0. Part of bazelbuild#1482. Closes bazelbuild#1652. - ScalaPB jars: 0.11.17 => 1.0.0-alpha.1 - `rules_python`: 0.38.0 => 1.2.0 - `rules_cc`: 0.0.9 => 0.1.1 - `abseil-cpp`: 20220623.1 => 20250127.0 - `rules_java`: 7.12.4 => 8.9.0 - `protobuf`: 21.7 => 29.3 - `rules_proto`: 6.0.2 => 7.1.0 - `google-common-protos`: 2.52.0 => 2.53.0 This precipitated the following updates also included in this commit: - Loads `java_proto_library` from `com_google_protobuf`. - Bumps `.bazelversion` to 7.5.0. - Sets `.bazelci/presubmit.yml` to use Bazel 7.5.0 instead of 6.5.0, and `last_rc` in place of `7.x`. - Sets `common --enable_workspace --noenable_bzlmod` in `.bazelrc` and `tools/bazel\.rc\.buildkite`. - Adds `allow_empty = True` to a `glob` expression in `//test/semanticdb:lib_with_tempsrc`. - Removes Scala 2.11 test cases. Bazel 6 is officially unsupported as of this change and the upcoming `rules_scala` 7.0.0 release. `WORKSPACE` builds succeed under Bazel 6.5.0 (with C++ compiler flags) for the time being, but are not guaranteed to continue working. (Bazel 6 Bzlmod builds would break anyway, because Bazel 6 doesn't provide `use_repo_rule`, required by `rules_jvm_external` 6.3, which is required by `protobuf` v29.) The `README` now documents that `scala_proto` or any rules otherwise depending on `protobuf` are no longer supported out of the box for Scala 2.11. Such users will have to ensure they register their own downgraded versions, at their own risk of future `rules_scala` 7.x incompatibility. Version bump related updates to `.bazelversion`,`WORKSPACE`, and `third_party/repositories` comprise the bulk of this change. Other small, yet important changes (other than those due to the version bumps noted above) include: - Adding a new `examples/overridden_artifacts` repository and the `overridden_artifacts_example` test case in `test/shell/test_examples.sh`. - Making `_validate_scalac_srcjar()` and `dt_patched_compiler_setup()` in `scala/private/macros/scala_repositories.bzl` more tolerant of dictionaries containing keys mapped to `None`. The new `overridden_artifacts_example` test covers this. Bazel 8 and `rules_java` 8 require `protobuf` >= v29. After the `protobuf` v29 bump, and before the ScalaPB 1.0.0-alpha.1 bump, `scala_proto` targets would fail with the following error: ```txt ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14: ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/.../bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) --jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError: 'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)' at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329) [ ...snip... ] java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:30) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14 Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ``` Here's why the other changes were necessary in light of the version bumps: - `java_proto_library` from `rules_java` is now officially deprecated, hence loading it from `com_google_protobuf`. - Setting `common --enable_workspace --noenable_bzlmod` (instead of `build`) fixes `test_semanticdb_handles_removed_sourcefiles`. This test relies on `bazel query`, which is also affected by these flags, hence `common` instead of `build`. Bazel 8 defaults to `--enable_bzlmod --noenable_workspace`, causing the `WORKSPACE` run of this test to fail. - `glob` requires an explicit `allow_empty = True` parameter now that `--incompatible_disallow_empty_glob` defaults to `True` in Bazel 8. - ScalaPB 0.9.8, the last version compatible with Scala 2.11, does not support `protobuf` >= 26.0. For this reason, we must remove the Scala 2.11 test cases. We should consider dropping Scala 2.11 support at this point, since there's no ScalaPB release for it that supports later versions of `protobuf`. That, and we could remove some of the special case code added in the following changes, amongst other 2.11 support details: - bazelbuild#1631 - bazelbuild#1648 - bazelbuild#1687 - bazelbuild#1688 Finally, the motivation for the non-version bump changes: - The design bug in the upcoming Bzlmod API for `overridden_artifacts` reported by @dmivankov precipitated the `examples/overriden_artifacts` repo and test. See: bazelbuild#1482 (comment) bazelbuild#1482 (comment) - The `_validate_scalac_srcjar()` update maintains the strict checks for mutually exclusive values, while preventing client code from having to explicitly filter out `None` entries. This pairs with the change in `dt_patched_compiler_setup()` that uses the `compiler_srcjar` dictionary. These changes helps keep the upcoming module extension a bit cleaner.
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Feb 28, 2025
Bumps dependencies to versions that are compatible with both Bazel 7.5.0 and 8.0.0. Part of bazelbuild#1482. Closes bazelbuild#1652. - ScalaPB jars: 0.11.17 => 1.0.0-alpha.1 - `rules_python`: 0.38.0 => 1.2.0 - `rules_cc`: 0.0.9 => 0.1.1 - `abseil-cpp`: 20220623.1 => 20250127.0 - `rules_java`: 7.12.4 => 8.9.0 - `protobuf`: 21.7 => 29.3 - `rules_proto`: 6.0.2 => 7.1.0 - `google-common-protos`: 2.52.0 => 2.53.0 This precipitated the following updates also included in this commit: - Loads `java_proto_library` from `com_google_protobuf`. - Bumps `.bazelversion` to 7.5.0. - Sets `.bazelci/presubmit.yml` to use Bazel 7.5.0 instead of 6.5.0, and `last_rc` in place of `7.x`. - Sets `common --enable_workspace --noenable_bzlmod` in `.bazelrc` and `tools/bazel\.rc\.buildkite`. - Adds `allow_empty = True` to a `glob` expression in `//test/semanticdb:lib_with_tempsrc`. - Removes Scala 2.11 test cases. Bazel 6 is officially unsupported as of this change and the upcoming `rules_scala` 7.0.0 release. `WORKSPACE` builds succeed under Bazel 6.5.0 (with C++ compiler flags) for the time being, but are not guaranteed to continue working. (Bazel 6 Bzlmod builds would break anyway, because Bazel 6 doesn't provide `use_repo_rule`, required by `rules_jvm_external` 6.3, which is required by `protobuf` v29.) The `README` now documents that `scala_proto` or any rules otherwise depending on `protobuf` are no longer supported out of the box for Scala 2.11. Such users will have to ensure they register their own downgraded versions, at their own risk of future `rules_scala` 7.x incompatibility. Version bump related updates to `.bazelversion`,`WORKSPACE`, and `third_party/repositories` comprise the bulk of this change. Other small, yet important changes (other than those due to the version bumps noted above) include: - Adding a new `examples/overridden_artifacts` repository and the `overridden_artifacts_example` test case in `test/shell/test_examples.sh`. - Making `_validate_scalac_srcjar()` and `dt_patched_compiler_setup()` in `scala/private/macros/scala_repositories.bzl` more tolerant of dictionaries containing keys mapped to `None`. The new `overridden_artifacts_example` test covers this. --- We're no longer planning to support Bazel 6 in the next major release per @simuons's decision in: - bazelbuild#1482 (comment) The plan is now to land the Bazel 7 and 8 compatibility updates first, then land the Bzlmod change. This enables us to make only one new major version release, instead of two (the first supporting with Bazel 6, the second dropping support). Bazel 8 and `rules_java` 8 require `protobuf` >= v29. After the `protobuf` v29 bump, and before the ScalaPB 1.0.0-alpha.1 bump, `scala_proto` targets would fail with the following error: ```txt ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14: ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/.../bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) --jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError: 'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)' at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329) [ ...snip... ] java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:30) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14 Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ``` Here's why the other changes were necessary in light of the version bumps: - `java_proto_library` from `rules_java` is now officially deprecated, hence loading it from `com_google_protobuf`. - Setting `common --enable_workspace --noenable_bzlmod` (instead of `build`) fixes `test_semanticdb_handles_removed_sourcefiles`. This test relies on `bazel query`, which is also affected by these flags, hence `common` instead of `build`. Bazel 8 defaults to `--enable_bzlmod --noenable_workspace`, causing the `WORKSPACE` run of this test to fail. - `glob` requires an explicit `allow_empty = True` parameter now that `--incompatible_disallow_empty_glob` defaults to `True` in Bazel 8. - ScalaPB 0.9.8, the last version compatible with Scala 2.11, does not support `protobuf` >= 26.0. For this reason, we must remove the Scala 2.11 test cases. We should consider dropping Scala 2.11 support at this point, since there's no ScalaPB release for it that supports later versions of `protobuf`. That, and we could remove some of the special case code added in the following changes, amongst other 2.11 support details: - bazelbuild#1631 - bazelbuild#1648 - bazelbuild#1687 - bazelbuild#1688 Finally, the motivation for the non-version bump changes: - The design bug in the upcoming Bzlmod API for `overridden_artifacts` reported by @dmivankov precipitated the `examples/overriden_artifacts` repo and test. See: bazelbuild#1482 (comment) bazelbuild#1482 (comment) - The `_validate_scalac_srcjar()` update maintains the strict checks for mutually exclusive values, while preventing client code from having to explicitly filter out `None` entries. This pairs with the change in `dt_patched_compiler_setup()` that uses the `compiler_srcjar` dictionary. These changes helps keep the upcoming module extension a bit cleaner.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This updates to the latest
protoc-bridge, removes code that isn't necessary any longer, and documents code that must remain for now. Part of #1482.This also required the following
scripts/create_repository.pyupdates:_2.13and_3versions of ScalaPB artifacts.MavenCoordinatesdataclass inscripts/create_repository.pyhas newunversioned_artifact,scala_version, andartifact_namemembers. Updates toMavenCoordinates.is_newer_than()now compare objects with matchingartifact_namevalues, and take thescala_versioninto account.These changes then precipitated:
artifact_name()call withartifact_nameaccesses.ArtifactLabelMaker._remove_scala_version_suffixwithMavenCoordinates.unversioned_artifact.__compare_versionsfromis_newer_thanto reuse the same code for comparing Scala versions and artifact versions. (Properly reversed the use oflhsandrhsin the process.)This change also removes an unnecessary
try/catchblock fromscripts.ScalaPbCodeGenerator.process()for Scala 2.12 and above. The block is still required for the Scala 2.11 implementation ofscripts.ScalaPbCodeGenerator.run()and inscalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run().Finally, this change adds notes in a few places to indicate where support remains for Scala 2.11. This supporting code can ultimately be removed if we ever decide to drop Scala 2.11 support.
Motivation
The initial motivation for this change was to try to eliminate the need for the custom
scripts.ScalaPbCodeGeneratorimplementation by upgrading toprotoc-bridge0.9.8. This version contains a change tocatchanyThrowableobjects raised by a generator implementation inprotocbridge.frontend.PluginFrontend.runWithInputStream().However, the last
protoc-bridgeversion to support Scala 2.11 is 0.7.14. Scala 2.11 won't be able to useprotoc-bridge0.9.8, so we still need tocatchanyThrowables ourselves. This includes thecatchblocks from the Scala 2.11scripts.ScalaPbCodeGeneratorandscalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run.Since
scalapb.ScalaPbCodeGeneratorfromcompilerplugin0.11.17 implementsCodeGenApp,scripts.ScalaPbCodeGeneratorfor Scala 2.12 should be unnecessary. The only reason we need it is to maintain the same interface as the Scala 2.11 implementation.The
_3versions of the most recent ScalaPB artifacts are only available for Scala 3.3 and later, andcompilerplugin_3currently still depends onprotoc-gen_2.13. Building Scala 3.3 and later versions withcompilerplugin_3andprotoc-bridge_3artifacts (instead ofprotoc-bridge_2.13artifacts) produced a build failure.Before the
MavenCoordinateschanges,scripts/create_repository.pywould flip ScalaPB artifacts between their_2.13and_3versions on subsequent runs (e.g.,protoc-bridge_2.13vs.protoc-bridge_3). See the comments added withinMavenCoordinates.new()for details.The commit message expands greatly on the above details. I've also broken my recent one-pull-request-at-a-time rule because this change is orthogonal to all the others. (All of the remaining changes described in #1482, possibly aside from documentation updates, should land sequentially.)