Skip to content

Commit 6e36591

Browse files
authored
Toolchainize all testing toolchains (#1653)
* Toolchainize all testing toolchains Moves the Scalatest, JUnit, and Specs2 toolchains into `@io_bazel_rules_scala_toolchains//testing`. Part of #1482. Updates all `WORKSPACE` files to set the appropriate `scala_toolchains` parameters and to remove the unnecessary repository import and toolchain registration macros. Adds a `fetch_sources_by_id` parameter to `repositories` from `third_party/repositories/repositories.bzl`. This enables `scala_toolchains` to build the `artifact_ids_to_fetch_sources` mapping from artifact ID lists returned by new macros extracted from `WORKSPACE` macros. The values assigned to each id match the original `fetch_sources` settings in the corresponding original `WORKSPACE` macros. Updates `scala/scala_maven_import_external.bzl` to generate a `load` line for `//scala:scala_import.bzl` based on the repo's canonical name, not `@io_bazel_rules_scala`. As usual, includes several other opportunistic removals of the `@io_bazel_rules_scala` repo name prefix to avoid an internal dependency on that name. This means Bzlmod users won't necessarily have to set the `repo_name` parameter of `bazel_dep` when using `rules_scala`. --- Introduces more macros to return a framework's Maven artifact dependencies, rather than inlining them in a `repositories` call. These inlined lists are replaced by macro invocations, and now the `scala_toolchains` macro can invoke these macros to collect artifact IDs to pass to `repositories`. This also allows for future changes to introduce a `scala_version` parameter if necessary, similar to how `scalafmt_artifact_ids` already works. This is important to avoid collisions when creating repositories for artifacts upon which more than one framework depends under Bzlmod. `WORKSPACE` doesn't seem affected by these collisions, but Bzlmod will produce errors like the following, where both `scala_proto` and `twitter_scrooge` depend upon `io_bazel_rules_scala_guava`: ```txt $ bazel build //src/... ERROR: .../scala/scala_maven_import_external.bzl:299:24: Traceback (most recent call last): File ".../scala/extensions/deps.bzl", line 140, column 21, in _scala_deps_impl scala_toolchains( File ".../scala/private/macros/toolchains.bzl", line 140, column 17, in scala_toolchains _scrooge( File ".../twitter_scrooge/twitter_scrooge.bzl", line 96, column 17, in twitter_scrooge repositories( File ".../third_party/repositories/repositories.bzl", line 113, column 37, in repositories _scala_maven_import_external( File ".../scala/scala_maven_import_external.bzl", line 263, column 30, in scala_maven_import_external jvm_maven_import_external( File ".../scala/scala_maven_import_external.bzl", line 299, column 24, in jvm_maven_import_external jvm_import_external(jar_urls = jar_urls, srcjar_urls = srcjar_urls, coordinates = artifact, **kwargs) Error in repository_rule: A repo named io_bazel_rules_scala_guava_2_13_15 is already generated by this module extension at .../scala/scala_maven_import_external.bzl:299:24 ERROR: Analysis of target '//src/java/io/bazel/rulesscala/worker:worker_test' failed; build aborted: error evaluating module extension scala_deps in //scala/extensions:deps.bzl ``` Recent updates to `scripts/create_repository.py` (#1639, #1642) make it easy to emit full direct dependency lists for artifacts included in `third_party/repositories/scala_*.bzl`. This increases the likelihood of collisions, since this expanded metadata forces the macros that instantiate artifact repos to instantiate even more repos. By fetching list of artifact IDs from these macros, `scala_toolchains` can now consolidate them into dictionary keys. Then it passes these unique keys to `repositories` directly, avoiding the problem of instantiating the same repo multiple times in the same module extension. This, in turn, also avoids the need to add parameters to the original `WORKSPACE` macros that instantiate dependencies to avoid collisions under Bzlmod. The `scala_toolchains` macro never needs to call these original macros, under either `WORKSPACE` or Bzlmod. Finally, it also reduces duplication between these artifact ID lists and the `_*_DEPS` symbols originally from `testing/BUILD` (and now in `testing/deps.bzl`). The dependency labels are now generated programatically. (Aside: As I mentioned, we may eventually need to pass a Scala version argument to these macros. It will be possible to cross that bridge without too much trouble if and when that day comes. Or I can try to future proof it in a follow up pull request.) * Move `scala_toolchains` to `scala/toolchains.bzl` Removes this symbol from `scala/scala.bzl` as well as `setup_scala_testing_toolchain`, and deletes `scala/private/macros/toolchains.bzl`. Part of #1482 and #1652. This is required for Bazel 8 and `rules_java` 8 compatibility, but is also compatible with Bazel 6 and 7. In #1652, @hvadehra suggested partitioning the `.bzl` files such that `WORKSPACE` doesn't `load` a file that tries to `load` symbols from `rules_java`. I successfully did so in a separate branch, and along with other minor changes, got `rules_scala` to build with `rules_java` 8.5.1. The other changes will come in separate pull requests, but it makes sense to land this change now before adding any other toolchains to `scala_toolchains`. --- Arguably, we should remove all macros exported from `scala/scala.bzl` that only instantiate toolchain dependencies and define toolchains. That may be a breaking change for some users, but will ultimately be necessary for these macros to remain compatible with Bazel 8. * Extract versioned `_JUNIT_DEPS` in `test/BUILD` Eliminates reliance on the default `@io_bazel_rules_scala_junit_junit` artifact repository. * Update `{junit,specs2_junit}_toolchain()` These macros now mirror the implementation of `scalatest_toolchain()`. However, I realized that the old pattern of calling `scalatest_repositories()` followed by `scalatest_toolchain()` will no longer work without first calling `scala_toolchains(scalatest = True)`. This is because the `alias` targets in `testing/BUILD` that replace the previous implementations all point to `@io_bazel_rules_scala_toolchains`. So if we want to keep these macros, it seems like we should maybe restore the original toolchain targets in `testing/BUILD`. If we don't, we can remove these macros, but we can document these as breaking changes, and update other documentation accordingly. * Move `scala/{private/macros/,}toolchains_repo.bzl` Like "Move `scala_toolchains` to `scala/toolchains.bzl`", removes the `scala_toolchains_repo` symbol from `scala/scala.bzl` and makes it available from `scala/toolchains_repo.bzl`. This avoids a future `test_scala_version 2.12.20` failure during Bazel 8 builds after adding `twitter_scrooge` toolchain support in the new `test_version/version_specific_tests_dir/scrooge_repositories.bzl` file. Otherwise, this new file would load `toolchains_repo` from `scala/scala.bzl`. The `test_version/test_scala_version_.../WORKSPACE` file generated from `test_version/WORKSPACE.template` would then transitively load `.bzl` files with `rules_java` symbols, breaking the test. ```txt $ RULES_SCALA_TEST_ONLY="test_scala_version 2.12.20" ./test_version.sh ERROR: Traceback (most recent call last): File ".../external/rules_scala/scala/private/common_attributes.bzl", line 18, column 28, in <toplevel> "deps": attr.label_list( Error in label_list: Illegal argument: element in 'providers' is of unexpected type. Either all elements should be providers, or all elements should be lists of providers, but got list with an element of type NoneType. ERROR: Error computing the main repository mapping: at test_version/test_scala_version_.../scrooge_repositories.bzl:1:6: at .../scala/scala.bzl:30:5: at .../scala/private/rules/scala_junit_test.bzl:5:5: initialization of module 'scala/private/common_attributes.bzl' failed ```
1 parent 719f353 commit 6e36591

File tree

38 files changed

+501
-430
lines changed

38 files changed

+501
-430
lines changed

WORKSPACE

+5-11
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,14 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3030

3131
scala_config(enable_compiler_dependency_tracking = True)
3232

33-
load("//scala:scala.bzl", "scala_toolchains")
33+
load("//scala:toolchains.bzl", "scala_toolchains")
3434

35-
scala_toolchains(fetch_sources = True)
35+
scala_toolchains(
36+
fetch_sources = True,
37+
testing = True,
38+
)
3639

3740
register_toolchains(
38-
"//testing:testing_toolchain",
3941
"//scala:unused_dependency_checker_error_toolchain",
4042
"//test/proto:scalapb_toolchain",
4143
"@io_bazel_rules_scala_toolchains//...:all",
@@ -69,14 +71,6 @@ load("//scala_proto:scala_proto.bzl", "scala_proto_repositories")
6971

7072
scala_proto_repositories()
7173

72-
load("//scalatest:scalatest.bzl", "scalatest_repositories")
73-
74-
scalatest_repositories()
75-
76-
load("//specs2:specs2_junit.bzl", "specs2_junit_repositories")
77-
78-
specs2_junit_repositories()
79-
8074
load("//scala/scalafmt:scalafmt_repositories.bzl", "scalafmt_default_config", "scalafmt_repositories")
8175

8276
scalafmt_default_config()

docs/testing.md

+17-5
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,10 @@ Test framework dependencies are configured via testing toolchain. For convenienc
3131
`setup_scala_testing_toolchain` can be used to define such toolchains.
3232

3333
```starlark
34-
load("@io_bazel_rules_scala//scala:scala.bzl", "setup_scala_testing_toolchain")
34+
load(
35+
"@io_bazel_rules_scala//testing:testing.bzl",
36+
"setup_scala_testing_toolchain",
37+
)
3538
```
3639

3740
Attributes
@@ -53,7 +56,10 @@ Examples (assumes maven deps are managed with rules_jvm_external):
5356

5457
```starlark
5558
# BUILD
56-
load("@io_bazel_rules_scala//scala:scala.bzl", "setup_scala_testing_toolchain")
59+
load(
60+
"@io_bazel_rules_scala//testing:testing.bzl",
61+
"setup_scala_testing_toolchain",
62+
)
5763

5864
setup_scala_testing_toolchain(
5965
name = "scalatest_toolchain",
@@ -77,7 +83,10 @@ register_toolchains('//:scalatest_toolchain')
7783
#### JUnit 4
7884
```starlark
7985
# BUILD
80-
load("@io_bazel_rules_scala//scala:scala.bzl", "setup_scala_testing_toolchain")
86+
load(
87+
"@io_bazel_rules_scala//testing:testing.bzl",
88+
"setup_scala_testing_toolchain",
89+
)
8190

8291
setup_scala_testing_toolchain(
8392
name = "junit_toolchain",
@@ -98,7 +107,10 @@ For Specs2 rules to work, `junit_classpath`, `specs2_junit_classpath` and `specs
98107
be configured.
99108
```starlark
100109
# BUILD
101-
load("@io_bazel_rules_scala//scala:scala.bzl", "setup_scala_testing_toolchain")
110+
load(
111+
"@io_bazel_rules_scala//testing:testing.bzl",
112+
"setup_scala_testing_toolchain",
113+
)
102114

103115
setup_scala_testing_toolchain(
104116
name = "specs2_toolchain",
@@ -122,4 +134,4 @@ Register the toolchain
122134
```starlark
123135
# WORKSPACE
124136
register_toolchains('//:specs2_toolchain')
125-
```
137+
```

dt_patches/test_dt_patches/WORKSPACE

+1-1
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ load("@compiler_sources//:extensions.bzl", "import_compiler_source_repos")
4040

4141
import_compiler_source_repos()
4242

43-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
43+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
4444

4545
scala_toolchains(
4646
fetch_sources = True,

dt_patches/test_dt_patches_user_srcjar/WORKSPACE

+1-1
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ srcjars_by_version = {
113113
},
114114
}
115115

116-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
116+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
117117

118118
scala_toolchains(
119119
fetch_sources = True,

examples/crossbuild/WORKSPACE

+2-8
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,9 @@ scala_config(
3838
],
3939
)
4040

41-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
41+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
4242

43-
scala_toolchains()
43+
scala_toolchains(scalatest = True)
4444

4545
register_toolchains("@io_bazel_rules_scala_toolchains//...:all")
4646

@@ -59,9 +59,3 @@ rules_proto_toolchains()
5959
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
6060

6161
protobuf_deps()
62-
63-
load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories", "scalatest_toolchain")
64-
65-
scalatest_repositories()
66-
67-
scalatest_toolchain()

examples/scala3/WORKSPACE

+1-1
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3131

3232
scala_config(scala_version = "3.6.2")
3333

34-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
34+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
3535

3636
scala_toolchains(fetch_sources = True)
3737

examples/semanticdb/WORKSPACE

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3434

3535
scala_config(scala_version = "2.13.15")
3636

37-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
37+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
3838

3939
scala_toolchains(fetch_sources = True)
4040

examples/testing/multi_frameworks_toolchain/BUILD

+6-5
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,15 @@
1-
load("@io_bazel_rules_scala//scala:scala.bzl", "setup_scala_testing_toolchain")
2-
load("@io_bazel_rules_scala//scala:scala_cross_version.bzl", "version_suffix")
3-
load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSION")
1+
load(
2+
"@io_bazel_rules_scala//testing:testing.bzl",
3+
"setup_scala_testing_toolchain",
4+
)
45

56
setup_scala_testing_toolchain(
67
name = "testing_toolchain",
78
junit_classpath = [
89
"@io_bazel_rules_scala_junit_junit",
910
"@io_bazel_rules_scala_org_hamcrest_hamcrest_core",
1011
],
11-
scalatest_classpath = [dep + version_suffix(SCALA_VERSION) for dep in [
12+
scalatest_classpath = [
1213
"@io_bazel_rules_scala_scalactic",
1314
"@io_bazel_rules_scala_scalatest",
1415
"@io_bazel_rules_scala_scalatest_compatible",
@@ -21,7 +22,7 @@ setup_scala_testing_toolchain(
2122
"@io_bazel_rules_scala_scalatest_matchers_core",
2223
"@io_bazel_rules_scala_scalatest_mustmatchers",
2324
"@io_bazel_rules_scala_scalatest_shouldmatchers",
24-
]],
25+
],
2526
specs2_classpath = [
2627
"@io_bazel_rules_scala_org_specs2_specs2_common",
2728
"@io_bazel_rules_scala_org_specs2_specs2_core",

examples/testing/multi_frameworks_toolchain/WORKSPACE

+5-12
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,12 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3131

3232
scala_config()
3333

34-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
34+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
3535

36-
scala_toolchains(fetch_sources = True)
36+
scala_toolchains(
37+
fetch_sources = True,
38+
testing = True,
39+
)
3740

3841
register_toolchains(
3942
":testing_toolchain",
@@ -55,13 +58,3 @@ rules_proto_toolchains()
5558
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
5659

5760
protobuf_deps()
58-
59-
load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories")
60-
load("@io_bazel_rules_scala//testing:junit.bzl", "junit_repositories")
61-
load("@io_bazel_rules_scala//testing:specs2_junit.bzl", "specs2_junit_repositories")
62-
63-
scalatest_repositories()
64-
65-
junit_repositories()
66-
67-
specs2_junit_repositories()

examples/testing/scalatest_repositories/WORKSPACE

+5-8
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,12 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3131

3232
scala_config()
3333

34-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
34+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
3535

36-
scala_toolchains(fetch_sources = True)
36+
scala_toolchains(
37+
fetch_sources = True,
38+
scalatest = True,
39+
)
3740

3841
register_toolchains("@io_bazel_rules_scala_toolchains//...:all")
3942

@@ -52,9 +55,3 @@ rules_proto_toolchains()
5255
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
5356

5457
protobuf_deps()
55-
56-
load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories", "scalatest_toolchain")
57-
58-
scalatest_repositories()
59-
60-
scalatest_toolchain()

examples/testing/specs2_junit_repositories/WORKSPACE

+5-8
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,12 @@ load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config")
3131

3232
scala_config()
3333

34-
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_toolchains")
34+
load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_toolchains")
3535

36-
scala_toolchains(fetch_sources = True)
36+
scala_toolchains(
37+
fetch_sources = True,
38+
specs2 = True,
39+
)
3740

3841
register_toolchains("@io_bazel_rules_scala_toolchains//...:all")
3942

@@ -52,9 +55,3 @@ rules_proto_toolchains()
5255
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
5356

5457
protobuf_deps()
55-
56-
load("@io_bazel_rules_scala//testing:specs2_junit.bzl", "specs2_junit_repositories", "specs2_junit_toolchain")
57-
58-
specs2_junit_repositories()
59-
60-
specs2_junit_toolchain()

junit/junit.bzl

+7-4
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@ load(
44
)
55
load("//third_party/repositories:repositories.bzl", "repositories")
66

7+
def junit_artifact_ids():
8+
return [
9+
"io_bazel_rules_scala_junit_junit",
10+
"io_bazel_rules_scala_org_hamcrest_hamcrest_core",
11+
]
12+
713
def junit_repositories(
814
maven_servers = _default_maven_server_urls(),
915
fetch_sources = True):
1016
repositories(
11-
for_artifact_ids = [
12-
"io_bazel_rules_scala_junit_junit",
13-
"io_bazel_rules_scala_org_hamcrest_hamcrest_core",
14-
],
17+
for_artifact_ids = junit_artifact_ids(),
1518
fetch_sources = fetch_sources,
1619
maven_servers = maven_servers,
1720
)

scala/private/macros/toolchains.bzl

-66
This file was deleted.

0 commit comments

Comments
 (0)