remove TokenStream from the bridge

[cyrgani]

TODO:

• figure out why the input token streams have worse formatting now
• improve perf run results
• cleanup
• description
• fix r-a

related: #101419

[jieyouxu]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: a1f3eec (a1f3eecc1c3e174e689386582cf8307d3b170a81, parent: 370143facfb348ad3b29749c0393402d76b280c3)

[rust-timer]

Finished benchmarking commit (a1f3eec): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	1.6%	[0.1%, 6.9%]	64
Regressions ❌ (secondary)	0.9%	[0.1%, 2.3%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-22.3%	[-40.7%, -2.0%]	10
All ❌✅ (primary)	1.6%	[0.1%, 6.9%]	64

Max RSS (memory usage)

Results (primary 2.6%, secondary -4.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.6%	[2.6%, 2.6%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.2%	[-4.2%, -4.2%]	1
All ❌✅ (primary)	2.6%	[2.6%, 2.6%]	1

Cycles

Results (primary 0.7%, secondary -10.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.4%	[2.0%, 8.6%]	14
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-9.7%	[-10.8%, -8.6%]	5
Improvements ✅ (secondary)	-10.5%	[-20.0%, -3.8%]	9
All ❌✅ (primary)	0.7%	[-10.8%, 8.6%]	19

Binary size

Results (primary 1.0%, secondary -0.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.0%	[0.1%, 1.6%]	21
Regressions ❌ (secondary)	0.2%	[0.1%, 0.4%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.0%	[-1.0%, -1.0%]	3
All ❌✅ (primary)	1.0%	[0.1%, 1.6%]	21

Bootstrap: 474.117s -> 476.858s (0.58%)
Artifact size: 397.79 MiB -> 397.77 MiB (-0.01%)

[cyrgani]

@jieyouxu can you start another perf run?

[Kobzol]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 31b33e0 (31b33e0c3dcaf50135ca0eb8d0708ae383a35d85, parent: e823167aa6f7f03aea8e91208ce9bace8ad9ebf2)

[rust-timer]

Finished benchmarking commit (31b33e0): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	13.4%	[0.2%, 87.1%]	67
Regressions ❌ (secondary)	24735.6%	[0.0%, 97823.1%]	21
Improvements ✅ (primary)	-0.5%	[-0.5%, -0.5%]	1
Improvements ✅ (secondary)	-0.1%	[-0.2%, -0.0%]	2
All ❌✅ (primary)	13.2%	[-0.5%, 87.1%]	68

Max RSS (memory usage)

Results (primary 0.0%, secondary 1.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.9%	[1.3%, 2.2%]	3
Regressions ❌ (secondary)	3.0%	[2.9%, 3.0%]	2
Improvements ✅ (primary)	-1.8%	[-2.8%, -0.6%]	3
Improvements ✅ (secondary)	-2.5%	[-2.5%, -2.5%]	1
All ❌✅ (primary)	0.0%	[-2.8%, 2.2%]	6

Cycles

Results (primary 21.5%, secondary 31513.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	21.5%	[2.0%, 87.2%]	41
Regressions ❌ (secondary)	37816.9%	[4.0%, 112549.0%]	15
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-6.3%	[-6.4%, -6.1%]	3
All ❌✅ (primary)	21.5%	[2.0%, 87.2%]	41

Binary size

Results (primary 1.5%, secondary 1.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.5%	[0.6%, 2.2%]	21
Regressions ❌ (secondary)	1.6%	[0.6%, 2.6%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.5%	[0.6%, 2.2%]	21

Bootstrap: 474.712s -> 475.37s (0.14%)
Artifact size: 397.79 MiB -> 397.76 MiB (-0.01%)

[Kobzol]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 51c7151 (51c7151b743f433c915d9a6b9f22555b7620f9ed, parent: 878374e07f3bf038c96e94e5bc917471878e4bf6)

[rust-timer]

Finished benchmarking commit (51c7151): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	1.6%	[0.3%, 4.0%]	18
Regressions ❌ (secondary)	0.4%	[0.1%, 1.3%]	4
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-23.0%	[-45.5%, -1.9%]	11
All ❌✅ (primary)	1.5%	[-0.7%, 4.0%]	19

Max RSS (memory usage)

Results (secondary -2.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	1
All ❌✅ (primary)	-	-	0

Cycles

Results (primary 3.3%, secondary -12.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.3%	[2.5%, 4.5%]	5
Regressions ❌ (secondary)	4.5%	[4.5%, 4.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-13.9%	[-28.0%, -4.8%]	10
All ❌✅ (primary)	3.3%	[2.5%, 4.5%]	5

Binary size

Results (primary 1.4%, secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.4%	[0.5%, 2.5%]	20
Regressions ❌ (secondary)	1.0%	[0.9%, 1.0%]	4
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.1%]	1
Improvements ✅ (secondary)	-0.9%	[-1.2%, -0.1%]	4
All ❌✅ (primary)	1.4%	[-0.1%, 2.5%]	21

Bootstrap: 475.726s -> 476.233s (0.11%)
Artifact size: 397.87 MiB -> 397.83 MiB (-0.01%)

[Kobzol]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 58b7e13 (58b7e137a08dabf1726a511136938bcc296d3e00, parent: a60d12cbccfbeaf153f3cecb90454aa696ea4b3b)

[rust-timer]

Finished benchmarking commit (58b7e13): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	1.7%	[0.3%, 4.1%]	17
Regressions ❌ (secondary)	0.3%	[0.1%, 0.9%]	3
Improvements ✅ (primary)	-0.8%	[-0.8%, -0.8%]	1
Improvements ✅ (secondary)	-22.9%	[-45.6%, -0.8%]	11
All ❌✅ (primary)	1.6%	[-0.8%, 4.1%]	18

Max RSS (memory usage)

Results (primary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.2%	[2.2%, 2.2%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.0%	[-1.2%, -0.7%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.1%	[-1.2%, 2.2%]	3

Cycles

Results (primary 2.0%, secondary -13.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.1%	[2.4%, 4.1%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.3%, -2.3%]	1
Improvements ✅ (secondary)	-13.5%	[-26.2%, -5.0%]	10
All ❌✅ (primary)	2.0%	[-2.3%, 4.1%]	5

Binary size

Results (primary 1.4%, secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.4%	[0.1%, 2.5%]	21
Regressions ❌ (secondary)	0.8%	[0.1%, 1.0%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.1%	[-1.2%, -1.1%]	3
All ❌✅ (primary)	1.4%	[0.1%, 2.5%]	21

Bootstrap: 473.737s -> 471.154s (-0.55%)
Artifact size: 397.70 MiB -> 397.69 MiB (-0.00%)