speed up String::push and String::insert

[lincot]

Addresses the concerns described in #116235.

The performance gain comes mainly from avoiding temporary buffers.

Complex pattern matching in encode_utf8 (introduced in #67569) has been simplified to a comparison and an exhaustive match in the encode_utf8_raw_unchecked helper function. It takes a slice of MaybeUninit<u8> because otherwise we'd have to construct a normal slice to uninitialized data, which is not desirable, I guess.

Several functions still have that unneeded zeroing, but a single instruction is not that important, I guess.

@rustbot label T-libs C-optimization A-str

[rustbot]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @scottmcm (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

• @rustbot author: the review is finished, PR author should check the comments and take action accordingly
• @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

[scottmcm]

I had a variety of thoughts; let me know what you think.

Also, is there anything here for which it would make sense to have a codegen test to confirm what's happening? Or some other test to help confirm it's better?

[lincot]

A codegen check for the absence of memcpy would be nice, since the original String::push has one.

[rustbot]

There are merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged.

You can start a rebase with the following commands:

$ # rebase
$ git pull --rebase https://github.com/rust-lang/rust.git master
$ git push --force-with-lease

The following commits are merge commits:

• 9511918

[lincot]

The proposed implementation uses get_unchecked_mut, which cannot be used in char::encode_utf8, which is now const. Also, get_unchecked_mut actually has a cost when running with debug assertions. So I am reverting to using pointers. The codegen of String::push seems to be unchanged: godbolt.

[lincot]

@rustbot review

[tgross35]

@scottmcm are you able to review this? I could probably take a look if not.

[tgross35]

I'll at least get this going

@bors try @rust-timer queue

[bors]

⌛ Trying commit 9cf92e5 with merge 26f0bba...

[bors]

☀️ Try build successful - checks-actions
Build commit: 26f0bba (26f0bbac902496404cadab55a28b41a2cafaa9c4)

[rust-timer]

Finished benchmarking commit (26f0bba): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.1%, 0.2%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.5%	[-0.6%, -0.3%]	3
Improvements ✅ (secondary)	-0.4%	[-0.8%, -0.3%]	11
All ❌✅ (primary)	-0.2%	[-0.6%, 0.2%]	6

Max RSS (memory usage)

Results (primary 3.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	7.5%	[1.2%, 14.0%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-5.3%	[-6.4%, -4.2%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.2%	[-6.4%, 14.0%]	6

Cycles

Results (primary 2.4%, secondary -2.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.4%	[2.4%, 2.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.2%	[-2.2%, -2.2%]	1
All ❌✅ (primary)	2.4%	[2.4%, 2.4%]	1

Binary size

Results (primary -0.1%, secondary -0.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.1%	[0.0%, 0.7%]	17
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-1.2%, -0.0%]	29
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	37
All ❌✅ (primary)	-0.1%	[-1.2%, 0.7%]	46

Bootstrap: 774.477s -> 773.266s (-0.16%)
Artifact size: 361.04 MiB -> 361.02 MiB (-0.00%)

[tgross35]

Sorry this has been sitting for so long, I have one last question then I think we can merge this. Mind posting the results of library/alloctests/benches/string.rs if you have run those?

[tgross35]

It's been long enough that I'm forgetting context here, but why was this changed away from MaybeUninit? Specifically thinking of a signature like

pub const unsafe fn encode_utf8_raw_unchecked(
    code: u32, dst: &mut [MaybeUninit<u8>]
) -> &mut [u8] {
    // Write the characters then call MaybeUninit::assume_init_ref
}

Then lengths get checked and push becomes slightly simpler with core::char::encode_utf8_raw_unchecked(ch as u32, self.vec.spare_capacity_mut()) (maybe needs an assert_unchecked(self.buf.capacity() - self.len > len) if LLVM doesn't pick up on that).