Skip to content

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Sep 19, 2025

Replace PyBytes_FromStringAndSize() and _PyBytes_Resize() with the PyBytesWriter API.

Replace PyBytes_FromStringAndSize() and _PyBytes_Resize() with the
PyBytesWriter API.
@vstinner
Copy link
Member Author

vstinner commented Sep 22, 2025

Benchmark ASCII characters:

import pyperf
runner = pyperf.Runner()
for size in (3, 100, 1000):
    runner.timeit(f'{size:,} chars',
        setup=f's="x"*{size}',
        stmt='s.encode("utf32")')

Result:

Benchmark ref pep782
100 chars 58.9 ns 61.5 ns: 1.04x slower
1,000 chars 185 ns 188 ns: 1.02x slower
Geometric mean (ref) 1.02x slower

Benchmark hidden because not significant (1): 3 chars

@vstinner
Copy link
Member Author

Benchmark UCS-4 characters:

import pyperf
runner = pyperf.Runner()
for size in (3, 100, 1000):
    runner.timeit(f'{size:,} UCS-4 chars',
        setup=f's=chr(0x10ffff) * {size}',
        stmt='s.encode("utf32")')

Result:

Benchmark ref pep782
3 UCS-4 chars 54.7 ns 63.9 ns: 1.17x slower
100 UCS-4 chars 99.4 ns 101 ns: 1.02x slower
1,000 UCS-4 chars 546 ns 542 ns: 1.01x faster
Geometric mean (ref) 1.06x slower

@vstinner
Copy link
Member Author

cc @serhiy-storchaka

@vstinner
Copy link
Member Author

The only significant difference is on "3 UCS-4 chars": 54.7 ns => 63.9 ns: 1.17x slower, +9.2 seconds. That's the cost of the PyBytesWriter API abstraction.

I added a "fast path" for UCS-1 which is the most common cases: it keeps PyBytes_FromStringAndSize(NULL, size). So there is no impact on performance.

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

@vstinner vstinner enabled auto-merge (squash) September 22, 2025 12:38
@vstinner vstinner merged commit 92ba2c9 into python:main Sep 22, 2025
77 of 79 checks passed
@vstinner vstinner deleted the utf32 branch September 22, 2025 20:05
@vstinner
Copy link
Member Author

Merged, thanks for the review @serhiy-storchaka.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants