Skip to content

Python host optimizations (bytes_to_hexstr, log formatting, _now_ms) #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

lukasnee
Copy link

@lukasnee lukasnee commented Mar 3, 2025

I found these bottlenecks while profiling my python application based on the min host using pyspeedscope. My application uploads firmware to a microcontroller and I was able to improve throughput by about 10% with these modifications.

Regarding commit perf(host): speed up bytes_to_hexstr up to 100x, here's quick benchmark test:

# test.py

import time

def bytes_to_hexstr_OLD(bytesequence: bytes) -> str:
    return "".join(f"{b:02X}" for b in bytesequence)

def bytes_to_hexstr_NEW(bytesequence: bytes) -> str:
    return bytesequence.hex().upper()

def timeit(f):
    start_time = time.time()
    f()
    end_time = time.time()
    return end_time - start_time

def test_case(bytesequence_size, run_count):
    input = bytes([i % 256 for i in range(bytesequence_size)])
    time1 = sum(timeit(lambda: bytes_to_hexstr_OLD(input))
                for _ in range(run_count)) / run_count
    time2 = sum(timeit(lambda: bytes_to_hexstr_NEW(input))
                for _ in range(run_count)) / run_count
    print(
        f"NEW vs. OLD {time1 / time2:.2f}x faster (bytesequence_size={bytesequence_size}, run_count={run_count})")

test_case(256, 1)
test_case(256, 10)
test_case(256, 100)
test_case(256, 1000)
test_case(256, 10000)
test_case(256*256, 1)
test_case(256*256, 10)
test_case(256*256, 100)
test_case(256*256, 1000)
python3 --version
Python 3.10.12
python3 test.py
NEW vs. OLD 4.44x faster (bytesequence_size=256, run_count=1)
NEW vs. OLD 63.05x faster (bytesequence_size=256, run_count=10)
NEW vs. OLD 60.77x faster (bytesequence_size=256, run_count=100)
NEW vs. OLD 77.76x faster (bytesequence_size=256, run_count=1000)
NEW vs. OLD 68.41x faster (bytesequence_size=256, run_count=10000)
NEW vs. OLD 127.16x faster (bytesequence_size=65536, run_count=1)
NEW vs. OLD 105.42x faster (bytesequence_size=65536, run_count=10)
NEW vs. OLD 122.96x faster (bytesequence_size=65536, run_count=100)
NEW vs. OLD 119.90x faster (bytesequence_size=65536, run_count=1000)

@lukasnee lukasnee changed the title Python host optimizations Python host optimizations (bytes_to_hexstr, log formatting, _now_ms) Mar 3, 2025
@lukasnee lukasnee force-pushed the python-host-optimizations branch from b834875 to 6f09169 Compare March 3, 2025 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant