Skip to content

Commit

Permalink
added specs
Browse files Browse the repository at this point in the history
  • Loading branch information
justrach committed Jan 10, 2025
1 parent 15ea1bf commit 232ab80
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 32 deletions.
78 changes: 46 additions & 32 deletions BENCHMARK.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@

# Kew Performance Benchmark Results


## System Information

| Component | Details |
|-----------|---------|
| Operating System | Darwin Darwin Kernel Version 24.1.0: Thu Nov 14 18:15:21 PST 2024; root:xnu-11215.41.3~13/RELEASE_ARM64_T6041 |
| Python Version | 3.12.5 |
| Processor | arm |
| CPU Cores | 14 |
| CPU Frequency | 4.00MHz |
| Total Memory | 48.0GB |
| Available Memory | 16.0GB |


## System Performance

The following benchmarks were run with:
Expand All @@ -9,46 +23,46 @@ The following benchmarks were run with:
- Multiple computational workloads

### Overall Performance
- Total Duration: 22.04 seconds
- Overall Throughput: 4.54 tasks/second
- Total Duration: 22.30 seconds
- Overall Throughput: 4.48 tasks/second

### Task-Specific Performance

### Large Matrix Multiplication (1000x1000)
| Metric | Value |
|--------|-------|
| Average Latency | 17.815s |
| Median Latency | 17.849s |
| 95th Percentile | 18.208s |
| 99th Percentile | 18.209s |
| Throughput | 1.13 tasks/second |
| Average Latency | 17.681s |
| Median Latency | 17.696s |
| 95th Percentile | 18.040s |
| 99th Percentile | 18.049s |
| Throughput | 1.12 tasks/second |

### Heavy Prime Calculation (up to 10M)
| Metric | Value |
|--------|-------|
| Average Latency | 18.581s |
| Median Latency | 18.555s |
| 95th Percentile | 18.863s |
| 99th Percentile | 18.863s |
| Throughput | 1.13 tasks/second |
| Average Latency | 18.409s |
| Median Latency | 18.390s |
| 95th Percentile | 18.734s |
| 99th Percentile | 18.743s |
| Throughput | 1.12 tasks/second |

### Complex FFT (10M points)
| Metric | Value |
|--------|-------|
| Average Latency | 16.033s |
| Median Latency | 15.951s |
| 95th Percentile | 18.436s |
| 99th Percentile | 18.979s |
| Throughput | 1.13 tasks/second |
| Average Latency | 16.209s |
| Median Latency | 15.958s |
| 95th Percentile | 18.852s |
| 99th Percentile | 18.861s |
| Throughput | 1.12 tasks/second |

### Distributed Data Processing
| Metric | Value |
|--------|-------|
| Average Latency | 8.800s |
| Median Latency | 8.981s |
| 95th Percentile | 11.974s |
| 99th Percentile | 12.649s |
| Throughput | 1.13 tasks/second |
| Average Latency | 9.173s |
| Median Latency | 9.349s |
| 95th Percentile | 12.984s |
| 99th Percentile | 12.985s |
| Throughput | 1.12 tasks/second |


## Load Distribution
Expand All @@ -61,32 +75,32 @@ Testing load distribution across multiple queues with:
### Queue Performance

#### Queue: load_test_queue_0
- Tasks processed: 78
- Average execution time: 0.384s
- Tasks processed: 73
- Average execution time: 0.378s

#### Queue: load_test_queue_1
- Tasks processed: 74
- Average execution time: 0.388s
- Tasks processed: 69
- Average execution time: 0.382s

#### Queue: load_test_queue_2
- Tasks processed: 74
- Average execution time: 0.388s
- Tasks processed: 66
- Average execution time: 0.389s


### Load Balance Score: 0.004s
### Load Balance Score: 0.011s
(Lower score indicates better load distribution)

---
*Benchmark run on 2025-01-10 17:16:56*
*Benchmark run on 2025-01-10 17:19:50*

## Extended Performance Comparison

### 1. Heavy Workload Performance

| Queue System | Total Duration (s) | Throughput (tasks/s) | Recovery from Failures | Data Persistence |
|--------------|-------------------|---------------------|---------------------|------------------|
| Kew | 22.04 | 4.54 | Yes | Yes |
| asyncio.Queue | 42.74 | 4.68 | No | No |
| Kew | 22.30 | 4.48 | Yes | Yes |
| asyncio.Queue | 43.71 | 4.58 | No | No |
| multiprocessing.Queue | ERROR | ERROR | Limited | No |

### Error Notes
Expand Down
39 changes: 39 additions & 0 deletions kew_benchmark_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
from asyncio import Queue as AsyncQueue
from concurrent.futures import ProcessPoolExecutor
import os
import platform
import psutil
import multiprocessing

async def matrix_multiplication(size: int = 100) -> dict:
"""Perform matrix multiplication - O(n³) complexity"""
Expand Down Expand Up @@ -227,7 +230,41 @@ async def dummy_task(task_num: int, sleep_time: float = 0.1) -> dict:
await asyncio.sleep(sleep_time)
return {"task_num": task_num, "processed_at": time.time()}

def get_system_info():
"""Get system information for benchmarking context"""
memory = psutil.virtual_memory()
cpu_freq = psutil.cpu_freq()

return {
'os': platform.system(),
'os_version': platform.version(),
'python_version': platform.python_version(),
'processor': platform.processor(),
'cpu_cores': multiprocessing.cpu_count(),
'cpu_freq': f"{cpu_freq.current:.2f}MHz" if cpu_freq else "Unknown",
'memory_total': f"{memory.total / (1024**3):.1f}GB",
'memory_available': f"{memory.available / (1024**3):.1f}GB",
}

def generate_markdown_report(perf_results, load_results):
# Get system information
sys_info = get_system_info()

# Add system info section to the markdown
system_info_md = f"""
## System Information
| Component | Details |
|-----------|---------|
| Operating System | {sys_info['os']} {sys_info['os_version']} |
| Python Version | {sys_info['python_version']} |
| Processor | {sys_info['processor']} |
| CPU Cores | {sys_info['cpu_cores']} |
| CPU Frequency | {sys_info['cpu_freq']} |
| Total Memory | {sys_info['memory_total']} |
| Available Memory | {sys_info['memory_available']} |
"""

task_metrics_md = "\n".join([
f"### {task_name}\n"
f"| Metric | Value |\n"
Expand All @@ -243,6 +280,8 @@ def generate_markdown_report(perf_results, load_results):
markdown = f"""
# Kew Performance Benchmark Results
{system_info_md}
## System Performance
The following benchmarks were run with:
Expand Down

0 comments on commit 232ab80

Please sign in to comment.