Skip to content

Commit e01a328

Browse files
authored
OSS-readiness: docs & config reference sync, dependency slimming (#29)
* chore: OSS-readiness pass (docs, config reference, deps) - pyproject: drop unused flask; move sentence-transformers to an [embeddings] extra (lazy-imported, only needed for local embedding backends); keep pymongo with a note (provides bson.ObjectId) - default_config.yaml: sync with config.py — add require_hmrd, literature review, steering brief, research director, strategy clustering, parent-selection shaping, re_evaluations, novelty - README: runnable quick-start snippet (asyncio.run), concrete first-run command, no-config defaults note, optional extras, new 'watching and steering a live run' section; point alphaevolve examples at configs/bench_alphaevolve.yaml - examples: README + config for noisy_optimization (re_evaluations); fix autocorrelation_C1 references to a non-existent config.yaml - experiments: mark as inherited upstream benchmarks; require ALGOTUNE_PATH/EVOLVEBENCH_PATH env vars (drop personal defaults) - CONTRIBUTING fork URL, scripts/requirements.txt (flask -> plotly), NOTICE year * fix: add scipy to core deps (strategy clustering kmeans2) scipy was only present transitively via sentence-transformers; moving that to the [embeddings] extra dropped it, breaking strategy_clusters._cluster and its tests. Declare it explicitly.
1 parent d05d221 commit e01a328

12 files changed

Lines changed: 209 additions & 15 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Thank you for your interest in contributing to KaiEvolve! This document provides
55
## Getting Started
66

77
1. Fork the repository
8-
2. Clone your fork: `git clone https://github.com/firstbatchxyz/kai-evolve.git`
8+
2. Clone your fork: `git clone https://github.com/YOUR-USERNAME/kai-evolve.git`
99
3. Install the package in development mode: `pip install -e ".[dev]"`
1010
4. Set up environment for testing:
1111
```bash
@@ -51,7 +51,7 @@ When developing features that interact with LLMs:
5151
python -m unittest discover tests
5252
```
5353
5. Commit your changes: `git commit -m "Add your descriptive commit message"`
54-
6. Push to your fork: `git push origin feature/your-feature-name`
54+
6. Push to your fork: `git push origin feat-your-feature-name`
5555
7. Submit a pull request to the main repository
5656

5757
## Adding Examples

‎NOTICE‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
KaiEvolve
2-
Copyright 2025 Dria
2+
Copyright 2025-2026 Dria
33

44
This product is a derivative work of OpenEvolve
55
(https://github.com/codelion/openevolve), Copyright 2025 Asankhaya Sharma

‎README.md‎

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,14 @@ cd kai-evolve
9898
pip install -e .
9999
```
100100

101+
Optional extras:
102+
```bash
103+
pip install -e ".[viewer]" # the `kai viewer` web UI (FastAPI + Jinja)
104+
pip install -e ".[embeddings]" # local embedding models for novelty detection /
105+
# strategy clustering (pulls in PyTorch; the
106+
# default "api" embedding backend needs no extra)
107+
```
108+
101109
### Quick Start
102110

103111
#### Setting up LLM Access
@@ -128,7 +136,9 @@ KaiEvolve uses the OpenAI SDK, which means it works with any LLM provider that s
128136
This setup ensures KaiEvolve can work with any LLM provider - OpenAI, Anthropic, Google, Cohere, local models via Ollama/vLLM, or any OpenAI-compatible endpoint.
129137

130138
```python
139+
import asyncio
131140
import os
141+
132142
from kaievolve import KaiEvolve
133143
134144
# Ensure API key is set
@@ -143,8 +153,8 @@ evolve = KaiEvolve(
143153
)
144154
145155
# Run the evolution
146-
best_program = await evolve.run(iterations=1000)
147-
print(f"Best program metrics:")
156+
best_program = asyncio.run(evolve.run(iterations=1000))
157+
print("Best program metrics:")
148158
for name, value in best_program.metrics.items():
149159
print(f" {name}: {value:.4f}")
150160
```
@@ -157,6 +167,22 @@ KaiEvolve can also be run from the command line:
157167
python kaievolve-run.py path/to/initial_program.py path/to/evaluator.py --config path/to/config.yaml --iterations 1000
158168
```
159169

170+
For a concrete first run, try the bundled circle packing example:
171+
172+
```bash
173+
pip install -r examples/circle_packing/requirements.txt # scipy + matplotlib
174+
python kaievolve-run.py examples/circle_packing/initial_program.py \
175+
examples/circle_packing/evaluator.py \
176+
--config examples/circle_packing/config_phase_1.yaml \
177+
--iterations 50
178+
```
179+
180+
> **Note**: if you omit `--config`, KaiEvolve warns and falls back to built-in
181+
> defaults: models `gpt-4o-mini`/`gpt-4o`, with the API base taken from the
182+
> `OPENAI_API_BASE` environment variable (default:
183+
> `https://openrouter.ai/api/v1`). Those model names only resolve if your
184+
> endpoint actually serves them, so passing an explicit config is recommended.
185+
160186
### Resuming from Checkpoints
161187

162188
KaiEvolve automatically saves checkpoints at intervals specified by the `checkpoint_interval` config parameter (default is 10 iterations). You can resume an evolution run from a saved checkpoint:
@@ -279,6 +305,30 @@ the solution view. See [`skills/visualization/SKILL.md`](skills/visualization/SK
279305
and the two reference visualizers under `examples/alphaevolve/` (circle packing and
280306
the autocorrelation step function).
281307

308+
### Watching and steering a live run
309+
310+
Active runs append a per-iteration `progress.jsonl` feed to their output
311+
directory; `kai monitor` and the web viewer read it, so both update in real
312+
time while a run is going (and keep working on finished runs).
313+
314+
Two steering channels can redirect a run **while it is running**:
315+
316+
- **Human steering brief** — point `prompt.steering_brief_path` at a markdown
317+
file and edit it mid-run: its contents are re-read every iteration and
318+
injected into generation prompts, so directives like "focus on the inner
319+
loop" or "avoid approach X" take effect without restarting or touching
320+
config. `kai steer` sets, appends to, or shows that file from the terminal.
321+
- **Research director** (opt-in) — an automated meta-agent that periodically
322+
reads the population and writes a strategic directive into the same steering
323+
channel. Enable it with `prompt.research_director_enabled: true`; optionally
324+
set `research_director_interval` (defaults to the migration interval) and
325+
`research_director_model` for a dedicated reasoning model (defaults to the
326+
run's model roster). See
327+
[`skills/research-director/SKILL.md`](skills/research-director/SKILL.md).
328+
329+
Both channels compose: the human brief and the director's directive are
330+
injected together.
331+
282332
## Configuration
283333

284334
KaiEvolve is highly configurable with advanced options:
@@ -500,7 +550,7 @@ Below is the optimal packing found by KaiEvolve after 800 iterations:
500550
### AlphaEvolve Benchmark Tasks
501551

502552
#### [examples/alphaevolve/](examples/alphaevolve/)
503-
Five open mathematical problems from the AlphaEvolve paper (autocorrelation_C1, packing_circles_max_sum_of_radii, no_isosceles_triangles, happy_ending, unit_distances), each packaged as an initial program plus evaluator plus config ready to evolve.
553+
Five open mathematical problems from the AlphaEvolve paper (autocorrelation_C1, packing_circles_max_sum_of_radii, no_isosceles_triangles, happy_ending, unit_distances), each packaged as an initial program plus evaluator. Run them with the suite config tuned for these tasks, [`configs/bench_alphaevolve.yaml`](configs/bench_alphaevolve.yaml).
504554

505555

506556

‎configs/default_config.yaml‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,10 +71,43 @@ prompt:
7171
# Feature extraction and program labeling thresholds
7272
# These control how the LLM perceives and categorizes programs
7373
suggest_simplification_after_chars: 500 # Suggest simplifying if program exceeds this many characters
74-
include_changes_under_chars: 100 # Include change descriptions in features if under this length
74+
include_changes_under_chars: 100 # Include change descriptions in features if under this length
7575
concise_implementation_max_lines: 10 # Label as "concise" if program has this many lines or fewer
7676
comprehensive_implementation_min_lines: 50 # Label as "comprehensive" if program has this many lines or more
7777

78+
# HMRD (Hypothesis/Method/Result/Discussion) program summaries
79+
require_hmrd: true # Prompts ask for HYPOTHESIS+METHOD tags; workers parse them and
80+
# attach the 4-section summary to each successful program
81+
82+
# Literature review: a bounded, running "lab notebook" distilled from the HMRD
83+
# summaries of prior programs, injected into every generation prompt
84+
literature_review_enabled: false # Enable the literature review channel
85+
literature_review_path: null # Review file path (null = <output_dir>/literature_review.md)
86+
literature_review_max_chars: 4000 # Bound on review size (~1000 tokens)
87+
literature_review_interval: null # Distill every N iterations (null = checkpoint_interval)
88+
89+
# Human steering brief: a markdown file you edit while a run is in progress;
90+
# it is re-read every iteration and injected into prompts (see `kai steer`)
91+
steering_brief_path: null # Path to the brief file (null = disabled)
92+
93+
# Research director: an automated meta-agent that periodically reads the
94+
# population and writes a strategic directive into the steering channel
95+
# (see skills/research-director/SKILL.md)
96+
research_director_enabled: false # Enable the research director (opt-in)
97+
research_director_interval: null # Fire every N iterations (null = database.migration_interval)
98+
research_director_model: null # Dedicated model for the director (null = run roster)
99+
100+
# Strategy clustering + cluster bandit: clusters programs by the embedding of
101+
# their HMRD summary into emergent strategies, then biases parent selection
102+
# toward promising/under-explored clusters
103+
strategy_clustering_enabled: false # Enable strategy clustering (opt-in)
104+
strategy_num_clusters: 6 # Number of strategy clusters
105+
strategy_epsilon: 0.05 # Selection-probability floor per cluster
106+
strategy_exploration_c: 0.5 # UCB exploration coefficient
107+
strategy_recency_bonus: 0.1 # Additive bonus for recently-improving clusters
108+
strategy_recency_window: 2 # Checkpoints counted as "recent"
109+
strategy_embedding_backend: "local" # "local" (needs `pip install kai-evolve[embeddings]`) or "api"
110+
78111
# Note: meta-prompting features are not yet implemented
79112

80113
# Database configuration
@@ -101,6 +134,12 @@ database:
101134
exploitation_ratio: 0.7 # Ratio of exploitation vs random selection
102135
# Note: diversity_metric is fixed to "edit_distance" (feature_based not implemented)
103136

137+
# Parent-selection shaping: parents are weighted by a robustly-scaled fitness
138+
# and down-weighted by how many children they already produced, so the search
139+
# neither collapses to uniform nor fixates on one heavily-mined lineage
140+
parent_selection_lambda: 2.0 # Sigmoid steepness over normalized score
141+
parent_selection_offspring_penalty: 0.5 # 0 disables the over-mining penalty
142+
104143
# Feature map dimensions for MAP-Elites
105144
# Default if not specified: ["complexity", "diversity"]
106145
#
@@ -135,6 +174,12 @@ evaluator:
135174
timeout: 300 # Maximum evaluation time in seconds
136175
max_retries: 3 # Maximum number of retries for evaluation
137176

177+
# Noise-aware fitness: for STOCHASTIC evaluators, a single evaluation is a
178+
# noisy sample of a program's true quality. When > 1, each program is
179+
# evaluated this many times and the numeric metrics are averaged (at N x the
180+
# evaluation cost). Leave at 1 for deterministic evaluators.
181+
re_evaluations: 1 # Evaluations per program, averaged
182+
138183
# Note: resource limits (memory_limit_mb, cpu_limit) are not yet implemented
139184

140185
# Evaluation strategies
@@ -151,3 +196,15 @@ evaluator:
151196
# LLM-based feedback (experimental)
152197
use_llm_feedback: false # Use LLM to evaluate code quality
153198
llm_feedback_weight: 0.1 # Weight for LLM feedback in final score
199+
200+
# Novelty detection (optional): rejects near-duplicate programs by embedding
201+
# similarity and asks the LLM to regenerate, keeping the population diverse
202+
novelty:
203+
enabled: false # Enable novelty detection
204+
embedding_backend: "local" # "local" (sentence-transformers, needs
205+
# `pip install kai-evolve[embeddings]`) or "api"
206+
embedding_model: "all-MiniLM-L6-v2" # "api" backend: use e.g. "openai/text-embedding-3-small"
207+
embedding_device: null # Device for local model: "cpu", "cuda", "mps" (null = auto)
208+
similarity_threshold: 0.95 # Reject programs with similarity >= this
209+
max_regeneration_attempts: 3 # Regeneration attempts for a novel program
210+
temperature_increment: 0.15 # Temperature bump per regeneration attempt

‎examples/alphaevolve/autocorrelation_C1/README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,15 +53,19 @@ So C=2.0 → 0, C=1.5 → 0.5, C=1.0 → 1.0. The target band is roughly score
5353

5454
- `initial_program.py` — starting point: simple random hill-climber on a uniform sequence
5555
- `evaluator.py` — canonical scorer; returns `combined_score`, `raw_C`, `sequence_length`, `runs_successfully`
56-
- `config.yaml` — 30 iterations, 2 islands × 2 models, diff mode
56+
- `visualize.py` — interactive solution view for the web viewer
5757
- `README.md` — this file
5858

59+
Use the suite config tuned for the AlphaEvolve tasks,
60+
[`configs/bench_alphaevolve.yaml`](../../../configs/bench_alphaevolve.yaml)
61+
(diff mode, 2 islands × 2 models).
62+
5963
## Running
6064

6165
```bash
6266
OPENAI_API_KEY=sk-or-... python scripts/bench.py run \
6367
--tasks examples/alphaevolve/autocorrelation_C1 \
64-
--config examples/alphaevolve/autocorrelation_C1/config.yaml \
68+
--config configs/bench_alphaevolve.yaml \
6569
--output-dir bench_results/autocorrelation_C1 \
6670
--runs 3 --label autocorr
6771
```
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Noisy optimization (stochastic evaluator)
2+
3+
A minimal benchmark for **noise-aware fitness**: the candidate's `construct()`
4+
returns an 8-number vector whose true quality is a deterministic concave
5+
function of its distance to a hidden target — but `evaluate()` reports that
6+
quality with fresh additive Gaussian noise on every call. Repeated evaluations
7+
of the *same* program differ, so selection can be fooled by a single lucky
8+
draw. This is exactly the regime `evaluator.re_evaluations` is for: when set
9+
above 1, each program is evaluated that many times and the metrics are
10+
averaged, shrinking the variance of the fitness estimate.
11+
12+
The evaluator runs `construct()` in a clean subprocess so candidates cannot
13+
introspect the evaluator's memory and read the hidden target — they have to
14+
actually optimize.
15+
16+
## Files
17+
18+
- `initial_program.py` — starting point: the all-zeros vector
19+
- `evaluator.py` — stochastic scorer (`combined_score` = true quality + noise);
20+
also exposes `true_evaluate()` so benchmarks can grade results noiselessly
21+
- `config.yaml` — small run with `re_evaluations: 3`
22+
23+
## Running
24+
25+
```bash
26+
python kaievolve-run.py examples/noisy_optimization/initial_program.py \
27+
examples/noisy_optimization/evaluator.py \
28+
--config examples/noisy_optimization/config.yaml
29+
```
30+
31+
Compare with `re_evaluations: 1` in the config to see how single noisy samples
32+
mislead selection.
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Noisy-optimization benchmark: stochastic evaluator (fresh Gaussian noise on
2+
# every call). re_evaluations > 1 averages repeated evaluations per program so
3+
# selection is not fooled by lucky draws.
4+
5+
max_iterations: 30
6+
checkpoint_interval: 10
7+
log_level: "INFO"
8+
random_seed: 42
9+
10+
diff_based_evolution: true
11+
12+
llm:
13+
api_base: "https://openrouter.ai/api/v1"
14+
models:
15+
- name: "google/gemini-3.1-flash-lite"
16+
weight: 1.0
17+
- name: "deepseek/deepseek-v4-flash"
18+
weight: 1.0
19+
temperature: 0.7
20+
timeout: 120
21+
retries: 2
22+
23+
database:
24+
population_size: 40
25+
archive_size: 20
26+
num_islands: 2
27+
28+
evaluator:
29+
timeout: 30
30+
cascade_evaluation: false
31+
re_evaluations: 3 # average 3 noisy samples per program (the point of this example)
32+
parallel_evaluations: 2
33+
use_llm_feedback: false

‎experiments/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# KaiEvolve AlgoTune Benchmarking System
22

3+
> **Note**: this directory is benchmarking infrastructure and result data
4+
> inherited from upstream [OpenEvolve](https://github.com/codelion/openevolve);
5+
> the recorded results predate the KaiEvolve fork. The scripts expect the
6+
> `ALGOTUNE_PATH` / `EVOLVEBENCH_PATH` environment variables to point at your
7+
> local checkouts.
8+
39
This directory contains the benchmarking infrastructure for running KaiEvolve on the AlgoTune dataset.
410

511
## Directory Structure

‎experiments/evolvebench/run_evolvebench.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,10 @@
1919
import time
2020
from pathlib import Path
2121

22-
# EvolveBench path (adjust if needed)
23-
EVOLVEBENCH_PATH = Path("/Users/codelion/Documents/GitLab/evolve-bench")
22+
# EvolveBench path
23+
if not os.environ.get("EVOLVEBENCH_PATH"):
24+
sys.exit("Set the EVOLVEBENCH_PATH environment variable to your evolve-bench checkout")
25+
EVOLVEBENCH_PATH = Path(os.environ["EVOLVEBENCH_PATH"])
2426
KAIEVOLVE_PATH = Path(__file__).parent.parent.parent
2527
CONFIGS_DIR = Path(__file__).parent / "configs"
2628
RESULTS_DIR = Path(__file__).parent / "results"

‎experiments/validate_initial_programs.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,9 @@
1717
from typing import Dict, List, Any, Tuple
1818

1919
# Add AlgoTune to Python path
20-
ALGOTUNE_PATH = os.environ.get('ALGOTUNE_PATH', '/Users/codelion/Documents/GitHub/AlgoTune')
20+
ALGOTUNE_PATH = os.environ.get('ALGOTUNE_PATH')
21+
if not ALGOTUNE_PATH:
22+
sys.exit("Set the ALGOTUNE_PATH environment variable to your AlgoTune checkout")
2123
sys.path.insert(0, ALGOTUNE_PATH)
2224

2325
from AlgoTuneTasks.factory import TaskFactory

0 commit comments

Comments
 (0)