Skip to content

Commit f79a8e5

Browse files
committed
don't mention poetry in doc
1 parent a39137e commit f79a8e5

File tree

2 files changed

+26
-22
lines changed

2 files changed

+26
-22
lines changed

src/aks-agent/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Ignore Poetry artifacts
2+
poetry.lock
3+
pyproject.toml

src/aks-agent/azext_aks_agent/tests/evals/README.md

Lines changed: 23 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,22 @@
11
# AKS Agent Evals
22

3+
## Environment Setup
4+
5+
Create and activate a virtual environment (example shown for bash-compatible shells):
6+
7+
```bash
8+
python -m venv .venv
9+
source .venv/bin/activate
10+
python -m pip install --upgrade pip
11+
python -m pip install -e .
12+
```
13+
14+
Optional tooling used by the eval harness (Braintrust uploads and semantic classifier helpers):
15+
16+
```bash
17+
python -m pip install braintrust openai autoevals
18+
```
19+
320
## Running Live Scenarios
421

522
```bash
@@ -12,7 +29,7 @@ KUBECONFIG=<path-to-kubeconfig> \
1229
AZURE_API_KEY=<key> \
1330
AZURE_API_BASE=<endpoint> \
1431
AZURE_API_VERSION=<version> \
15-
poetry run pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 01_list_all_nodes -m aks_eval
32+
python -m pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 01_list_all_nodes -m aks_eval
1633
```
1734

1835
Per-scenario overrides (`resource_group`, `cluster_name`, `kubeconfig`, `test_env_vars`) still apply. Use `--skip-setup` or `--skip-cleanup` to bypass hooks. Expect the test to log iteration progress, classifier scores, and (on the final iteration) a Braintrust link when uploads are enabled.
@@ -43,10 +60,10 @@ KUBECONFIG=<path-to-kubeconfig> \
4360
AZURE_API_KEY=<key> \
4461
AZURE_API_BASE=<endpoint> \
4562
AZURE_API_VERSION=<version> \
46-
poetry run pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 02_list_clusters -m aks_eval
63+
python -m pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 02_list_clusters -m aks_eval
4764

4865
# Re-run offline using the recorded response
49-
poetry run pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 02_list_clusters -m aks_eval
66+
python -m pytest azext_aks_agent/tests/evals/test_ask_agent.py -k 02_list_clusters -m aks_eval
5067
```
5168

5269
If a mock is missing, pytest skips the scenario with instructions to regenerate it.
@@ -72,18 +89,6 @@ Set the following environment variables to push results:
7289
- `BRAINTRUST_API_KEY` and `BRAINTRUST_ORG` (required)
7390
- Optional overrides: `BRAINTRUST_PROJECT` (default `aks-agent`), `BRAINTRUST_DATASET` (default `aks-agent/ask`), `EXPERIMENT_ID`
7491

75-
Install the Braintrust client once networking is available:
76-
77-
```bash
78-
poetry add braintrust --group dev
79-
```
80-
81-
For a one-shot setup that installs all dev dependencies (Braintrust, openai, autoevals, pytest plugins), run:
82-
83-
```bash
84-
poetry install --with dev
85-
```
86-
8792
Each iteration logs to Braintrust; the console prints a clickable link (for aware terminals) when uploads succeed.
8893

8994
**Tips**
@@ -96,11 +101,7 @@ Each iteration logs to Braintrust; the console prints a clickable link (for awar
96101

97102
- Enabled by default; set `ENABLE_CLASSIFIER=false` to opt out.
98103
- Requires Azure OpenAI credentials: `AZURE_API_BASE`, `AZURE_API_KEY`, `AZURE_API_VERSION`, and a classifier deployment specified via `CLASSIFIER_MODEL` (e.g. `azure/<deployment>`). Defaults to the same deployment as `MODEL` when not provided.
99-
- Install classifier dependencies when online:
100-
101-
```bash
102-
poetry add openai autoevals --group dev
103-
```
104+
- Install classifier dependencies when online (see Environment Setup above if not already installed).
104105

105106
- Scenarios can override the grading style by adding:
106107

@@ -115,7 +116,7 @@ Classifier scores and rationales are attached to Braintrust uploads and printed
115116
**Debugging classifiers**
116117
117118
```
118-
poetry run pytest ... -o log_cli=true -o log_cli_level=DEBUG -s
119+
python -m pytest ... -o log_cli=true -o log_cli_level=DEBUG -s
119120
```
120121

121122
Look for `classifier score ...` lines to confirm the semantic judge executed.
@@ -134,7 +135,7 @@ Look for `classifier score ...` lines to confirm the semantic judge executed.
134135

135136
## Quick Checklist
136137

137-
- `poetry install --with dev` to ensure Braintrust + classifier dependencies are present.
138+
- Install dependencies inside a virtual environment (`python -m pip install -e .`) and, if needed, the optional tooling (`python -m pip install braintrust openai autoevals`).
138139
- `RUN_LIVE=true`: set Azure creds (`AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION`), `MODEL`, kubeconfig, and optional Braintrust vars.
139140
- `RUN_LIVE` unset/false: ensure each scenario directory has `mocks/response.txt`.
140141
- Classifier overrides: `CLASSIFIER_MODEL` (defaults to `MODEL`) and per-scenario `evaluation.correctness.type`.

0 commit comments

Comments
 (0)