Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 1 addition & 12 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,7 @@ NETWORK=
NETUID=
LOG_ID_PREFIX=

# Pyth data backend. `hermes` (default) keeps the legacy
# benchmarks.pyth.network + hermes.pyth.network endpoints. `pro` switches
# the validator to https://pyth.dourolabs.app/v1/fixed_rate@200ms/history
# (public; `fixed_rate@200ms` is the one channel that satisfies every feed's
# min_channel, including stocks/metals/oil which 404 on `real_time`)
# and the miner's get_asset_price to Pyth Lazer
# https://pyth-lazer.dourolabs.app/v1/latest_price (requires PYTH_API_KEY).
PYTH_BACKEND=hermes
# Miner-only. Bearer token from pythdata.app -> Pyth Terminal, used by
# the miner's get_asset_price live-price fetch via Pyth Lazer when
# PYTH_BACKEND=pro. Validators do NOT need this: on `pro` they read the
# public history endpoint above, which takes no auth.
# Pyth API key for Lazer (Pro) backend. If set, the miner will use Pyth Lazer instead of Hermes for assets with a Lazer feed.
PYTH_API_KEY=

# Bigtable storage backend for miner predictions. Only required when the
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ on:
jobs:
ci:
runs-on: ubuntu-latest
env:
PYTH_BACKEND: pro
steps:
- name: Checkout code
uses: actions/checkout@v5
Expand Down
5 changes: 4 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
repos:
- repo: https://github.com/psf/black
rev: 25.1.0
# Keep in sync with the black version resolved in uv.lock (and used by
# `uv run black` / CI). black formatting changes between releases, so a
# stale rev here reformats differently than CI and fights the pipeline.
rev: 26.5.1
hooks:
- id: black
language_version: python3.11
Expand Down
119 changes: 60 additions & 59 deletions README.md

Large diffs are not rendered by default.

136 changes: 0 additions & 136 deletions docs/validator_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,6 @@
- [5. Options](#5-options)
- [5.1. Common Options](#51-common-options)
- [`--axon.port INTEGER`](#--axonport-integer)
- [`--sma.low.days INTEGER`](#--smalowdays-integer)
- [`--sma.high.days INTEGER`](#--smahighdays-integer)
- [`--softmax.low.beta FLOAT`](#--softmaxlowbeta-float)
- [`--softmax.high.beta FLOAT`](#--softmaxhighbeta-float)
- [`--cycle_interval_minutes.low INTEGER`](#--cycle_interval_minuteslow-integer)
- [`--cycle_interval_minutes.high INTEGER`](#--cycle_interval_minuteshigh-integer)
- [`--logging.debug`](#--loggingdebug)
Expand Down Expand Up @@ -287,138 +283,6 @@ pm2 start validator.test.config.js -- --axon.port 8091

<sup>[Back to top ^][table-of-contents]</sup>

#### `--sma.low.days INTEGER`

The window for the simple moving average (SMA) of the validator scores for the 24h prompt.

Default: `10`

Example:

```js
// validator.config.js
module.exports = {
apps: [
{
name: "validator",
interpreter: "python3",
script: "./neurons/validator.py",
args: "--sma.low.days 10",
env: {
PYTHONPATH: ".",
},
},
],
};
```

Alternatively, you can add the args directly to the command:

```shell
pm2 start validator.config.js -- --sma.low.days 10
```

<sup>[Back to top ^][table-of-contents]</sup>

#### `--sma.high.days INTEGER`

The window for the simple moving average (SMA) of the validator scores for the 1h prompt.

Default: `3`

Example:

```js
// validator.config.js
module.exports = {
apps: [
{
name: "validator",
interpreter: "python3",
script: "./neurons/validator.py",
args: "--sma.high.days 3",
env: {
PYTHONPATH: ".",
},
},
],
};
```

Alternatively, you can add the args directly to the command:

```shell
pm2 start validator.config.js -- --sma.high.days 3
```

<sup>[Back to top ^][table-of-contents]</sup>

#### `--softmax.low.beta FLOAT`

Negative beta to give higher weight to lower scores for the 24h prompt

Default: `-0.1`

Example:

```js
// validator.config.js
module.exports = {
apps: [
{
name: "validator",
interpreter: "python3",
script: "./neurons/validator.py",
args: "--softmax.low.beta -0.1",
env: {
PYTHONPATH: ".",
},
},
],
};
```

Alternatively, you can add the args directly to the command:

```shell
pm2 start validator.config.js -- --softmax.low.beta -0.1
```

<sup>[Back to top ^][table-of-contents]</sup>

#### `--softmax.high.beta FLOAT`

Negative beta to give higher weight to lower scores for the 1h prompt

Default: `-0.2`

Example:

```js
// validator.config.js
module.exports = {
apps: [
{
name: "validator",
interpreter: "python3",
script: "./neurons/validator.py",
args: "--softmax.high.beta -0.2",
env: {
PYTHONPATH: ".",
},
},
],
};
```

Alternatively, you can add the args directly to the command:

```shell
pm2 start validator.config.js -- --softmax.high.beta -0.2
```

<sup>[Back to top ^][table-of-contents]</sup>

#### `--cycle_interval_minutes.low INTEGER`

Cycle interval in minutes for the 24h prompt.
Expand Down
20 changes: 0 additions & 20 deletions entrypoint-validator.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,6 @@ mainnet_netuid=50
netuid=${NETUID:-$mainnet_netuid}

vpermit_tao_limit=999999
default_softmax_beta_low=-0.4
softmax_beta_low="${SOFTMAX_BETA_LOW:-$default_softmax_beta_low}"
default_softmax_beta_high=-0.4
softmax_beta_high="${SOFTMAX_BETA_HIGH:-$default_softmax_beta_high}"
default_sma_days_low=10
sma_days_low="${SMA_DAYS_LOW:-$default_sma_days_low}"
default_sma_days_high=5
sma_days_high="${SMA_DAYS_HIGH:-$default_sma_days_high}"

default_validator_coldkey_name=validator
validator_coldkey_name="${VALIDATOR_COLDKEY_NAME:-$default_validator_coldkey_name}"
Expand All @@ -31,10 +23,6 @@ python3.11 ./neurons/validator.py \
--netuid $netuid \
--logging.debug \
--neuron.axon_off true \
--sma.low.days $sma_days_low \
--sma.high.days $sma_days_high \
--softmax.low.beta $softmax_beta_low \
--softmax.high.beta $softmax_beta_high \
--neuron.vpermit_tao_limit $vpermit_tao_limit \
--gcp.log_id_prefix $log_id_prefix \
--neuron.nprocs 8 \
Expand All @@ -48,10 +36,6 @@ python3.11 ./neurons/validator.py \
--netuid $netuid \
--logging.debug \
--neuron.axon_off true \
--sma.low.days $sma_days_low \
--sma.high.days $sma_days_high \
--softmax.low.beta $softmax_beta_low \
--softmax.high.beta $softmax_beta_high \
--neuron.vpermit_tao_limit $vpermit_tao_limit \
--gcp.log_id_prefix $log_id_prefix \
--neuron.nprocs 8 \
Expand All @@ -65,10 +49,6 @@ python3.11 ./neurons/validator.py \
--netuid $netuid \
--logging.debug \
--neuron.axon_off true \
--sma.low.days $sma_days_low \
--sma.high.days $sma_days_high \
--softmax.low.beta $softmax_beta_low \
--softmax.high.beta $softmax_beta_high \
--neuron.vpermit_tao_limit $vpermit_tao_limit \
--gcp.log_id_prefix $log_id_prefix \
--neuron.nprocs 8 \
Expand Down
2 changes: 1 addition & 1 deletion miner.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ module.exports = {
name: "miner",
interpreter: "python3",
script: "./neurons/miner.py",
args: "--netuid 50 --logging.debug --logging.trace --wallet.name miner --wallet.hotkey default --axon.port 8091 --blacklist.force_validator_permit true --blacklist.validator_min_stake 1000",
args: "--netuid 50 --logging.debug --logging.trace --wallet.name miner --wallet.hotkey default --axon.port 8091 --blacklist.force_validator_permit true --blacklist.validator_min_stake 65000",
Comment thread
Thykof marked this conversation as resolved.
Comment thread
Thykof marked this conversation as resolved.
env: {
PYTHONPATH: ".",
},
Expand Down
40 changes: 13 additions & 27 deletions neurons/validator.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
from synth.validator.miner_data_handler import MinerDataHandler
from synth.validator.price_data_provider import PriceDataProvider
from synth.validator.storage_backend import STORAGE_BACKEND_BIGTABLE
from synth.validator import competition_config
from synth.validator.prompt_config import (
PromptConfig,
LOW_FREQUENCY,
Expand Down Expand Up @@ -89,10 +90,6 @@ def __init__(self, config=None, cycle_name: str = CYCLE_LOW_FREQUENCY):
self.price_data_provider = PriceDataProvider()

self.miner_uids: list[int] = []
LOW_FREQUENCY.softmax_beta = self.config.softmax.low.beta
HIGH_FREQUENCY.softmax_beta = self.config.softmax.high.beta
LOW_FREQUENCY.window_days = self.config.sma.low.days
HIGH_FREQUENCY.window_days = self.config.sma.high.days
LOW_FREQUENCY.data_retention_days = self.config.retention.low.days
HIGH_FREQUENCY.data_retention_days = self.config.retention.high.days
LOW_FREQUENCY.cycle_interval_minutes = (
Expand Down Expand Up @@ -246,36 +243,25 @@ def forward_score(self):
# with predictions and calculate the rewards,
# we store the rewards in the miner_scores table
# ========================================== #
bt.logging.info(
f"forward score {LOW_FREQUENCY.label} frequency", "forward_score"
)
current_time = get_current_time()
scored_time: datetime = round_time_to_minutes(current_time)

success_low = calculate_scores(
self.miner_data_handler,
self.price_data_provider,
scored_time,
LOW_FREQUENCY,
self.config.neuron.nprocs,
)
success_count = 0
for comp in competition_config.ALL_COMPETITIONS:
bt.logging.info(f"forward score {comp.label}", "forward_score")

scored_time: datetime = round_time_to_minutes(current_time)
current_time = get_current_time()
bt.logging.info(
f"forward score {HIGH_FREQUENCY.label} frequency", "forward_score"
)
success_high = calculate_scores(
self.miner_data_handler,
self.price_data_provider,
scored_time,
HIGH_FREQUENCY,
self.config.neuron.nprocs,
)
if calculate_scores(
self.miner_data_handler,
self.price_data_provider,
scored_time,
comp,
self.config.neuron.nprocs,
):
success_count += 1

self.cleanup_history()

if not success_low and not success_high:
if success_count == 0:
return

# ================= Step 4 ================= #
Expand Down
36 changes: 11 additions & 25 deletions synth/miner/price_simulation.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,8 @@
)

# Hermes Pyth API documentation: https://hermes.pyth.network/docs/
# Pyth Lazer (the Pro replacement for latest_price) docs:
# https://docs.pyth.network/price-feeds/pro/api/rest#post-v1latest_price
# Selected by env: PYTH_BACKEND=pro (default `hermes`). When `pro`, requests
# go to Lazer with a Bearer PYTH_API_KEY. WTIOIL has no working Lazer feed
# today so we route it through Hyperliquid (same coin `xyz:CL` the validator
# uses for WTIOIL history).
# Pyth Lazer https://docs.pyth.network/price-feeds/pro/api/rest#post-v1latest_price
# If env: PYTH_API_KEY=<api key> is set, the miner will use Pyth Lazer instead of Hermes for assets with a Lazer feed.

TOKEN_MAP = {
"BTC": "e62df6c8b4a85fe1a67db44dc12de5db330f7ac66b72dc658afedf0f4a415b43",
Expand All @@ -35,10 +31,7 @@
}

# Lazer u32 feed IDs sourced from https://pyth.dourolabs.app/v1/symbols by
# matching each asset's hermes_id. Re-discover with
# verify/pyth-lazer-listing.py. WTIOIL has no active Lazer feed (the closest
# replacement USOILSPOT is inactive); it is intentionally absent and routed
# to Hyperliquid in the `pro` branch instead.
# matching each asset's hermes_id.
LAZER_FEED_ID_MAP: dict[str, int] = {
"BTC": 1,
"ETH": 2,
Expand All @@ -53,11 +46,10 @@
"TSLAX": 1847,
}

# Hyperliquid `coin` codes for assets that have no usable Pyth feed. Mirrors
# `PriceDataProvider.HYPERLIQUID_SYMBOL_MAP` on the validator side so the
# miner's spot price comes from the same source the validator scores against.
# Hyperliquid `coin` codes for assets that have no usable Pyth feed.
HYPERLIQUID_ASSET_MAP = {
"WTIOIL": "xyz:CL",
"SPCX": "xyz:SPCX",
}

# `fixed_rate@200ms` meets every feed's min_channel (crypto majors accept
Expand Down Expand Up @@ -88,12 +80,7 @@ def _fetch_price_hermes(asset: str) -> float | None:
return live_price


def _fetch_price_lazer(asset: str) -> float | None:
api_key = os.environ.get("PYTH_API_KEY")
if not api_key:
print("PYTH_API_KEY not set; required when PYTH_BACKEND=pro")
return None

def _fetch_price_lazer(asset: str, api_key: str) -> float | None:
payload = {
"channel": LAZER_CHANNEL,
"priceFeedIds": [LAZER_FEED_ID_MAP[asset]],
Expand Down Expand Up @@ -155,12 +142,11 @@ def _fetch_price_hyperliquid(asset: str) -> float | None:
reraise=True,
)
def get_asset_price(asset="BTC") -> float | None:
backend = os.environ.get("PYTH_BACKEND", "hermes").lower()
if backend == "pro":
if asset in HYPERLIQUID_ASSET_MAP:
return _fetch_price_hyperliquid(asset)
if asset in LAZER_FEED_ID_MAP:
return _fetch_price_lazer(asset)
if asset in HYPERLIQUID_ASSET_MAP:
return _fetch_price_hyperliquid(asset)
api_key = os.environ.get("PYTH_API_KEY")
if asset in LAZER_FEED_ID_MAP and api_key:
return _fetch_price_lazer(asset, api_key)
return _fetch_price_hermes(asset)


Expand Down
Loading
Loading