Entity miners Initial Infra

.gitignore

@@ -1,6 +1,7 @@
 # pickling and json files
 *.json
 *.pickle
+*.gz
 # model parameters
 !vali_objects/utils/model_parameters/all_model_parameters.json
 !vali_objects/utils/model_parameters/slippage_estimates.json

docs/running_signals_server.md

@@ -40,6 +40,169 @@ you can search for its PID and kill it with the following commands.
 `pkill -f run_receive_signals_server.sh` </br>
 `pkill -f run_receive_signals_server.py`
 
+## API Endpoint Documentation
+
+### Receive Signal
+
+`POST /api/receive-signal`
+
+This endpoint receives trading signals from external systems and stores them locally for the miner to process and send to validators.
+
+**Required Headers**:
+```
+Content-Type: application/json
+```
+
+**Request Body Fields**:
+
+#### Required Fields
+
+- `api_key` (string): Your API key as configured in `mining/miner_secrets.json`. Used for authentication.
+- `execution_type` (string): The execution type for the order. Must be one of:
+  - `"MARKET"`: Execute immediately at current market price
+  - `"LIMIT"`: Execute at a specific price when market reaches that level
+  - `"BRACKET"`: Limit order with attached stop-loss and/or take-profit orders
+  - `"LIMIT_CANCEL"`: Cancel an existing limit order
+- `trade_pair` (string or object): The trading pair for the order. Can be either:
+  - Trade pair ID string (e.g., `"BTCUSD"`, `"ETHUSD"`, `"EURUSD"`)
+  - Trade pair object with `trade_pair_id` field
+- `order_type` (string): The direction of the order. Must be one of:
+  - `"LONG"`: Open or increase a long position
+  - `"SHORT"`: Open or increase a short position
+  - `"FLAT"`: Close the current position
+
+#### Order Size (Exactly ONE Required)
+
+You must provide **exactly one** of the following fields to specify the order size:
+
+- `leverage` (float): The portfolio weight for the position (e.g., `0.1` for 10% weight)
+- `value` (float): The USD value of the order (e.g., `10000` for $10,000)
+- `quantity` (float): The quantity in base asset units (lots, shares, coins, etc.)
+
+#### Optional Fields for LIMIT and BRACKET Orders
+
+- `limit_price` (float): **Required for LIMIT/BRACKET orders**. The price at which the limit order should fill.
+- `stop_loss` (float): Optional for LIMIT orders. Creates a stop-loss bracket order upon fill.
+- `take_profit` (float): Optional for LIMIT orders. Creates a take-profit bracket order upon fill.
+
+#### Optional Fields for LIMIT_CANCEL Orders
+
+- `order_uuid` (string): **Required for LIMIT_CANCEL orders**. The UUID of the limit order to cancel.
+
+#### Optional Fields for Entity Miners
+
+- `subaccount_id` (integer): The subaccount ID for entity miners (e.g., `0`, `1`, `2`). Only applicable for registered entity miners with subaccounts. Regular miners should omit this field.
+
+**Example Requests**:
+
+#### Market Order (Standard Miner)
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "MARKET",
+  "trade_pair": "BTCUSD",
+  "order_type": "LONG",
+  "leverage": 0.1
+}
+```
+
+#### Limit Order with Brackets
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "BRACKET",
+  "trade_pair": "ETHUSD",
+  "order_type": "SHORT",
+  "leverage": 0.2,
+  "limit_price": 3500.00,
+  "stop_loss": 3600.00,
+  "take_profit": 3300.00
+}
+```
+
+#### Market Order with USD Value
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "MARKET",
+  "trade_pair": "EURUSD",
+  "order_type": "LONG",
+  "value": 10000
+}
+```
+
+#### Close Position (Flat Order)
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "MARKET",
+  "trade_pair": "BTCUSD",
+  "order_type": "FLAT",
+  "leverage": 0
+}
+```
+
+#### Cancel Limit Order
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "LIMIT_CANCEL",
+  "trade_pair": "BTCUSD",
+  "order_type": "FLAT",
+  "order_uuid": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+#### Entity Miner Subaccount Order
+```json
+{
+  "api_key": "your_api_key_here",
+  "execution_type": "MARKET",
+  "trade_pair": "BTCUSD",
+  "order_type": "LONG",
+  "leverage": 0.1,
+  "subaccount_id": 0
+}
+```
+
+**Response**:
+
+Success (200):
+```json
+{
+  "message": "Signal {'trade_pair': ..., 'order_type': 'LONG', ...} received successfully"
+}
+```
+
+Error (400):
+```json
+{
+  "error": "Error message describing the issue"
+}
+```
+
+Error (401):
+```json
+{
+  "error": "Invalid API key"
+}
+```
+
+**Supported Trade Pairs**:
+
+- **Crypto**: BTCUSD, ETHUSD, SOLUSD, XRPUSD, DOGEUSD, ADAUSD
+- **Forex**: EURUSD, GBPUSD, AUDUSD, USDCAD, USDCHF, NZDUSD, and other major currency pairs
+
+For the complete list of supported trade pairs and their current status, refer to `vali_objects/vali_config.py`.
+
+**Notes**:
+
+1. Orders are stored locally and processed by the miner in the order they are received
+2. The miner will send these orders to validators via the Bittensor network
+3. Only one order per trade pair is processed at a time; duplicate signals for the same trade pair will overwrite previous unprocessed signals
+4. For entity miners, the `subaccount_id` is used to construct a synthetic hotkey for position tracking
+5. Regular miners should omit the `subaccount_id` field entirely
+
 ## Testing sending a signal
 
 You can test a sample signal to ensure your server is running properly by running the

entitiy_management/README.txt

@@ -0,0 +1,55 @@
+propose a solution for a new feature "Entity miners"
+
+
+ One miner hotkey VANTA_ENTITY_HOTKEY will correspond to an entity.
+
+  We will track entities with an EntityManager which persists data to disk, offers getters and setters via a client,
+  and has a server class that delegates to a manager instance (just like challenge_period flow).
+
+  Each entity i,e VANTA_ENTITY_HOTKEY can have subaccounts (monotonically increasing id).
+  Subaccounts get their own synthetic hotkey which is f"{VANTA_ENTITY_HOTKEY}_{subaccount_id}"
+  If a subaccount gets eliminated, that id can never be assigned again. An entity can only have MAX_SUBACCOUNTS_PER_ENTITY
+  subaccounts at once. The limit is 500. Thus instead of tracking eliminated subaccount ids,
+  we can simply maintain the active subaccount ids as well as the next id to be assigned
+
+
+  We must support rest api requests of entity data using an EntityManagerClient in rest server.
+
+    1. `POST register_subaccount` → returns {success, subaccount_id, subaccount_uuid}
+        1. Verifies entity collateral and slot allowance
+    2. `GET subaccount_status/{subaccouunt_id}` → active/eliminated/unknown
+
+This is the approach we want to utilize for the subaccount registration process:
+VantaRestServer endpoint exposed which does the collateral operations (placeholder for now)
+ and then returns the newly-registered subaccount id to the caller.
+  The validator then send a synapse message to all other validators so they are synced with the new subaccount id.
+  Refer for the flow in broadcast_asset_selection_to_validators to see how we should do this.
+
+
+EntityManager (RPCServerBase) will have its own daemon that periodically assess elimination criteria for entitiy miners.
+Put a placeholder in this logic for now.
+
+
+Most endpoints in VantaRestServer will support subaccounts directly since the passed in hotkey can be synthetic and
+our existing code will be able to work with synthetic hotkeys as long as we adjust the metagraph logic to detect
+synthetic hotkeys (have an underscore) and then making the appropriate call to the EntityManagerClient to see if
+that subaccount is still active. and if the VANTA_ENTITY_HOTKEY hotkey is still in the raw metagraph. Our has_hotkey method
+with this update should allow to work smoothly but let me know if there are other key parts of our system that
+need to be updated to support synthetic hotkeys.
+
+
+1. The entity hotkey (VANTA_ENTITY_HOTKEY) cannot place orders itself. Only its subaccounts can. This will need
+to be enforced in validator.py.
+
+2. Account sizes for synthetic hotkeys is set to a fixed value using a ContractClient after a blackbox function
+transfers collateral from VANTA_ENTITY_HOTKEY. Leave placeholder functions for this. This account size init is done during
+the subaccount registration flow.
+
+3. debt based scoring will read debt ledgers for all miners including subaccounts. It needs to agrgeagte the debt
+ledgers for all subaccounts into a single debt ledger representing the sum of all subaccount performance.
+The key for this debt ledger will simply be the entity hotkey (VANTA_ENTITY_HOTKEY).
+
+4.  Sub-accounts challenge period is an instantaneous pass if they get 3% returns against 6% drawdown within 90 days. Just like how in mdd checker, we can get returns and drawdown in different intervals, we will implement this in our
+EntityManager daemon. A PerfLedgerClient is thus needed.
+
+- Each entity miner can host up to **500 active sub-accounts**