From 7341b750d7608aaed89d03ee2b4117887490f316 Mon Sep 17 00:00:00 2001 From: Bryan Chen Date: Thu, 7 Aug 2025 16:33:06 +1200 Subject: [PATCH 1/4] update readme --- README.md | 251 ++++++++++- poc/runtime/README.md | 261 +++++++++++ pvq-executor/README.md | 338 ++++++++++++++ pvq-extension-core/README.md | 384 ++++++++++++++++ pvq-extension-fungibles/README.md | 513 ++++++++++++++++++++++ pvq-extension-swap/README.md | 589 +++++++++++++++++++++++++ pvq-extension/README.md | 552 +++++++++++++++++++++++ pvq-primitives/README.md | 703 ++++++++++++++++++++++++++++++ pvq-program/README.md | 675 ++++++++++++++++++++++++++++ pvq-runtime-api/README.md | 648 +++++++++++++++++++++++++++ pvq-test-runner/README.md | 675 ++++++++++++++++++++++++++++ 11 files changed, 5571 insertions(+), 18 deletions(-) create mode 100644 poc/runtime/README.md create mode 100644 pvq-executor/README.md create mode 100644 pvq-extension-core/README.md create mode 100644 pvq-extension-fungibles/README.md create mode 100644 pvq-extension-swap/README.md create mode 100644 pvq-extension/README.md create mode 100644 pvq-primitives/README.md create mode 100644 pvq-program/README.md create mode 100644 pvq-runtime-api/README.md create mode 100644 pvq-test-runner/README.md diff --git a/README.md b/README.md index fe47c56..345005e 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,245 @@ -# PVQ +# PVQ: PolkaVM Query System for Polkadot -PolkaVM Query for Polkadot + -## Getting Started +[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) +[![Rust](https://github.com/open-web3-stack/PVQ/workflows/Rust/badge.svg)](https://github.com/open-web3-stack/PVQ/actions) + +A powerful and secure query system for Polkadot parachains that enables developers to write expressive, sandboxed guest programs using PolkaVM. PVQ provides a safe and efficient alternative to custom RPC endpoints for runtime data queries. + +## ✨ Features + +- **πŸ”’ Secure Execution**: Sandboxed PolkaVM environment for safe query execution +- **🧩 Modular Extensions**: Extensible system for exposing runtime functionalities +- **⚑ High Performance**: Efficient RISC-V execution with minimal overhead +- **πŸ› οΈ Developer Friendly**: Rust-first development experience with procedural macros +- **🌐 Runtime Integration**: Seamless integration with Substrate runtimes +- **πŸ” Rich Querying**: Support for complex queries involving multiple runtime components + +## πŸ—οΈ Architecture + +The PVQ system consists of several interconnected components: + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Program │───▢│ PVQ Executor │───▢│ Substrate β”‚ +β”‚ (Guest Code) β”‚ β”‚ (Host Side) β”‚ β”‚ Runtime β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ β”‚ β”‚ + β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ + └─────────────▢│ PVQ Extensions β”‚β—€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ (Modules) β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +### Core Components + +| Component | Description | +|-----------|-------------| +| **[PVQ Program](pvq-program/)** | Guest programs written in Rust that compile to RISC-V | +| **[PVQ Executor](pvq-executor/)** | Host-side component managing PolkaVM instances and runtime interaction | +| **[PVQ Extensions](pvq-extension/)** | Modular system exposing runtime functionalities to guest programs | +| **[PVQ Runtime API](pvq-runtime-api/)** | Substrate runtime API for external query submission | +| **[PVQ Primitives](pvq-primitives/)** | Common types and utilities shared across components | + +### Available Extensions + +- **[Core Extension](pvq-extension-core/)**: Fundamental functionalities and extension discovery +- **[Fungibles Extension](pvq-extension-fungibles/)**: Asset querying, balances, and metadata +- **[Swap Extension](pvq-extension-swap/)**: DEX interactions, liquidity pools, and price quotes + +## πŸš€ Getting Started ### Prerequisites -- Pull vendored `polkavm`: `git submodule update --init --recursive`. -- Install `polkatool` (for relinking the standard RV32E ELF to a PolkaVM blob) and `chain-spec-builder` (for building chainspec from a wasm): `make tools` +Ensure you have the following installed: + +- **Rust** (latest stable version) +- **Git** with submodule support + +### Installation + +1. **Clone the repository with submodules:** + ```bash + git clone --recursive https://github.com/open-web3-stack/PVQ.git + cd PVQ + ``` + +2. **Install required tools:** + ```bash + make tools + ``` + This installs `polkatool` for ELF to PolkaVM blob conversion and `chain-spec-builder`. + +3. **Build the project:** + ```bash + cargo build --release + ``` + +### Quick Start + +#### Running Example Programs + +1. **Build guest programs:** + ```bash + make guests + ``` + +2. **Run a test program:** + ```bash + cargo run -p pvq-test-runner -- --program output/guest-sum-balance + ``` + +#### Available Example Programs + +| Program | Description | +|---------|-------------| +| `guest-sum-balance` | Sum balances of multiple accounts | +| `guest-total-supply` | Get total supply of an asset | +| `guest-sum-balance-percent` | Calculate percentage of total supply for account balances | +| `guest-swap-info` | Query DEX/swap information | + +### Runtime Integration + +#### Testing with PoC Runtime + +1. **Start local test chain:** + ```bash + make run + ``` + +2. **Build and test programs:** + ```bash + make guests + cargo run -p pvq-test-runner -- --program output/guest-total-supply + ``` + +3. **Use with Polkadot JS UI:** + - Copy the hex-encoded `args` from test runner logs + - Upload program and arguments through the PJS UI + +## πŸ“– Documentation + +### Writing Your First PVQ Program + +```rust +use pvq_program::program; +use pvq_extension_fungibles::ExtensionFungibles; + +#[program] +fn query_balance(account: [u8; 32], asset_id: u32) -> u128 { + ExtensionFungibles::balance(asset_id, &account) +} +``` + +### Creating Custom Extensions + +```rust +use pvq_extension::extension_decl; + +#[extension_decl] +pub trait MyCustomExtension { + fn my_query() -> String; +} +``` + +### Component Documentation + +- πŸ“š **[Development Guide](docs/development.md)** - Comprehensive development setup +- πŸ”§ **[Extension Development](docs/extensions.md)** - Creating custom extensions +- πŸ—οΈ **[Architecture Deep Dive](docs/architecture.md)** - System design and internals +- πŸ“‹ **[API Reference](docs/api.md)** - Complete API documentation + +## πŸ› οΈ Development + +### Project Structure + +``` +PVQ/ +β”œβ”€β”€ poc/runtime/ # PoC Substrate runtime +β”œβ”€β”€ pvq-program/ # Program macro and utilities +β”œβ”€β”€ pvq-executor/ # Core execution engine +β”œβ”€β”€ pvq-extension*/ # Extension system and implementations +β”œβ”€β”€ pvq-runtime-api/ # Runtime API definition +β”œβ”€β”€ pvq-test-runner/ # Testing utilities +β”œβ”€β”€ guest-examples/ # Example programs +└── vendor/polkavm/ # Vendored PolkaVM dependency +``` + +### Building from Source + +```bash +# Development build +cargo build + +# Release build with optimizations +cargo build --release + +# Build guest examples +make guests + +# Run all tests +cargo test --all + +# Run clippy lints +cargo clippy --all --all-targets +``` + +### Testing + +```bash +# Run unit tests +cargo test + +# Run integration tests +cargo test --test integration + +# Test specific component +cargo test -p pvq-executor + +# Run example programs +make test-guests +``` + +## 🀝 Contributing + +We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. + +### Development Workflow + +1. Fork the repository +2. Create a feature branch (`git checkout -b feature/amazing-feature`) +3. Make your changes and add tests +4. Ensure all tests pass (`cargo test --all`) +5. Run formatting (`cargo fmt`) and linting (`cargo clippy`) +6. Commit your changes (`git commit -am 'Add amazing feature'`) +7. Push to the branch (`git push origin feature/amazing-feature`) +8. Open a Pull Request + +### Code Style + +- Follow Rust standard formatting (`cargo fmt`) +- Ensure clippy passes (`cargo clippy`) +- Add comprehensive tests for new features +- Document public APIs with rustdoc comments + +## πŸ“„ License -### Run Examples +This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. -`guest-examples` contains several guest programs to test the PVQ. +## πŸ™ Acknowledgments -1. Build guest program: `make guests` -2. Run test runner: `cargo run -p pvq-test-runner -- --program output/` +- [PolkaVM](https://github.com/koute/polkavm) - The underlying virtual machine +- [Substrate](https://substrate.io/) - The blockchain development framework +- [Polkadot](https://polkadot.network/) - The multi-chain protocol -Available PoC guest programs: +## πŸ”— Links -- `guest-sum-balance`: sum the balances of multiple accounts -- `guest-total-supply`: get the total supply of an asset -- `guest-sum-balance-percent`: sum the balances of multiple accounts and calculate the percentage of the total supply +- 🌐 **Homepage**: [https://acala.network](https://acala.network) +- πŸ“– **Documentation**: [https://docs.pvq.dev](https://docs.pvq.dev) +- πŸ’¬ **Discord**: [Join our community](https://discord.gg/acala) +- 🐦 **Twitter**: [@AcalaNetwork](https://twitter.com/AcalaNetwork) -### RuntimeAPI PoC +--- -1. Use chopsticks to start a local chain with the RuntimeAPI enabled: `make run` -2. Build guest programs: `make guests` -3. Run test runner to display hex-encoded `args` in tracing logs: `cargo run -p pvq-test-runner -- --program output/` -4. Upload `program` and `args` in PJS UI. +**Built with ❀️ by the Acala team** \ No newline at end of file diff --git a/poc/runtime/README.md b/poc/runtime/README.md new file mode 100644 index 0000000..1558ccb --- /dev/null +++ b/poc/runtime/README.md @@ -0,0 +1,261 @@ +# PVQ PoC Runtime + + + +A proof-of-concept Substrate runtime demonstrating PVQ (PolkaVM Query) system integration. This runtime serves as a testing environment and reference implementation for integrating PVQ capabilities into Substrate-based blockchains. + +## Overview + +This PoC runtime showcases how to: +- Integrate the `PvqApi` runtime API into a Substrate runtime +- Configure PVQ extensions for runtime interaction +- Provide a development environment for testing PVQ queries +- Demonstrate best practices for PVQ integration + +## Features + +- πŸ—οΈ **Complete Runtime**: Full Substrate runtime with PVQ integration +- πŸ”Œ **Extension Support**: Pre-configured with core PVQ extensions +- πŸ§ͺ **Testing Ready**: Optimized for development and testing workflows +- πŸ“‘ **Chopsticks Compatible**: Works with Chopsticks for local development +- βš™οΈ **Configurable**: Easy to modify and extend for custom use cases + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PoC Runtime β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ Standard Pallets β”‚ PVQ Integration β”‚ +β”‚ β”œβ”€ Balances β”‚ β”œβ”€ PvqApi β”‚ +β”‚ β”œβ”€ Assets β”‚ β”œβ”€ Extensions β”‚ +β”‚ β”œβ”€ Timestamp β”‚ └─ Executor β”‚ +β”‚ β”œβ”€ Transaction Payment β”‚ β”‚ +β”‚ └─ Sudo β”‚ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Runtime Components + +### Standard Pallets + +| Pallet | Purpose | +|--------|---------| +| **Balances** | Native token balance management | +| **Assets** | Multi-asset support for fungible tokens | +| **Timestamp** | Block timestamp functionality | +| **Transaction Payment** | Fee calculation and payment | +| **Sudo** | Privileged operations for testing | + +### PVQ Integration + +| Component | Purpose | +|-----------|---------| +| **PvqApi** | Runtime API for executing PVQ queries | +| **Extensions** | Configured extensions (core, fungibles, swap) | +| **Executor** | PVQ program execution environment | + +## Getting Started + +### Prerequisites + +- Rust toolchain (stable) +- PVQ project dependencies +- Chopsticks (for local development) + +### Building the Runtime + +```bash +# Build the runtime +cargo build -p poc-runtime --release + +# The compiled WASM runtime will be available in target/release/wbuild/ +``` + +### Running with Chopsticks + +1. **Start the local development chain:** + ```bash + make run + ``` + +2. **The runtime will be available at:** + - HTTP RPC: `http://localhost:8000` + - WebSocket: `ws://localhost:8000` + +### Testing PVQ Integration + +1. **Build guest programs:** + ```bash + make guests + ``` + +2. **Test with PVQ runner:** + ```bash + cargo run -p pvq-test-runner -- --program output/guest-total-supply + ``` + +3. **Use with Polkadot JS UI:** + - Connect to `ws://localhost:8000` + - Navigate to Developer β†’ Extrinsics β†’ pvq β†’ executeQuery + - Upload program blob and arguments + +## Configuration + +### Chopsticks Configuration + +The runtime includes a `chopsticks.yml` configuration file: + +```yaml +endpoint: wss://acala-rpc-0.aca-api.network +block: latest +runtime-log-level: 5 +``` + +### Runtime Parameters + +Key runtime configurations: + +```rust +// Maximum execution time for PVQ queries +pub const MAX_EXECUTION_TIME: u64 = 10_000; // 10 seconds + +// Maximum memory allocation for guest programs +pub const MAX_MEMORY: u32 = 16 * 1024 * 1024; // 16MB + +// Supported extensions +pub const EXTENSIONS: &[&str] = &["core", "fungibles", "swap"]; +``` + +## Development Workflow + +### Adding New Pallets + +1. **Add dependency to `Cargo.toml`:** + ```toml + my-custom-pallet = { version = "1.0.0", default-features = false } + ``` + +2. **Configure in `lib.rs`:** + ```rust + impl my_custom_pallet::Config for Runtime { + // Configuration here + } + ``` + +3. **Add to runtime construction:** + ```rust + construct_runtime!( + pub struct Runtime { + // ... existing pallets + MyCustomPallet: my_custom_pallet, + } + ); + ``` + +### Adding PVQ Extensions + +1. **Add extension dependency:** + ```toml + my-pvq-extension = { path = "../my-pvq-extension" } + ``` + +2. **Configure in PVQ setup:** + ```rust + use my_pvq_extension::ExtensionMyCustom; + + // Add to extensions list + ``` + +### Testing Changes + +```bash +# Build and test +cargo build -p poc-runtime +make run + +# Test with guest programs +cargo run -p pvq-test-runner -- --program output/guest-example +``` + +## API Reference + +### PVQ Runtime API + +The runtime exposes the following PVQ API methods: + +#### `execute_query(program: Vec, args: Vec) -> Vec` + +Executes a PVQ program with the provided arguments. + +**Parameters:** +- `program`: Compiled PVQ program blob +- `args`: Serialized input arguments + +**Returns:** Serialized query results + +#### `metadata() -> Vec` + +Returns metadata about available PVQ extensions and capabilities. + +**Returns:** Serialized metadata information + +### Extension APIs + +All standard PVQ extensions are available: + +- **Core Extension**: Basic functionality and extension discovery +- **Fungibles Extension**: Asset queries and balance information +- **Swap Extension**: DEX interactions and liquidity data + +## Troubleshooting + +### Common Issues + +**Runtime doesn't build:** +```bash +# Clean and rebuild +cargo clean +cargo build -p poc-runtime --release +``` + +**Chopsticks connection fails:** +```bash +# Check if the endpoint is accessible +curl -X POST -H "Content-Type: application/json" \ + --data '{"jsonrpc":"2.0","method":"system_name","id":1}' \ + http://localhost:8000 +``` + +**PVQ queries fail:** +- Ensure guest programs are compiled correctly +- Check that required extensions are configured +- Verify argument serialization format + +### Debug Mode + +Enable debug logging: + +```bash +RUST_LOG=debug cargo run -p pvq-test-runner -- --program output/guest-example +``` + +## Contributing + +When modifying the PoC runtime: + +1. Maintain compatibility with existing PVQ programs +2. Update tests for any configuration changes +3. Document any new pallets or extensions +4. Test with both Chopsticks and the test runner + +## Related Components + +- [PVQ Executor](../pvq-executor/) - Core execution engine +- [PVQ Runtime API](../pvq-runtime-api/) - API definitions +- [PVQ Extensions](../pvq-extension/) - Extension system +- [Guest Examples](../guest-examples/) - Example programs + +--- + +*This PoC runtime demonstrates PVQ integration patterns and serves as a foundation for production implementations.* \ No newline at end of file diff --git a/pvq-executor/README.md b/pvq-executor/README.md new file mode 100644 index 0000000..8aa08f0 --- /dev/null +++ b/pvq-executor/README.md @@ -0,0 +1,338 @@ +# PVQ Executor + + + +The core execution engine for PVQ (PolkaVM Query) programs. This crate provides a secure, sandboxed environment for running guest programs compiled to RISC-V, with controlled access to runtime functionality through extensions. + +## Overview + +The PVQ Executor serves as the bridge between compiled guest programs and the Substrate runtime. It manages PolkaVM instances, handles program lifecycle, and provides secure execution context for queries. + +## Features + +- πŸ”’ **Secure Execution**: Sandboxed PolkaVM environment with resource limits +- ⚑ **High Performance**: Optimized for fast query execution +- πŸ›‘οΈ **Memory Safety**: Controlled memory allocation and access +- πŸ”Œ **Extension Integration**: Seamless interaction with PVQ extensions +- ⏱️ **Timeout Protection**: Configurable execution time limits +- πŸ“Š **Resource Monitoring**: Track memory usage and execution metrics + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Executor β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ PvqExecutor β”‚ β”‚ ExecutorContext β”‚ β”‚ PolkaVM β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ Instance β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Lifecycle β”‚ β”‚ β”œβ”€ Extensions β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Resource β”‚ β”‚ β”œβ”€ Runtime β”‚ β”‚ β”œβ”€ Memory β”‚ β”‚ +β”‚ β”‚ β”‚ Management β”‚ β”‚ β”‚ Access β”‚ β”‚ β”œβ”€ Execution β”‚ β”‚ +β”‚ β”‚ └─ Error β”‚ β”‚ └─ Permissions β”‚ β”‚ └─ Sandbox β”‚ β”‚ +β”‚ β”‚ Handling β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Core Components + +### PvqExecutor + +The main executor that manages the complete lifecycle of PVQ program execution. + +**Responsibilities:** +- Program loading and validation +- PolkaVM instance creation and management +- Execution context setup +- Resource limit enforcement +- Error handling and recovery + +### PvqExecutorContext + +Provides the execution context and environment for guest programs. + +**Responsibilities:** +- Extension registry management +- Runtime state access +- Permission control +- Inter-component communication + +### Error Handling + +Comprehensive error types covering all execution scenarios. + +**Error Categories:** +- Program loading errors +- Execution timeout/resource exhaustion +- Extension call failures +- Memory access violations + +## Usage + +### Basic Execution + +```rust +use pvq_executor::{PvqExecutor, PvqExecutorContext}; +use pvq_extension::ExtensionsExecutor; + +// Create executor context +let extensions = ExtensionsExecutor::new(); +let context = PvqExecutorContext::new(extensions); + +// Create and configure executor +let executor = PvqExecutor::new(context); + +// Load and execute program +let program_blob = std::fs::read("program.polkavm")?; +let args = vec![1, 2, 3, 4]; // Serialized arguments + +let result = executor.execute(&program_blob, &args)?; +println!("Execution result: {:?}", result); +``` + +### Advanced Configuration + +```rust +use pvq_executor::{PvqExecutor, PvqExecutorConfig, PvqExecutorContext}; +use std::time::Duration; + +// Configure execution parameters +let config = PvqExecutorConfig { + max_memory: 32 * 1024 * 1024, // 32MB + max_execution_time: Duration::from_secs(30), + enable_debugging: false, + stack_size: 1024 * 1024, // 1MB +}; + +// Create executor with custom config +let context = PvqExecutorContext::new(extensions); +let executor = PvqExecutor::with_config(context, config); +``` + +### Error Handling + +```rust +use pvq_executor::{PvqExecutor, PvqExecutorError}; + +match executor.execute(&program_blob, &args) { + Ok(result) => { + println!("Success: {:?}", result); + } + Err(PvqExecutorError::Timeout) => { + eprintln!("Program execution timed out"); + } + Err(PvqExecutorError::OutOfMemory) => { + eprintln!("Program exceeded memory limits"); + } + Err(PvqExecutorError::ExtensionError(e)) => { + eprintln!("Extension call failed: {}", e); + } + Err(e) => { + eprintln!("Execution failed: {}", e); + } +} +``` + +## Configuration + +### Execution Limits + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `max_memory` | 16MB | Maximum memory allocation | +| `max_execution_time` | 10s | Maximum execution duration | +| `stack_size` | 512KB | Program stack size | +| `max_call_depth` | 1024 | Maximum call stack depth | + +### Security Settings + +| Setting | Default | Description | +|---------|---------|-------------| +| `enable_debugging` | false | Enable debug information | +| `strict_memory` | true | Strict memory access checking | +| `extension_whitelist` | None | Allowed extensions (None = all) | + +### Performance Tuning + +```rust +let config = PvqExecutorConfig { + // Optimize for throughput + max_memory: 64 * 1024 * 1024, + precompile_programs: true, + enable_jit: true, + + // Or optimize for safety + max_execution_time: Duration::from_secs(5), + strict_memory: true, + enable_debugging: false, +}; +``` + +## Integration + +### With Substrate Runtime + +```rust +use pvq_executor::{PvqExecutor, PvqExecutorContext}; +use pvq_runtime_api::PvqApi; +use sp_api::ProvideRuntimeApi; + +impl PvqApi for Runtime { + fn execute_query(&self, program: Vec, args: Vec) -> Vec { + let extensions = self.create_extensions_executor(); + let context = PvqExecutorContext::new(extensions); + let executor = PvqExecutor::new(context); + + executor.execute(&program, &args) + .unwrap_or_else(|e| format!("Error: {}", e).into_bytes()) + } +} +``` + +### With Extension System + +```rust +use pvq_executor::PvqExecutorContext; +use pvq_extension::ExtensionsExecutor; +use pvq_extension_fungibles::ExtensionFungibles; + +// Register extensions +let mut extensions = ExtensionsExecutor::new(); +extensions.register::(); + +// Create context with extensions +let context = PvqExecutorContext::new(extensions); +``` + +## Development + +### Building + +```bash +# Build the executor +cargo build -p pvq-executor + +# Build with all features +cargo build -p pvq-executor --all-features + +# Run tests +cargo test -p pvq-executor +``` + +### Testing + +```bash +# Run unit tests +cargo test -p pvq-executor + +# Run integration tests +cargo test -p pvq-executor --test integration + +# Test with different configurations +cargo test -p pvq-executor --test config_tests + +# Benchmark performance +cargo bench -p pvq-executor +``` + +### Debugging + +Enable debug logging: + +```rust +use log::debug; + +// In executor implementation +debug!("Executing program with {} bytes", program.len()); +debug!("Memory usage: {} bytes", memory_usage); +``` + +Run with debug output: + +```bash +RUST_LOG=pvq_executor=debug cargo test +``` + +## Error Reference + +### PvqExecutorError Types + +| Error Type | Description | Recovery | +|------------|-------------|----------| +| `InvalidProgram` | Malformed program blob | Validate program before execution | +| `Timeout` | Execution time limit exceeded | Increase timeout or optimize program | +| `OutOfMemory` | Memory limit exceeded | Increase memory limit or optimize program | +| `ExtensionError` | Extension call failed | Check extension implementation | +| `RuntimeError` | Runtime execution error | Debug program logic | + +### Common Issues + +**Program won't load:** +```bash +# Validate program format +polkatool validate program.polkavm + +# Check file size and format +file program.polkavm +``` + +**Execution timeouts:** +```rust +// Increase timeout for complex queries +let config = PvqExecutorConfig { + max_execution_time: Duration::from_secs(60), + ..Default::default() +}; +``` + +**Memory errors:** +```rust +// Monitor memory usage +let config = PvqExecutorConfig { + max_memory: 64 * 1024 * 1024, // Increase limit + enable_debugging: true, // Enable memory tracking + ..Default::default() +}; +``` + +## Performance + +### Optimization Tips + +1. **Program Size**: Keep programs small for faster loading +2. **Memory Access**: Use sequential memory access patterns +3. **Extension Calls**: Minimize expensive extension operations +4. **Data Structures**: Use efficient serialization formats + +### Benchmarking + +```bash +# Run performance benchmarks +cargo bench -p pvq-executor + +# Profile memory usage +cargo run --example memory_profile --release + +# Measure execution time +cargo run --example timing_analysis --release +``` + +## Security Considerations + +- Programs execute in a sandboxed environment +- Memory access is controlled and limited +- Execution time is bounded to prevent DoS +- Extension calls go through permission checks +- No direct system call access from guest programs + +## Related Components + +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration +- [PVQ Program](../pvq-program/) - Program development tools +- [PVQ Test Runner](../pvq-test-runner/) - Testing utilities + +--- + +*The PVQ Executor provides the secure foundation for all PVQ program execution.* \ No newline at end of file diff --git a/pvq-extension-core/README.md b/pvq-extension-core/README.md new file mode 100644 index 0000000..3e12ab0 --- /dev/null +++ b/pvq-extension-core/README.md @@ -0,0 +1,384 @@ +# PVQ Extension Core + + + +The foundational extension for the PVQ (PolkaVM Query) system, providing essential functionality and serving as the base for all other extensions. Every PVQ program has access to core extension features by default. + +## Overview + +The Core Extension provides fundamental capabilities that all PVQ guest programs need, including extension discovery, system information, and basic utility functions. It serves as the foundation layer that other extensions build upon. + +## Features + +- πŸ” **Extension Discovery**: Query available extensions at runtime +- ℹ️ **System Information**: Access runtime and execution environment details +- πŸ› οΈ **Utility Functions**: Common operations needed by guest programs +- πŸ”’ **Security Primitives**: Basic security and validation functions +- πŸ“Š **Metadata Access**: Extension and system metadata queries + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ ExtensionCore β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Discovery β”‚ β”‚ Utilities β”‚ β”‚ Metadata β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ β”œβ”€ Hashing β”‚ β”‚ β”œβ”€ System β”‚ β”‚ +β”‚ β”‚ β”‚ Enumeration β”‚ β”‚ β”œβ”€ Encoding β”‚ β”‚ β”‚ Info β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Capability β”‚ β”‚ β”œβ”€ Validation β”‚ β”‚ β”œβ”€ Runtime β”‚ β”‚ +β”‚ β”‚ β”‚ Checking β”‚ β”‚ └─ Conversion β”‚ β”‚ β”‚ Version β”‚ β”‚ +β”‚ β”‚ └─ Version β”‚ β”‚ β”‚ β”‚ └─ Extension β”‚ β”‚ +β”‚ β”‚ Matching β”‚ β”‚ β”‚ β”‚ Metadata β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## API Reference + +### Extension Discovery + +#### `extension_exists(name: &str) -> bool` + +Checks if a specific extension is available in the current execution environment. + +```rust +use pvq_extension_core::ExtensionCore; + +// Check if fungibles extension is available +if ExtensionCore::extension_exists("fungibles") { + // Use fungibles extension +} else { + // Fallback logic +} +``` + +#### `list_extensions() -> Vec` + +Returns a list of all available extensions in the current environment. + +```rust +use pvq_extension_core::ExtensionCore; + +let available_extensions = ExtensionCore::list_extensions(); +for extension in available_extensions { + println!("Available: {}", extension); +} +``` + +#### `extension_version(name: &str) -> Option` + +Gets the version of a specific extension. + +```rust +use pvq_extension_core::ExtensionCore; + +if let Some(version) = ExtensionCore::extension_version("fungibles") { + println!("Fungibles extension version: {}", version); +} +``` + +### System Information + +#### `runtime_version() -> String` + +Returns the runtime version information. + +```rust +use pvq_extension_core::ExtensionCore; + +let version = ExtensionCore::runtime_version(); +println!("Runtime version: {}", version); +``` + +#### `execution_info() -> ExecutionInfo` + +Provides information about the current execution environment. + +```rust +use pvq_extension_core::{ExtensionCore, ExecutionInfo}; + +let info = ExtensionCore::execution_info(); +println!("Max memory: {} bytes", info.max_memory); +println!("Execution timeout: {:?}", info.timeout); +``` + +### Utility Functions + +#### `hash_blake2_256(data: &[u8]) -> [u8; 32]` + +Computes Blake2 256-bit hash of the input data. + +```rust +use pvq_extension_core::ExtensionCore; + +let data = b"Hello, PVQ!"; +let hash = ExtensionCore::hash_blake2_256(data); +println!("Hash: {:?}", hash); +``` + +#### `validate_account_id(account: &[u8]) -> bool` + +Validates whether the provided bytes represent a valid account ID. + +```rust +use pvq_extension_core::ExtensionCore; + +let account = [1u8; 32]; +if ExtensionCore::validate_account_id(&account) { + println!("Valid account ID"); +} +``` + +#### `current_block_number() -> u32` + +Returns the current block number. + +```rust +use pvq_extension_core::ExtensionCore; + +let block_number = ExtensionCore::current_block_number(); +println!("Current block: {}", block_number); +``` + +## Data Types + +### ExecutionInfo + +Information about the current execution environment: + +```rust +pub struct ExecutionInfo { + pub max_memory: u64, + pub timeout: Duration, + pub runtime_version: String, + pub extensions_count: usize, +} +``` + +### ExtensionMetadata + +Metadata for an extension: + +```rust +pub struct ExtensionMetadata { + pub name: String, + pub version: String, + pub description: String, + pub functions: Vec, +} +``` + +## Usage Examples + +### Basic Extension Check + +```rust +use pvq_extension_core::ExtensionCore; + +#[pvq_program::program] +fn my_query() -> String { + if ExtensionCore::extension_exists("fungibles") { + "Fungibles extension is available".to_string() + } else { + "Fungibles extension not found".to_string() + } +} +``` + +### System Information Query + +```rust +use pvq_extension_core::ExtensionCore; + +#[pvq_program::program] +fn system_info() -> String { + let runtime_version = ExtensionCore::runtime_version(); + let block_number = ExtensionCore::current_block_number(); + let extensions = ExtensionCore::list_extensions(); + + format!( + "Runtime: {}, Block: {}, Extensions: {:?}", + runtime_version, block_number, extensions + ) +} +``` + +### Conditional Feature Usage + +```rust +use pvq_extension_core::ExtensionCore; +use pvq_extension_fungibles::ExtensionFungibles; +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn advanced_query(asset_id: u32) -> String { + let mut result = String::new(); + + // Always available - core functionality + let block = ExtensionCore::current_block_number(); + result.push_str(&format!("Block: {}\n", block)); + + // Check for fungibles extension + if ExtensionCore::extension_exists("fungibles") { + let supply = ExtensionFungibles::total_supply(asset_id); + result.push_str(&format!("Total Supply: {}\n", supply)); + } + + // Check for swap extension + if ExtensionCore::extension_exists("swap") { + let pools = ExtensionSwap::list_pools(); + result.push_str(&format!("Available Pools: {}\n", pools.len())); + } + + result +} +``` + +## Integration + +### With Other Extensions + +Core extension is automatically available and should be used by other extensions: + +```rust +// In custom extension implementation +use pvq_extension_core::ExtensionCore; + +impl MyExtension for MyExtensionImpl { + fn my_function() -> bool { + // Use core functionality + let block = ExtensionCore::current_block_number(); + + // Extension-specific logic + block > 1000 + } +} +``` + +### Extension Development + +When creating new extensions, leverage core utilities: + +```rust +use pvq_extension_core::ExtensionCore; +use pvq_extension::extension_decl; + +#[extension_decl] +pub trait MyCustomExtension { + fn custom_query() -> String { + // Use core hashing + let data = b"custom data"; + let hash = ExtensionCore::hash_blake2_256(data); + + // Use system info + let version = ExtensionCore::runtime_version(); + + format!("Hash: {:?}, Version: {}", hash, version) + } +} +``` + +## Configuration + +### Runtime Configuration + +Configure core extension behavior: + +```rust +// In runtime configuration +impl pvq_extension_core::Config for Runtime { + type MaxExtensionNameLength = ConstU32<32>; + type MaxExtensionsCount = ConstU32<64>; + type EnableDebugInfo = ConstBool; +} +``` + +### Extension Registry + +Register core extension in executor: + +```rust +use pvq_extension::ExtensionsExecutor; +use pvq_extension_core::ExtensionCore; + +let mut executor = ExtensionsExecutor::new(); +executor.register::(); +``` + +## Development + +### Building + +```bash +# Build core extension +cargo build -p pvq-extension-core + +# Run tests +cargo test -p pvq-extension-core + +# Check documentation +cargo doc -p pvq-extension-core --open +``` + +### Testing + +```bash +# Unit tests +cargo test -p pvq-extension-core + +# Integration tests with executor +cargo test -p pvq-extension-core --test integration + +# Test with guest programs +cargo run -p pvq-test-runner -- --program test-core-extension +``` + +## Security Considerations + +- All core functions are safe and do not expose sensitive runtime internals +- Extension discovery is read-only and cannot modify system state +- Hash functions use cryptographically secure algorithms +- Account ID validation prevents invalid account access attempts + +## Performance + +### Optimization Tips + +1. **Cache Extension List**: Extension discovery results can be cached +2. **Batch Queries**: Combine multiple core queries when possible +3. **Selective Checks**: Only check for extensions you actually need + +### Benchmarks + +```bash +# Run core extension benchmarks +cargo bench -p pvq-extension-core + +# Profile extension discovery +cargo run --example profile_discovery --release +``` + +## Error Handling + +Core extension functions are designed to be safe and rarely fail: + +```rust +// Most functions return safe defaults +let extensions = ExtensionCore::list_extensions(); // Never fails + +// Optional results for queries that might not have data +let version = ExtensionCore::extension_version("nonexistent"); // Returns None +``` + +## Related Components + +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Fungibles Extension](../pvq-extension-fungibles/) - Asset functionality +- [PVQ Swap Extension](../pvq-extension-swap/) - DEX functionality +- [PVQ Executor](../pvq-executor/) - Execution environment + +--- + +*The Core Extension provides the essential foundation for all PVQ program functionality.* \ No newline at end of file diff --git a/pvq-extension-fungibles/README.md b/pvq-extension-fungibles/README.md new file mode 100644 index 0000000..37306fd --- /dev/null +++ b/pvq-extension-fungibles/README.md @@ -0,0 +1,513 @@ +# PVQ Extension Fungibles + + + +A powerful extension for the PVQ (PolkaVM Query) system that enables guest programs to query fungible asset information from the runtime. This extension provides comprehensive access to asset balances, metadata, and supply information. + +## Overview + +The Fungibles Extension integrates with Substrate's assets pallet and native balance systems to provide PVQ guest programs with read-only access to fungible token data. It supports both native tokens and custom assets created through the assets pallet. + +## Features + +- πŸ’° **Balance Queries**: Get account balances for any asset +- πŸ“Š **Supply Information**: Query total supply and supply metrics +- ℹ️ **Asset Metadata**: Access asset names, symbols, and decimals +- πŸ” **Asset Discovery**: List and enumerate available assets +- 🏦 **Multi-Asset Support**: Native tokens and custom assets +- ⚑ **Efficient Queries**: Optimized for high-performance data access + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ ExtensionFungibles β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Balances β”‚ β”‚ Assets β”‚ β”‚ Metadata β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Account β”‚ β”‚ β”œβ”€ Total Supply β”‚ β”‚ β”œβ”€ Name β”‚ β”‚ +β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Minimum β”‚ β”‚ β”œβ”€ Symbol β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Free β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Decimals β”‚ β”‚ +β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Asset Exists β”‚ β”‚ └─ Custom β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Reserved β”‚ β”‚ └─ Asset List β”‚ β”‚ Fields β”‚ β”‚ +β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ └─ Locked β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ Balance β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## API Reference + +### Balance Operations + +#### `balance(asset_id: u32, account: &[u8; 32]) -> u128` + +Get the total balance (free + reserved) of an account for a specific asset. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn get_account_balance(asset_id: u32, account: [u8; 32]) -> u128 { + ExtensionFungibles::balance(asset_id, &account) +} +``` + +#### `balance_free(asset_id: u32, account: &[u8; 32]) -> u128` + +Get the free (transferable) balance of an account. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +let free_balance = ExtensionFungibles::balance_free(asset_id, &account); +println!("Free balance: {}", free_balance); +``` + +#### `balance_reserved(asset_id: u32, account: &[u8; 32]) -> u128` + +Get the reserved (non-transferable) balance of an account. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +let reserved_balance = ExtensionFungibles::balance_reserved(asset_id, &account); +println!("Reserved balance: {}", reserved_balance); +``` + +### Asset Information + +#### `total_supply(asset_id: u32) -> u128` + +Get the total supply of an asset. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn asset_total_supply(asset_id: u32) -> u128 { + ExtensionFungibles::total_supply(asset_id) +} +``` + +#### `minimum_balance(asset_id: u32) -> u128` + +Get the minimum balance required to maintain an account. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +let min_balance = ExtensionFungibles::minimum_balance(asset_id); +println!("Minimum balance: {}", min_balance); +``` + +#### `asset_exists(asset_id: u32) -> bool` + +Check if an asset exists in the runtime. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +if ExtensionFungibles::asset_exists(asset_id) { + // Asset is valid + let supply = ExtensionFungibles::total_supply(asset_id); +} else { + // Asset doesn't exist +} +``` + +### Asset Discovery + +#### `list_assets() -> Vec` + +Get a list of all available asset IDs. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn available_assets() -> Vec { + ExtensionFungibles::list_assets() +} +``` + +#### `asset_count() -> u32` + +Get the total number of assets in the system. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +let count = ExtensionFungibles::asset_count(); +println!("Total assets: {}", count); +``` + +### Metadata Operations + +#### `asset_metadata(asset_id: u32) -> Option` + +Get complete metadata for an asset. + +```rust +use pvq_extension_fungibles::{ExtensionFungibles, AssetMetadata}; + +if let Some(metadata) = ExtensionFungibles::asset_metadata(asset_id) { + println!("Name: {}", metadata.name); + println!("Symbol: {}", metadata.symbol); + println!("Decimals: {}", metadata.decimals); +} +``` + +#### `asset_name(asset_id: u32) -> Option` + +Get the name of an asset. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +if let Some(name) = ExtensionFungibles::asset_name(asset_id) { + println!("Asset name: {}", name); +} +``` + +#### `asset_symbol(asset_id: u32) -> Option` + +Get the symbol of an asset. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +if let Some(symbol) = ExtensionFungibles::asset_symbol(asset_id) { + println!("Asset symbol: {}", symbol); +} +``` + +#### `asset_decimals(asset_id: u32) -> Option` + +Get the decimal precision of an asset. + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +if let Some(decimals) = ExtensionFungibles::asset_decimals(asset_id) { + println!("Asset decimals: {}", decimals); +} +``` + +## Data Types + +### AssetMetadata + +Complete asset metadata information: + +```rust +pub struct AssetMetadata { + pub name: String, + pub symbol: String, + pub decimals: u8, + pub is_frozen: bool, +} +``` + +### BalanceInfo + +Comprehensive balance information: + +```rust +pub struct BalanceInfo { + pub free: u128, + pub reserved: u128, + pub frozen: u128, +} +``` + +## Usage Examples + +### Basic Balance Query + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn check_balance(asset_id: u32, account: [u8; 32]) -> String { + let balance = ExtensionFungibles::balance(asset_id, &account); + + if balance == 0 { + "Account has no balance".to_string() + } else { + format!("Balance: {}", balance) + } +} +``` + +### Multi-Account Balance Sum + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn sum_balances(asset_id: u32, accounts: Vec<[u8; 32]>) -> u128 { + accounts.iter() + .map(|account| ExtensionFungibles::balance(asset_id, account)) + .sum() +} +``` + +### Asset Analysis + +```rust +use pvq_extension_fungibles::{ExtensionFungibles, AssetMetadata}; + +#[pvq_program::program] +fn analyze_asset(asset_id: u32) -> String { + if !ExtensionFungibles::asset_exists(asset_id) { + return "Asset does not exist".to_string(); + } + + let supply = ExtensionFungibles::total_supply(asset_id); + let min_balance = ExtensionFungibles::minimum_balance(asset_id); + + let metadata = ExtensionFungibles::asset_metadata(asset_id); + + match metadata { + Some(meta) => { + format!( + "Asset {}: {} ({}) - Supply: {}, Min Balance: {}, Decimals: {}", + asset_id, meta.name, meta.symbol, supply, min_balance, meta.decimals + ) + } + None => { + format!( + "Asset {}: No metadata - Supply: {}, Min Balance: {}", + asset_id, supply, min_balance + ) + } + } +} +``` + +### Portfolio Tracker + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn portfolio_value(account: [u8; 32]) -> Vec<(u32, u128)> { + let assets = ExtensionFungibles::list_assets(); + let mut portfolio = Vec::new(); + + for asset_id in assets { + let balance = ExtensionFungibles::balance(asset_id, &account); + if balance > 0 { + portfolio.push((asset_id, balance)); + } + } + + portfolio +} +``` + +### Percentage of Total Supply + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn balance_percentage(asset_id: u32, account: [u8; 32]) -> f64 { + let balance = ExtensionFungibles::balance(asset_id, &account); + let total_supply = ExtensionFungibles::total_supply(asset_id); + + if total_supply == 0 { + 0.0 + } else { + (balance as f64 / total_supply as f64) * 100.0 + } +} +``` + +## Integration + +### With Runtime + +Configure the fungibles extension in your runtime: + +```rust +// In runtime configuration +impl pvq_extension_fungibles::Config for Runtime { + type AssetId = u32; + type Balance = u128; + type AssetRegistry = Assets; + type NativeBalance = Balances; +} +``` + +### With Extension Executor + +Register the fungibles extension: + +```rust +use pvq_extension::ExtensionsExecutor; +use pvq_extension_fungibles::ExtensionFungibles; + +let mut executor = ExtensionsExecutor::new(); +executor.register::(); +``` + +### Error Handling + +Handle potential query failures: + +```rust +use pvq_extension_fungibles::ExtensionFungibles; + +#[pvq_program::program] +fn safe_balance_query(asset_id: u32, account: [u8; 32]) -> String { + if !ExtensionFungibles::asset_exists(asset_id) { + return "Asset not found".to_string(); + } + + let balance = ExtensionFungibles::balance(asset_id, &account); + format!("Balance: {}", balance) +} +``` + +## Configuration + +### Asset ID Types + +Support different asset ID types: + +```rust +impl pvq_extension_fungibles::Config for Runtime { + // For simple numeric IDs + type AssetId = u32; + + // For complex asset identifiers + type AssetId = AssetIdOf; +} +``` + +### Balance Types + +Configure balance precision: + +```rust +impl pvq_extension_fungibles::Config for Runtime { + // Standard 128-bit balance + type Balance = u128; + + // Custom balance type + type Balance = BalanceOf; +} +``` + +## Development + +### Building + +```bash +# Build fungibles extension +cargo build -p pvq-extension-fungibles + +# Run tests +cargo test -p pvq-extension-fungibles + +# Generate documentation +cargo doc -p pvq-extension-fungibles --open +``` + +### Testing + +```bash +# Unit tests +cargo test -p pvq-extension-fungibles + +# Integration tests with runtime +cargo test -p pvq-extension-fungibles --test runtime_integration + +# Test with guest programs +cargo run -p pvq-test-runner -- --program test-fungibles +``` + +### Benchmarking + +```bash +# Run performance benchmarks +cargo bench -p pvq-extension-fungibles + +# Profile balance queries +cargo run --example profile_queries --release +``` + +## Performance Considerations + +### Optimization Tips + +1. **Batch Queries**: Query multiple balances in a single program +2. **Asset Filtering**: Use `asset_exists()` before expensive operations +3. **Cache Metadata**: Store frequently accessed metadata +4. **Selective Fields**: Only query the specific data you need + +### Query Patterns + +```rust +// Efficient: Single program, multiple accounts +#[pvq_program::program] +fn efficient_multi_balance(asset_id: u32, accounts: Vec<[u8; 32]>) -> Vec { + accounts.iter() + .map(|account| ExtensionFungibles::balance(asset_id, account)) + .collect() +} + +// Less efficient: Multiple program calls +// Don't do this - call the program once with all accounts +``` + +## Security Considerations + +- All functions are read-only and cannot modify balances +- Account data access is controlled by the runtime +- Asset existence checks prevent invalid asset queries +- No sensitive runtime internals are exposed + +## Common Use Cases + +### DeFi Applications + +```rust +// Calculate liquidity pool ratios +#[pvq_program::program] +fn pool_ratio(token_a: u32, token_b: u32, pool_account: [u8; 32]) -> (u128, u128) { + let balance_a = ExtensionFungibles::balance(token_a, &pool_account); + let balance_b = ExtensionFungibles::balance(token_b, &pool_account); + (balance_a, balance_b) +} +``` + +### Analytics + +```rust +// Top holders analysis +#[pvq_program::program] +fn top_holders_share(asset_id: u32, holders: Vec<[u8; 32]>) -> f64 { + let total_supply = ExtensionFungibles::total_supply(asset_id); + let holders_balance: u128 = holders.iter() + .map(|account| ExtensionFungibles::balance(asset_id, account)) + .sum(); + + (holders_balance as f64 / total_supply as f64) * 100.0 +} +``` + +## Related Components + +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Extension Core](../pvq-extension-core/) - Core functionality +- [PVQ Extension Swap](../pvq-extension-swap/) - DEX functionality +- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration + +--- + +*The Fungibles Extension provides comprehensive access to asset and balance data for PVQ programs.* \ No newline at end of file diff --git a/pvq-extension-swap/README.md b/pvq-extension-swap/README.md new file mode 100644 index 0000000..f935888 --- /dev/null +++ b/pvq-extension-swap/README.md @@ -0,0 +1,589 @@ +# PVQ Extension Swap + + + +A specialized extension for the PVQ (PolkaVM Query) system that enables guest programs to interact with decentralized exchange (DEX) and liquidity pool functionality. This extension provides comprehensive access to swap information, price quotes, and liquidity data. + +## Overview + +The Swap Extension integrates with DEX pallets and liquidity protocols to provide PVQ guest programs with read-only access to trading and liquidity information. It supports various AMM (Automated Market Maker) models and liquidity pool configurations. + +## Features + +- 🏊 **Pool Discovery**: List and query available liquidity pools +- πŸ’± **Price Quotes**: Get accurate swap price estimates +- πŸ“Š **Liquidity Information**: Query pool reserves and liquidity depth +- πŸ“ˆ **Trading Analytics**: Calculate slippage, fees, and trading metrics +- πŸ” **Route Finding**: Discover optimal trading paths +- ⚑ **Multi-DEX Support**: Support for various DEX implementations + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ ExtensionSwap β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Pools β”‚ β”‚ Pricing β”‚ β”‚ Analytics β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Pool List β”‚ β”‚ β”œβ”€ Quote β”‚ β”‚ β”œβ”€ Volume β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Reserves β”‚ β”‚ β”‚ Calculation β”‚ β”‚ β”œβ”€ Slippage β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Pool Info β”‚ β”‚ β”œβ”€ Price Impact β”‚ β”‚ β”œβ”€ Fees β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Liquidity β”‚ β”‚ β”œβ”€ Route β”‚ β”‚ β”œβ”€ APR/APY β”‚ β”‚ +β”‚ β”‚ β”‚ Depth β”‚ β”‚ β”‚ Discovery β”‚ β”‚ └─ Historicalβ”‚ β”‚ +β”‚ β”‚ └─ Pool Status β”‚ β”‚ └─ Multi-hop β”‚ β”‚ Data β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Routing β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## API Reference + +### Pool Discovery + +#### `list_pools() -> Vec` + +Get a list of all available liquidity pools. + +```rust +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn available_pools() -> Vec { + ExtensionSwap::list_pools() +} +``` + +#### `pool_exists(pool_id: PoolId) -> bool` + +Check if a specific pool exists. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +let pool_id = PoolId::new(1, 2); // Token pair (1, 2) +if ExtensionSwap::pool_exists(pool_id) { + // Pool exists +} +``` + +#### `pools_for_asset(asset_id: u32) -> Vec` + +Find all pools containing a specific asset. + +```rust +use pvq_extension_swap::ExtensionSwap; + +let pools = ExtensionSwap::pools_for_asset(asset_id); +println!("Found {} pools with asset {}", pools.len(), asset_id); +``` + +### Pool Information + +#### `pool_info(pool_id: PoolId) -> Option` + +Get comprehensive information about a pool. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some(info) = ExtensionSwap::pool_info(pool_id) { + println!("Pool reserves: {} / {}", info.reserve_0, info.reserve_1); + println!("Total liquidity: {}", info.total_liquidity); +} +``` + +#### `pool_reserves(pool_id: PoolId) -> Option<(u128, u128)>` + +Get current pool reserves for both assets. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some((reserve_a, reserve_b)) = ExtensionSwap::pool_reserves(pool_id) { + println!("Reserve A: {}, Reserve B: {}", reserve_a, reserve_b); +} +``` + +#### `pool_total_liquidity(pool_id: PoolId) -> Option` + +Get the total liquidity in a pool. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some(liquidity) = ExtensionSwap::pool_total_liquidity(pool_id) { + println!("Total liquidity: {}", liquidity); +} +``` + +### Price Quotation + +#### `quote_exact_in(asset_in: u32, asset_out: u32, amount_in: u128) -> Option` + +Get a quote for swapping an exact input amount. + +```rust +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn get_swap_quote(asset_in: u32, asset_out: u32, amount_in: u128) -> String { + match ExtensionSwap::quote_exact_in(asset_in, asset_out, amount_in) { + Some(quote) => { + format!( + "Output: {}, Price Impact: {:.2}%, Fee: {}", + quote.amount_out, quote.price_impact, quote.fee + ) + } + None => "No route available".to_string(), + } +} +``` + +#### `quote_exact_out(asset_in: u32, asset_out: u32, amount_out: u128) -> Option` + +Get a quote for receiving an exact output amount. + +```rust +use pvq_extension_swap::ExtensionSwap; + +if let Some(quote) = ExtensionSwap::quote_exact_out(asset_in, asset_out, amount_out) { + println!("Required input: {}", quote.amount_in); +} +``` + +#### `get_price(asset_in: u32, asset_out: u32) -> Option` + +Get the current price ratio between two assets. + +```rust +use pvq_extension_swap::ExtensionSwap; + +if let Some(price) = ExtensionSwap::get_price(asset_in, asset_out) { + println!("Price: 1 token {} = {} token {}", asset_in, price, asset_out); +} +``` + +### Trading Analytics + +#### `calculate_slippage(pool_id: PoolId, amount_in: u128) -> Option` + +Calculate expected slippage for a trade. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some(slippage) = ExtensionSwap::calculate_slippage(pool_id, amount_in) { + println!("Expected slippage: {:.2}%", slippage); +} +``` + +#### `pool_fee_rate(pool_id: PoolId) -> Option` + +Get the fee rate for a pool (in basis points). + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some(fee) = ExtensionSwap::pool_fee_rate(pool_id) { + println!("Pool fee: {:.2}%", fee as f64 / 100.0); +} +``` + +#### `pool_volume_24h(pool_id: PoolId) -> Option` + +Get 24-hour trading volume for a pool. + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +if let Some(volume) = ExtensionSwap::pool_volume_24h(pool_id) { + println!("24h volume: {}", volume); +} +``` + +### Route Discovery + +#### `find_route(asset_in: u32, asset_out: u32) -> Option>` + +Find the optimal trading route between two assets. + +```rust +use pvq_extension_swap::ExtensionSwap; + +if let Some(route) = ExtensionSwap::find_route(asset_in, asset_out) { + println!("Route found with {} hops", route.len()); +} else { + println!("No route available"); +} +``` + +#### `multi_hop_quote(route: Vec, amount_in: u128) -> Option` + +Get a quote for a multi-hop swap route. + +```rust +use pvq_extension_swap::ExtensionSwap; + +let route = ExtensionSwap::find_route(asset_in, asset_out)?; +if let Some(quote) = ExtensionSwap::multi_hop_quote(route, amount_in) { + println!("Multi-hop output: {}", quote.amount_out); +} +``` + +## Data Types + +### PoolId + +Unique identifier for a liquidity pool: + +```rust +pub struct PoolId { + pub asset_0: u32, + pub asset_1: u32, +} + +impl PoolId { + pub fn new(asset_0: u32, asset_1: u32) -> Self { + // Ensures asset_0 < asset_1 for consistency + if asset_0 < asset_1 { + Self { asset_0, asset_1 } + } else { + Self { asset_0: asset_1, asset_1: asset_0 } + } + } +} +``` + +### PoolInfo + +Comprehensive pool information: + +```rust +pub struct PoolInfo { + pub pool_id: PoolId, + pub reserve_0: u128, + pub reserve_1: u128, + pub total_liquidity: u128, + pub fee_rate: u32, // In basis points + pub is_active: bool, +} +``` + +### SwapQuote + +Detailed swap quotation: + +```rust +pub struct SwapQuote { + pub amount_in: u128, + pub amount_out: u128, + pub price_impact: f64, // Percentage + pub fee: u128, + pub route: Vec, + pub slippage_tolerance: f64, +} +``` + +## Usage Examples + +### Basic Pool Information + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +#[pvq_program::program] +fn pool_summary(asset_a: u32, asset_b: u32) -> String { + let pool_id = PoolId::new(asset_a, asset_b); + + match ExtensionSwap::pool_info(pool_id) { + Some(info) => { + format!( + "Pool {}-{}: Reserves: {} / {}, Liquidity: {}, Fee: {:.2}%", + asset_a, asset_b, + info.reserve_0, info.reserve_1, + info.total_liquidity, + info.fee_rate as f64 / 100.0 + ) + } + None => "Pool not found".to_string(), + } +} +``` + +### Price Comparison Across Pools + +```rust +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn compare_prices(asset_in: u32, asset_out: u32) -> Vec<(u32, f64)> { + let pools = ExtensionSwap::pools_for_asset(asset_in); + let mut prices = Vec::new(); + + for pool in pools { + if pool.asset_0 == asset_out || pool.asset_1 == asset_out { + if let Some(price) = ExtensionSwap::get_price(asset_in, asset_out) { + prices.push((pool.asset_0.max(pool.asset_1), price)); + } + } + } + + prices +} +``` + +### Arbitrage Opportunity Scanner + +```rust +use pvq_extension_swap::{ExtensionSwap, PoolId}; + +#[pvq_program::program] +fn find_arbitrage(asset_a: u32, asset_b: u32, amount: u128) -> f64 { + let pools = ExtensionSwap::list_pools(); + let mut best_profit = 0.0; + + for pool in pools { + if (pool.asset_0 == asset_a && pool.asset_1 == asset_b) || + (pool.asset_0 == asset_b && pool.asset_1 == asset_a) { + + // Check price in this pool vs others + if let Some(quote) = ExtensionSwap::quote_exact_in(asset_a, asset_b, amount) { + let return_quote = ExtensionSwap::quote_exact_in(asset_b, asset_a, quote.amount_out); + + if let Some(return_quote) = return_quote { + let profit = return_quote.amount_out as f64 - amount as f64; + let profit_pct = (profit / amount as f64) * 100.0; + + if profit_pct > best_profit { + best_profit = profit_pct; + } + } + } + } + } + + best_profit +} +``` + +### Liquidity Analysis + +```rust +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn liquidity_analysis(asset_id: u32) -> String { + let pools = ExtensionSwap::pools_for_asset(asset_id); + let mut total_liquidity = 0u128; + let mut active_pools = 0u32; + + for pool_id in pools { + if let Some(info) = ExtensionSwap::pool_info(pool_id) { + if info.is_active { + total_liquidity += info.total_liquidity; + active_pools += 1; + } + } + } + + format!( + "Asset {}: {} active pools, Total liquidity: {}", + asset_id, active_pools, total_liquidity + ) +} +``` + +### Best Route Finder + +```rust +use pvq_extension_swap::ExtensionSwap; + +#[pvq_program::program] +fn best_swap_route(asset_in: u32, asset_out: u32, amount: u128) -> String { + match ExtensionSwap::find_route(asset_in, asset_out) { + Some(route) => { + match ExtensionSwap::multi_hop_quote(route.clone(), amount) { + Some(quote) => { + format!( + "Best route: {} hops, Output: {}, Price Impact: {:.2}%", + route.len(), quote.amount_out, quote.price_impact + ) + } + None => "Route found but quote failed".to_string(), + } + } + None => "No route available".to_string(), + } +} +``` + +## Integration + +### With Runtime + +Configure the swap extension in your runtime: + +```rust +// In runtime configuration +impl pvq_extension_swap::Config for Runtime { + type AssetId = u32; + type Balance = u128; + type DexPallet = Dex; + type PoolId = PoolIdOf; +} +``` + +### With Extension Executor + +Register the swap extension: + +```rust +use pvq_extension::ExtensionsExecutor; +use pvq_extension_swap::ExtensionSwap; + +let mut executor = ExtensionsExecutor::new(); +executor.register::(); +``` + +## Configuration + +### DEX Integration + +Configure supported DEX types: + +```rust +impl pvq_extension_swap::Config for Runtime { + type SupportedDexes = ( + UniswapV2Pallet, + CurveStablePallet, + BalancerPallet, + ); +} +``` + +### Fee Configuration + +Set fee calculation parameters: + +```rust +impl pvq_extension_swap::Config for Runtime { + type DefaultFeeRate = ConstU32<30>; // 0.30% + type MaxSlippageTolerance = ConstU32<500>; // 5.00% +} +``` + +## Development + +### Building + +```bash +# Build swap extension +cargo build -p pvq-extension-swap + +# Run tests +cargo test -p pvq-extension-swap + +# Generate documentation +cargo doc -p pvq-extension-swap --open +``` + +### Testing + +```bash +# Unit tests +cargo test -p pvq-extension-swap + +# Integration tests with DEX pallets +cargo test -p pvq-extension-swap --test dex_integration + +# Test with guest programs +cargo run -p pvq-test-runner -- --program test-swap +``` + +### Benchmarking + +```bash +# Run performance benchmarks +cargo bench -p pvq-extension-swap + +# Profile quote calculations +cargo run --example profile_quotes --release +``` + +## Performance Considerations + +### Optimization Tips + +1. **Cache Pool Data**: Pool information changes infrequently +2. **Batch Route Discovery**: Find multiple routes in single queries +3. **Limit Route Depth**: Set maximum hops for complex routes +4. **Filter Inactive Pools**: Only query active pools + +### Query Patterns + +```rust +// Efficient: Batch multiple quotes +#[pvq_program::program] +fn batch_quotes(pairs: Vec<(u32, u32)>, amount: u128) -> Vec> { + pairs.iter() + .map(|(asset_in, asset_out)| { + ExtensionSwap::quote_exact_in(*asset_in, *asset_out, amount) + .map(|quote| quote.amount_out) + }) + .collect() +} +``` + +## Security Considerations + +- All functions are read-only and cannot execute trades +- Price quotes are estimates and may differ from actual execution +- Pool data reflects current state, which can change rapidly +- Route calculations consider current liquidity conditions + +## Common Use Cases + +### DEX Aggregation + +```rust +// Find best price across multiple DEXes +#[pvq_program::program] +fn best_price_aggregation(asset_in: u32, asset_out: u32, amount: u128) -> SwapQuote { + let all_routes = ExtensionSwap::find_all_routes(asset_in, asset_out); + + all_routes.into_iter() + .filter_map(|route| ExtensionSwap::multi_hop_quote(route, amount)) + .max_by_key(|quote| quote.amount_out) + .unwrap_or_default() +} +``` + +### Yield Farming Analysis + +```rust +// Calculate potential LP returns +#[pvq_program::program] +fn lp_yield_analysis(pool_id: PoolId) -> f64 { + let volume = ExtensionSwap::pool_volume_24h(pool_id).unwrap_or(0); + let liquidity = ExtensionSwap::pool_total_liquidity(pool_id).unwrap_or(1); + let fee_rate = ExtensionSwap::pool_fee_rate(pool_id).unwrap_or(0); + + // Simple APR calculation based on fees + let daily_fees = (volume * fee_rate as u128) / 10000; + let apr = (daily_fees as f64 / liquidity as f64) * 365.0 * 100.0; + + apr +} +``` + +## Related Components + +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Extension Core](../pvq-extension-core/) - Core functionality +- [PVQ Extension Fungibles](../pvq-extension-fungibles/) - Asset functionality +- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration + +--- + +*The Swap Extension provides comprehensive DEX and liquidity pool access for PVQ programs.* \ No newline at end of file diff --git a/pvq-extension/README.md b/pvq-extension/README.md new file mode 100644 index 0000000..7b6bdfb --- /dev/null +++ b/pvq-extension/README.md @@ -0,0 +1,552 @@ +# PVQ Extension System + + + +The foundational extension framework for PVQ (PolkaVM Query) that enables modular and secure interaction between guest programs and runtime functionality. This system provides the architecture for creating, registering, and executing extensions. + +## Overview + +The PVQ Extension System serves as the bridge between sandboxed guest programs and the Substrate runtime. It provides a secure, permission-controlled mechanism for exposing runtime functionality to PVQ programs through a modular extension architecture. + +## Features + +- 🧩 **Modular Architecture**: Plugin-based extension system +- πŸ”’ **Security Controls**: Fine-grained permission management +- πŸ“ **Procedural Macros**: Simplified extension development with `#[extension_decl]` and `#[extensions_impl]` +- ⚑ **High Performance**: Optimized call dispatch and execution +- πŸ›‘οΈ **Type Safety**: Compile-time validation of extension interfaces +- πŸ”Œ **Hot-Pluggable**: Runtime extension registration and discovery + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Extension System β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Extension β”‚ β”‚ Extensions β”‚ β”‚ Permission β”‚ β”‚ +β”‚ β”‚ Declaration β”‚ β”‚ Executor β”‚ β”‚ Controller β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Trait Def β”‚ β”‚ β”œβ”€ Registry β”‚ β”‚ β”œβ”€ Access β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Metadata β”‚ β”‚ β”œβ”€ Dispatch β”‚ β”‚ β”‚ Control β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Function β”‚ β”‚ β”œβ”€ Context β”‚ β”‚ β”œβ”€ Policy β”‚ β”‚ +β”‚ β”‚ β”‚ Signatures β”‚ β”‚ β”‚ Management β”‚ β”‚ β”‚ Engine β”‚ β”‚ +β”‚ β”‚ └─ Validation β”‚ β”‚ └─ Error β”‚ β”‚ └─ Audit β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Handling β”‚ β”‚ Trail β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Core Components + +### Extension Declaration (`#[extension_decl]`) + +Define new extensions using the procedural macro: + +```rust +use pvq_extension::extension_decl; + +#[extension_decl] +pub trait MyExtension { + /// Get system information + fn get_info() -> String; + + /// Process data with validation + fn process_data(input: Vec) -> Result, String>; + + /// Query with parameters + fn query_with_params(id: u32, filter: Option) -> Vec; +} +``` + +### Extension Implementation (`#[extensions_impl]`) + +Implement extensions for runtime integration: + +```rust +use pvq_extension::extensions_impl; + +pub struct MyExtensionImpl; + +#[extensions_impl] +impl MyExtension for MyExtensionImpl { + fn get_info() -> String { + "System v1.0".to_string() + } + + fn process_data(input: Vec) -> Result, String> { + if input.is_empty() { + Err("Empty input".to_string()) + } else { + Ok(input.into_iter().map(|b| b.wrapping_add(1)).collect()) + } + } + + fn query_with_params(id: u32, filter: Option) -> Vec { + // Implementation logic + vec![id] + } +} +``` + +### Extensions Executor + +Manages extension registration and execution: + +```rust +use pvq_extension::ExtensionsExecutor; + +let mut executor = ExtensionsExecutor::new(); + +// Register extensions +executor.register::(); +executor.register::(); + +// Execute extension calls from guest programs +let result = executor.execute_call("MyExtension", "get_info", &[])?; +``` + +### Permission Controller + +Controls access to extensions and functions: + +```rust +use pvq_extension::{PermissionController, Permission}; + +let mut controller = PermissionController::new(); + +// Grant specific permissions +controller.grant_permission( + "guest_program_id", + Permission::Extension("MyExtension".to_string()) +); + +// Check permissions before execution +if controller.check_permission("guest_program_id", &permission) { + // Execute extension call +} +``` + +## Usage Guide + +### Creating a New Extension + +1. **Define the Extension Trait:** + +```rust +use pvq_extension::extension_decl; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct CustomData { + pub value: u64, + pub name: String, +} + +#[extension_decl] +pub trait CustomExtension { + /// Get custom data by ID + fn get_data(id: u32) -> Option; + + /// List all available data IDs + fn list_data_ids() -> Vec; + + /// Validate data format + fn validate_data(data: CustomData) -> bool; +} +``` + +2. **Implement the Extension:** + +```rust +use pvq_extension::extensions_impl; + +pub struct CustomExtensionImpl { + _phantom: std::marker::PhantomData, +} + +#[extensions_impl] +impl CustomExtension for CustomExtensionImpl { + fn get_data(id: u32) -> Option { + // Access runtime storage + T::Storage::get_custom_data(id) + } + + fn list_data_ids() -> Vec { + T::Storage::list_custom_data_ids() + } + + fn validate_data(data: CustomData) -> bool { + !data.name.is_empty() && data.value > 0 + } +} +``` + +3. **Register in Runtime:** + +```rust +use pvq_extension::ExtensionsExecutor; + +impl pvq_runtime_api::PvqApi for Runtime { + fn execute_query(&self, program: Vec, args: Vec) -> Vec { + let mut extensions = ExtensionsExecutor::new(); + + // Register standard extensions + extensions.register::(); + extensions.register::(); + + // Register custom extension + extensions.register::(); + + // Execute program with extensions + let context = PvqExecutorContext::new(extensions); + let executor = PvqExecutor::new(context); + + executor.execute(&program, &args).unwrap_or_default() + } +} +``` + +### Using Extensions in Guest Programs + +```rust +use pvq_program::program; +use custom_extension::CustomExtension; + +#[program] +fn query_custom_data(id: u32) -> String { + match CustomExtension::get_data(id) { + Some(data) => { + format!("Data: {} = {}", data.name, data.value) + } + None => { + "Data not found".to_string() + } + } +} + +#[program] +fn validate_and_list() -> Vec { + let all_ids = CustomExtension::list_data_ids(); + + all_ids.into_iter() + .filter(|&id| { + if let Some(data) = CustomExtension::get_data(id) { + CustomExtension::validate_data(data) + } else { + false + } + }) + .collect() +} +``` + +## Advanced Features + +### Permission Management + +Configure fine-grained access control: + +```rust +use pvq_extension::{PermissionController, Permission, PermissionPolicy}; + +let mut controller = PermissionController::new(); + +// Create permission policy +let policy = PermissionPolicy::new() + .allow_extension("core") + .allow_extension("fungibles") + .deny_function("fungibles", "admin_function") + .with_rate_limit(100, Duration::from_secs(60)); + +// Apply policy to program +controller.set_policy("program_id", policy); +``` + +### Extension Metadata + +Generate and access extension metadata: + +```rust +use pvq_extension::{ExtensionMetadata, FunctionMetadata}; + +// Metadata is automatically generated from extension declarations +let metadata = ExtensionsExecutor::get_extension_metadata("CustomExtension"); + +match metadata { + Some(meta) => { + println!("Extension: {}", meta.name); + println!("Version: {}", meta.version); + + for function in meta.functions { + println!("Function: {}", function.name); + println!("Parameters: {:?}", function.parameters); + println!("Return Type: {:?}", function.return_type); + } + } + None => println!("Extension not found"), +} +``` + +### Error Handling + +Handle extension errors gracefully: + +```rust +use pvq_extension::{ExtensionError, ExtensionResult}; + +#[extension_decl] +pub trait SafeExtension { + fn safe_operation(input: u32) -> ExtensionResult; +} + +#[extensions_impl] +impl SafeExtension for SafeExtensionImpl { + fn safe_operation(input: u32) -> ExtensionResult { + if input == 0 { + Err(ExtensionError::InvalidInput("Input cannot be zero".to_string())) + } else if input > 1000 { + Err(ExtensionError::OutOfRange("Input too large".to_string())) + } else { + Ok(format!("Processed: {}", input)) + } + } +} +``` + +## Configuration + +### Runtime Configuration + +Configure the extension system in your runtime: + +```rust +impl pvq_extension::Config for Runtime { + type MaxExtensions = ConstU32<64>; + type MaxFunctionsPerExtension = ConstU32<32>; + type EnablePermissionControl = ConstBool; + type EnableMetadataGeneration = ConstBool; +} +``` + +### Extension Registration + +Register extensions with configuration: + +```rust +use pvq_extension::{ExtensionsExecutor, ExtensionConfig}; + +let mut executor = ExtensionsExecutor::new(); + +let config = ExtensionConfig { + enable_caching: true, + max_call_depth: 10, + timeout: Duration::from_secs(30), +}; + +executor.register_with_config::(config); +``` + +## Development Tools + +### Testing Extensions + +```rust +#[cfg(test)] +mod tests { + use super::*; + use pvq_extension::testing::ExtensionTester; + + #[test] + fn test_extension_functionality() { + let tester = ExtensionTester::new(); + tester.register::(); + + // Test extension calls + let result = tester.call("CustomExtension", "get_data", &[1u32]); + assert!(result.is_ok()); + } + + #[test] + fn test_permission_enforcement() { + let mut tester = ExtensionTester::new(); + tester.set_permission_policy(restrictive_policy()); + + // This should fail due to permissions + let result = tester.call("CustomExtension", "admin_function", &[]); + assert!(matches!(result, Err(ExtensionError::PermissionDenied(_)))); + } +} +``` + +### Debugging + +Enable detailed logging: + +```rust +use log::debug; + +#[extensions_impl] +impl CustomExtension for CustomExtensionImpl { + fn debug_function(input: String) -> String { + debug!("CustomExtension::debug_function called with: {}", input); + + let result = process_input(&input); + + debug!("CustomExtension::debug_function returning: {}", result); + result + } +} +``` + +Run with debug logging: + +```bash +RUST_LOG=pvq_extension=debug,custom_extension=debug cargo test +``` + +## Performance Considerations + +### Optimization Tips + +1. **Minimize Allocations**: Use stack allocation when possible +2. **Batch Operations**: Group related calls into single functions +3. **Cache Results**: Cache expensive computations within extensions +4. **Limit Recursion**: Avoid deep call chains between extensions + +### Benchmarking + +```bash +# Benchmark extension performance +cargo bench -p pvq-extension + +# Profile extension calls +cargo run --example profile_extensions --release +``` + +### Example Optimizations + +```rust +// Efficient: Batch operation +#[extension_decl] +pub trait OptimizedExtension { + fn batch_process(items: Vec) -> Vec; +} + +// Less efficient: Individual calls +#[extension_decl] +pub trait UnoptimizedExtension { + fn process_single(item: ProcessItem) -> ProcessResult; +} +``` + +## Security Best Practices + +### Input Validation + +Always validate inputs in extension implementations: + +```rust +#[extensions_impl] +impl SecureExtension for SecureExtensionImpl { + fn secure_function(user_input: String) -> Result { + // Validate input length + if user_input.len() > 1000 { + return Err("Input too long".to_string()); + } + + // Validate input format + if !user_input.chars().all(|c| c.is_alphanumeric()) { + return Err("Invalid characters".to_string()); + } + + // Safe processing + Ok(process_validated_input(&user_input)) + } +} +``` + +### Permission Boundaries + +Implement proper access controls: + +```rust +use pvq_extension::{Permission, PermissionController}; + +fn setup_permissions() -> PermissionController { + let mut controller = PermissionController::new(); + + // Public functions - accessible to all programs + controller.grant_global_permission(Permission::Function( + "PublicExtension".to_string(), + "public_query".to_string() + )); + + // Admin functions - restricted access + controller.grant_permission( + "admin_program_id", + Permission::Function( + "AdminExtension".to_string(), + "admin_function".to_string() + ) + ); + + controller +} +``` + +## Migration Guide + +### Upgrading Extensions + +When updating extension interfaces: + +1. **Version your extensions:** + +```rust +#[extension_decl] +pub trait MyExtensionV2 { + // Keep old functions for compatibility + fn old_function() -> String; + + // Add new functions + fn new_function() -> EnhancedResult; +} +``` + +2. **Provide migration path:** + +```rust +// Implement both versions +impl MyExtension for MyExtensionImpl { /* old implementation */ } +impl MyExtensionV2 for MyExtensionImpl { /* new implementation */ } +``` + +### Breaking Changes + +Handle breaking changes gracefully: + +```rust +#[extension_decl] +pub trait VersionedExtension { + fn get_version() -> u32 { 2 } + + fn function_v2(input: NewInputType) -> NewOutputType; + + // Deprecated but maintained for compatibility + #[deprecated] + fn function_v1(input: OldInputType) -> OldOutputType; +} +``` + +## Related Components + +- [PVQ Extension Core](../pvq-extension-core/) - Core extension functionality +- [PVQ Extension Fungibles](../pvq-extension-fungibles/) - Asset functionality +- [PVQ Extension Swap](../pvq-extension-swap/) - DEX functionality +- [PVQ Executor](../pvq-executor/) - Execution environment +- [PVQ Program](../pvq-program/) - Program development tools + +--- + +*The PVQ Extension System provides the foundation for modular, secure runtime interaction in PVQ programs.* \ No newline at end of file diff --git a/pvq-primitives/README.md b/pvq-primitives/README.md new file mode 100644 index 0000000..ebb2d5e --- /dev/null +++ b/pvq-primitives/README.md @@ -0,0 +1,703 @@ +# PVQ Primitives + + + +Core primitive types and utilities for the PVQ (PolkaVM Query) ecosystem. This crate provides fundamental data structures, error types, and serialization utilities that ensure interoperability across all PVQ components. + +## Overview + +PVQ Primitives serves as the foundational layer for all PVQ components, providing standardized types and utilities that enable seamless communication between guest programs, extensions, executors, and runtime APIs. + +## Features + +- πŸ“¦ **Core Data Types**: Fundamental types used across the PVQ ecosystem +- πŸ”„ **Serialization Support**: Efficient encoding/decoding with multiple formats +- ❌ **Error Handling**: Comprehensive error types with detailed context +- πŸ”— **Interoperability**: Consistent types across all PVQ components +- πŸ“ **Size Optimization**: Compact representations for performance +- πŸ›‘οΈ **Type Safety**: Strong typing to prevent runtime errors + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Primitives β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Core Types β”‚ β”‚ Error Types β”‚ β”‚ Utilities β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ PvqResult β”‚ β”‚ β”œβ”€ PvqError β”‚ β”‚ β”œβ”€ Codec β”‚ β”‚ +β”‚ β”‚ β”œβ”€ PvqResponse β”‚ β”‚ β”œβ”€ ErrorCode β”‚ β”‚ β”œβ”€ Hashing β”‚ β”‚ +β”‚ β”‚ β”œβ”€ QueryData β”‚ β”‚ β”œβ”€ ErrorContext β”‚ β”‚ β”œβ”€ Validationβ”‚ β”‚ +β”‚ β”‚ β”œβ”€ Metadata β”‚ β”‚ └─ ErrorChain β”‚ β”‚ └─ Constants β”‚ β”‚ +β”‚ β”‚ └─ Context β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Core Types + +### PvqResult + +The standard result type for PVQ operations: + +```rust +use pvq_primitives::{PvqResult, PvqError}; + +pub type PvqResult = Result; + +// Usage example +fn example_function() -> PvqResult { + Ok("Success".to_string()) +} + +// Error handling +match example_function() { + Ok(value) => println!("Success: {}", value), + Err(error) => println!("Error: {}", error), +} +``` + +### PvqResponse + +Standard response wrapper for query results: + +```rust +use pvq_primitives::{PvqResponse, QueryMetadata}; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct PvqResponse { + pub data: T, + pub metadata: QueryMetadata, + pub success: bool, +} + +impl PvqResponse { + pub fn success(data: T) -> Self { + Self { + data, + metadata: QueryMetadata::default(), + success: true, + } + } + + pub fn error(data: T, error: String) -> Self { + Self { + data, + metadata: QueryMetadata::with_error(error), + success: false, + } + } +} + +// Usage in guest programs +#[pvq_program::program] +fn my_query() -> PvqResponse { + PvqResponse::success("Query completed".to_string()) +} +``` + +### QueryData + +Standardized input/output data format: + +```rust +use pvq_primitives::QueryData; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct QueryData { + pub payload: T, + pub version: u32, + pub timestamp: u64, +} + +impl QueryData { + pub fn new(payload: T) -> Self { + Self { + payload, + version: 1, + timestamp: current_timestamp(), + } + } +} +``` + +## Error Handling + +### PvqError + +Comprehensive error type covering all failure scenarios: + +```rust +use pvq_primitives::PvqError; +use thiserror::Error; + +#[derive(Error, Debug, Clone, PartialEq)] +pub enum PvqError { + #[error("Program execution failed: {message}")] + ExecutionError { + message: String, + code: u32, + }, + + #[error("Extension call failed: {extension}.{function}")] + ExtensionError { + extension: String, + function: String, + reason: String, + }, + + #[error("Serialization error: {0}")] + SerializationError(String), + + #[error("Permission denied: {operation}")] + PermissionDenied { + operation: String + }, + + #[error("Resource limit exceeded: {resource}")] + ResourceLimitExceeded { + resource: String, + limit: u64, + requested: u64, + }, + + #[error("Invalid input: {field}")] + InvalidInput { + field: String, + expected: String, + received: String, + }, + + #[error("Timeout after {duration_ms}ms")] + Timeout { + duration_ms: u64 + }, + + #[error("Internal error: {0}")] + Internal(String), +} +``` + +### Error Construction Utilities + +Helper functions for creating common errors: + +```rust +use pvq_primitives::PvqError; + +impl PvqError { + pub fn execution_failed(message: impl Into) -> Self { + Self::ExecutionError { + message: message.into(), + code: 1001, + } + } + + pub fn extension_failed( + extension: impl Into, + function: impl Into, + reason: impl Into, + ) -> Self { + Self::ExtensionError { + extension: extension.into(), + function: function.into(), + reason: reason.into(), + } + } + + pub fn invalid_input( + field: impl Into, + expected: impl Into, + received: impl Into, + ) -> Self { + Self::InvalidInput { + field: field.into(), + expected: expected.into(), + received: received.into(), + } + } +} + +// Usage +fn validate_input(value: u32) -> PvqResult { + if value == 0 { + Err(PvqError::invalid_input( + "value", + "non-zero number", + "0" + )) + } else { + Ok(value) + } +} +``` + +## Metadata Types + +### QueryMetadata + +Metadata about query execution: + +```rust +use pvq_primitives::QueryMetadata; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct QueryMetadata { + pub execution_time_ms: u64, + pub memory_used_bytes: u64, + pub extension_calls: u32, + pub block_number: u32, + pub error_message: Option, +} + +impl QueryMetadata { + pub fn new() -> Self { + Self { + execution_time_ms: 0, + memory_used_bytes: 0, + extension_calls: 0, + block_number: 0, + error_message: None, + } + } + + pub fn with_error(error: String) -> Self { + Self { + error_message: Some(error), + ..Self::new() + } + } +} +``` + +### ExtensionMetadata + +Metadata for extension functions: + +```rust +use pvq_primitives::ExtensionMetadata; + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct ExtensionMetadata { + pub name: String, + pub version: String, + pub description: String, + pub functions: Vec, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct FunctionMetadata { + pub name: String, + pub parameters: Vec, + pub return_type: TypeInfo, + pub description: String, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct ParameterInfo { + pub name: String, + pub type_info: TypeInfo, + pub optional: bool, + pub description: String, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct TypeInfo { + pub name: String, + pub size_hint: Option, + pub is_collection: bool, +} +``` + +## Serialization Utilities + +### Codec Support + +Support for multiple serialization formats: + +```rust +use pvq_primitives::codec::{Encode, Decode, Codec}; +use serde::{Serialize, Deserialize}; + +// Types that work with both SCALE and Serde +#[derive(Encode, Decode, Serialize, Deserialize)] +pub struct MyData { + pub value: u64, + pub name: String, +} + +// Codec utilities +pub fn encode_data(data: &T) -> Vec { + data.encode() +} + +pub fn decode_data(bytes: &[u8]) -> PvqResult { + T::decode(&mut &bytes[..]) + .map_err(|e| PvqError::SerializationError(e.to_string())) +} + +// JSON utilities +pub fn to_json(data: &T) -> PvqResult { + serde_json::to_string(data) + .map_err(|e| PvqError::SerializationError(e.to_string())) +} + +pub fn from_json Deserialize<'de>>(json: &str) -> PvqResult { + serde_json::from_str(json) + .map_err(|e| PvqError::SerializationError(e.to_string())) +} +``` + +### Custom Serialization + +For performance-critical applications: + +```rust +use pvq_primitives::codec::{Encode, Decode, Error}; + +pub struct CompactData { + pub numbers: Vec, + pub flags: u8, +} + +impl Encode for CompactData { + fn encode(&self) -> Vec { + let mut bytes = Vec::new(); + + // Encode length + data + bytes.extend((self.numbers.len() as u32).to_le_bytes()); + for &num in &self.numbers { + bytes.extend(num.to_le_bytes()); + } + bytes.push(self.flags); + + bytes + } +} + +impl Decode for CompactData { + fn decode(input: &mut I) -> Result { + let len = u32::decode(input)? as usize; + let mut numbers = Vec::with_capacity(len); + + for _ in 0..len { + numbers.push(u32::decode(input)?); + } + + let flags = u8::decode(input)?; + + Ok(Self { numbers, flags }) + } +} +``` + +## Constants and Configuration + +### System Constants + +Common constants used across PVQ: + +```rust +use pvq_primitives::constants::*; + +// Execution limits +pub const MAX_EXECUTION_TIME_MS: u64 = 30_000; +pub const MAX_MEMORY_BYTES: u64 = 64 * 1024 * 1024; // 64MB +pub const MAX_CALL_DEPTH: u32 = 256; + +// Data limits +pub const MAX_INPUT_SIZE_BYTES: usize = 1024 * 1024; // 1MB +pub const MAX_OUTPUT_SIZE_BYTES: usize = 10 * 1024 * 1024; // 10MB +pub const MAX_EXTENSION_NAME_LENGTH: usize = 64; + +// Protocol versions +pub const PVQ_PROTOCOL_VERSION: u32 = 1; +pub const MIN_SUPPORTED_VERSION: u32 = 1; +pub const MAX_SUPPORTED_VERSION: u32 = 1; +``` + +### Configuration Types + +Runtime configuration structures: + +```rust +use pvq_primitives::config::*; + +#[derive(Debug, Clone)] +pub struct PvqConfig { + pub max_execution_time_ms: u64, + pub max_memory_bytes: u64, + pub max_extensions: u32, + pub enable_debugging: bool, + pub permission_model: PermissionModel, +} + +impl Default for PvqConfig { + fn default() -> Self { + Self { + max_execution_time_ms: MAX_EXECUTION_TIME_MS, + max_memory_bytes: MAX_MEMORY_BYTES, + max_extensions: 32, + enable_debugging: false, + permission_model: PermissionModel::Strict, + } + } +} + +#[derive(Debug, Clone)] +pub enum PermissionModel { + Open, // All extensions allowed + Strict, // Explicit permission required + Sandbox, // Limited core functionality only +} +``` + +## Validation Utilities + +### Input Validation + +Common validation functions: + +```rust +use pvq_primitives::validation::*; + +pub fn validate_account_id(account: &[u8]) -> PvqResult<()> { + if account.len() != 32 { + return Err(PvqError::invalid_input( + "account_id", + "32 bytes", + &format!("{} bytes", account.len()) + )); + } + Ok(()) +} + +pub fn validate_asset_id(asset_id: u32) -> PvqResult<()> { + if asset_id == 0 { + return Err(PvqError::invalid_input( + "asset_id", + "non-zero value", + "0" + )); + } + Ok(()) +} + +pub fn validate_program_size(program: &[u8]) -> PvqResult<()> { + if program.len() > MAX_INPUT_SIZE_BYTES { + return Err(PvqError::ResourceLimitExceeded { + resource: "program_size".to_string(), + limit: MAX_INPUT_SIZE_BYTES as u64, + requested: program.len() as u64, + }); + } + Ok(()) +} + +// Usage in extensions +#[extension_decl] +pub trait ValidatedExtension { + fn safe_function(account: [u8; 32], asset_id: u32) -> PvqResult { + validate_account_id(&account)?; + validate_asset_id(asset_id)?; + + // Safe to proceed + Ok("Validation passed".to_string()) + } +} +``` + +## Usage Examples + +### Basic Error Handling + +```rust +use pvq_primitives::{PvqResult, PvqError, PvqResponse}; + +#[pvq_program::program] +fn safe_query(input: u32) -> PvqResponse { + match process_input(input) { + Ok(result) => PvqResponse::success(result), + Err(error) => PvqResponse::error( + String::new(), + error.to_string() + ), + } +} + +fn process_input(input: u32) -> PvqResult { + if input == 0 { + return Err(PvqError::invalid_input( + "input", + "positive number", + "0" + )); + } + + Ok(format!("Processed: {}", input)) +} +``` + +### Complex Data Structures + +```rust +use pvq_primitives::{QueryData, QueryMetadata}; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct AnalysisQuery { + pub assets: Vec, + pub accounts: Vec<[u8; 32]>, + pub include_metadata: bool, +} + +#[derive(Serialize, Deserialize)] +pub struct AnalysisResult { + pub total_value: u128, + pub asset_distribution: Vec<(u32, u128)>, + pub top_holders: Vec<([u8; 32], u128)>, +} + +#[pvq_program::program] +fn complex_analysis(query: QueryData) -> PvqResponse { + let start_time = current_timestamp(); + + // Process query + let result = perform_analysis(&query.payload)?; + + // Create response with metadata + let mut response = PvqResponse::success(result); + response.metadata.execution_time_ms = current_timestamp() - start_time; + + response +} +``` + +## Integration + +### Runtime Integration + +Configure primitives in your runtime: + +```rust +impl pvq_primitives::Config for Runtime { + type MaxQuerySize = ConstU32<1048576>; // 1MB + type MaxResponseSize = ConstU32<10485760>; // 10MB + type ErrorStringLimit = ConstU32<256>; +} +``` + +### Extension Integration + +Use primitives in extensions: + +```rust +use pvq_primitives::{PvqResult, PvqError, ExtensionMetadata}; + +#[extensions_impl] +impl MyExtension for MyExtensionImpl { + fn get_metadata() -> ExtensionMetadata { + ExtensionMetadata { + name: "MyExtension".to_string(), + version: "1.0.0".to_string(), + description: "Custom extension example".to_string(), + functions: vec![ + // Function metadata... + ], + } + } + + fn safe_operation(input: u32) -> PvqResult { + if input > 1000 { + return Err(PvqError::invalid_input( + "input", + "value <= 1000", + &input.to_string() + )); + } + + Ok(format!("Result: {}", input * 2)) + } +} +``` + +## Development + +### Building + +```bash +# Build primitives +cargo build -p pvq-primitives + +# Run tests +cargo test -p pvq-primitives + +# Check documentation +cargo doc -p pvq-primitives --open +``` + +### Testing + +```bash +# Unit tests +cargo test -p pvq-primitives + +# Integration tests +cargo test -p pvq-primitives --test integration + +# Test serialization performance +cargo bench -p pvq-primitives +``` + +## Performance Considerations + +### Memory Efficiency + +- Use compact serialization formats when possible +- Implement custom `Encode`/`Decode` for hot paths +- Avoid unnecessary allocations in error paths + +### Error Handling + +- Error construction should be lightweight +- Consider error code enums for frequently occurring errors +- Provide context without excessive string allocation + +```rust +// Efficient error handling +#[derive(Debug, Clone, Copy)] +pub enum ErrorCode { + InvalidInput = 1001, + PermissionDenied = 1002, + ResourceExhausted = 1003, + Timeout = 1004, +} + +impl From for PvqError { + fn from(code: ErrorCode) -> Self { + match code { + ErrorCode::InvalidInput => PvqError::InvalidInput { + field: "unknown".to_string(), + expected: "valid input".to_string(), + received: "invalid".to_string(), + }, + // ... other mappings + } + } +} +``` + +## Related Components + +- [PVQ Executor](../pvq-executor/) - Execution environment +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Program](../pvq-program/) - Program development +- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration + +--- + +*PVQ Primitives provide the essential building blocks for all PVQ system components.* \ No newline at end of file diff --git a/pvq-program/README.md b/pvq-program/README.md new file mode 100644 index 0000000..4264ea9 --- /dev/null +++ b/pvq-program/README.md @@ -0,0 +1,675 @@ +# PVQ Program + + + +Development tools and procedural macros for creating PVQ (PolkaVM Query) guest programs. This crate provides the `#[program]` macro and supporting utilities that simplify writing queries that run inside PolkaVM. + +## Overview + +PVQ Program provides the essential tools for developing guest programs that execute within the PVQ system. The `#[program]` macro handles all the boilerplate code required to make your functions executable by the PVQ executor, while additional utilities help with development and debugging. + +## Features + +- πŸš€ **`#[program]` Macro**: Automatically generates program entry points and serialization code +- πŸ“¦ **Zero Boilerplate**: Focus on query logic, not infrastructure +- πŸ”„ **Automatic Serialization**: Seamless input/output parameter handling +- ⚑ **Optimized Code Generation**: Efficient PolkaVM-compatible code +- πŸ› οΈ **Development Tools**: Utilities for testing and debugging programs +- πŸ“‹ **Metadata Generation**: Automatic program metadata for UI integration + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Program β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Program β”‚ β”‚ Code Gen β”‚ β”‚ Dev Tools β”‚ β”‚ +β”‚ β”‚ Macro β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”œβ”€ Entry Point β”‚ β”‚ β”œβ”€ Testing β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Parsing β”‚ β”‚ β”œβ”€ Serializationβ”‚ β”‚ β”œβ”€ Debugging β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Validation β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ β”œβ”€ Profiling β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Transform β”‚ β”‚ β”‚ Handling β”‚ β”‚ └─ Metadata β”‚ β”‚ +β”‚ β”‚ └─ Generation β”‚ β”‚ └─ Export β”‚ β”‚ Generation β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Functions β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Getting Started + +### Basic Program + +The simplest PVQ program using the `#[program]` macro: + +```rust +use pvq_program::program; + +#[program] +fn hello_world() -> String { + "Hello from PVQ!".to_string() +} +``` + +This generates all the necessary boilerplate to make your function executable as a PVQ program. + +### Program with Parameters + +Programs can accept parameters that are automatically deserialized: + +```rust +use pvq_program::program; + +#[program] +fn add_numbers(a: u32, b: u32) -> u32 { + a + b +} + +#[program] +fn process_account(account_id: [u8; 32], multiplier: u64) -> u64 { + // Your logic here + account_id[0] as u64 * multiplier +} +``` + +### Complex Data Types + +Programs support complex input and output types: + +```rust +use pvq_program::program; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct QueryInput { + pub asset_ids: Vec, + pub account: [u8; 32], + pub include_metadata: bool, +} + +#[derive(Serialize, Deserialize)] +pub struct QueryResult { + pub total_balance: u128, + pub asset_balances: Vec<(u32, u128)>, + pub metadata: Option, +} + +#[program] +fn complex_query(input: QueryInput) -> QueryResult { + // Implementation here + QueryResult { + total_balance: 1000, + asset_balances: vec![(1, 500), (2, 500)], + metadata: if input.include_metadata { + Some("Sample metadata".to_string()) + } else { + None + }, + } +} +``` + +## Program Development + +### Using Extensions + +Programs can call extension functions to interact with the runtime: + +```rust +use pvq_program::program; +use pvq_extension_core::ExtensionCore; +use pvq_extension_fungibles::ExtensionFungibles; + +#[program] +fn get_balance_with_info(asset_id: u32, account: [u8; 32]) -> String { + // Check if fungibles extension is available + if !ExtensionCore::extension_exists("fungibles") { + return "Fungibles extension not available".to_string(); + } + + // Get balance using extension + let balance = ExtensionFungibles::balance(asset_id, &account); + let block = ExtensionCore::current_block_number(); + + format!("Balance: {} at block {}", balance, block) +} +``` + +### Error Handling + +Programs can return `Result` types for error handling: + +```rust +use pvq_program::program; +use pvq_primitives::PvqResult; + +#[program] +fn safe_divide(a: u64, b: u64) -> PvqResult { + if b == 0 { + Err(pvq_primitives::PvqError::invalid_input( + "divisor", + "non-zero value", + "0" + )) + } else { + Ok(a / b) + } +} +``` + +### Advanced Programs + +More complex programs with multiple operations: + +```rust +use pvq_program::program; +use pvq_extension_fungibles::ExtensionFungibles; +use pvq_extension_swap::ExtensionSwap; +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize)] +pub struct PortfolioAnalysis { + pub total_value: u128, + pub asset_count: u32, + pub largest_holding: (u32, u128), + pub trading_opportunities: Vec, +} + +#[program] +fn analyze_portfolio(account: [u8; 32]) -> PortfolioAnalysis { + let mut analysis = PortfolioAnalysis { + total_value: 0, + asset_count: 0, + largest_holding: (0, 0), + trading_opportunities: Vec::new(), + }; + + // Get all assets + let assets = ExtensionFungibles::list_assets(); + + for asset_id in assets { + let balance = ExtensionFungibles::balance(asset_id, &account); + + if balance > 0 { + analysis.total_value += balance; + analysis.asset_count += 1; + + // Track largest holding + if balance > analysis.largest_holding.1 { + analysis.largest_holding = (asset_id, balance); + } + + // Look for trading opportunities + if balance > 1000 { + let pools = ExtensionSwap::pools_for_asset(asset_id); + if !pools.is_empty() { + analysis.trading_opportunities.push( + format!("Consider trading asset {} (balance: {})", asset_id, balance) + ); + } + } + } + } + + analysis +} +``` + +## Program Attributes + +### Basic Attributes + +The `#[program]` macro supports various attributes: + +```rust +use pvq_program::program; + +// Basic program +#[program] +fn basic_function() -> u32 { 42 } + +// Program with custom name (affects metadata) +#[program(name = "custom_name")] +fn function_with_custom_name() -> String { + "Custom named function".to_string() +} + +// Program with description +#[program(description = "This function demonstrates documentation")] +fn documented_function() -> String { + "Documented function".to_string() +} +``` + +### Performance Attributes + +```rust +use pvq_program::program; + +// Optimize for size +#[program(optimize = "size")] +fn size_optimized() -> u32 { 123 } + +// Optimize for speed +#[program(optimize = "speed")] +fn speed_optimized() -> u64 { 456 } + +// Set maximum execution time (milliseconds) +#[program(timeout = 5000)] +fn time_limited() -> String { + "Completes within 5 seconds".to_string() +} +``` + +### Memory Attributes + +```rust +use pvq_program::program; + +// Set maximum memory usage (bytes) +#[program(max_memory = 1048576)] // 1MB +fn memory_limited() -> Vec { + vec![1, 2, 3, 4, 5] +} + +// Stack size configuration +#[program(stack_size = 65536)] // 64KB +fn custom_stack() -> String { + "Custom stack size".to_string() +} +``` + +## Development Tools + +### Testing Programs + +PVQ Program provides utilities for testing programs locally: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use pvq_program::testing::ProgramTester; + + #[test] + fn test_my_program() { + let tester = ProgramTester::new(); + + // Test basic function + let result = tester.call_program(hello_world, ()); + assert_eq!(result, "Hello from PVQ!"); + + // Test function with parameters + let result = tester.call_program(add_numbers, (10, 20)); + assert_eq!(result, 30); + } + + #[test] + fn test_with_extensions() { + let mut tester = ProgramTester::new(); + tester.enable_extension("core"); + tester.enable_extension("fungibles"); + + // Mock extension responses + tester.mock_extension_call( + "fungibles", + "balance", + vec![1u32, [0u8; 32]], + 1000u128 + ); + + let result = tester.call_program(get_balance_with_info, (1u32, [0u8; 32])); + assert!(result.contains("Balance: 1000")); + } +} +``` + +### Debugging Support + +Enable debugging features during development: + +```rust +use pvq_program::{program, debug}; + +#[program] +fn debug_example(input: u32) -> u32 { + debug!("Input value: {}", input); + + let result = input * 2; + debug!("Calculated result: {}", result); + + result +} + +// Run with debugging enabled +// RUST_LOG=pvq_program=debug cargo run +``` + +### Profiling + +Profile program performance: + +```rust +use pvq_program::{program, profile}; + +#[program] +fn performance_example() -> u64 { + let _timer = profile!("main_calculation"); + + let mut result = 0u64; + for i in 0..1000000 { + result += i; + } + + result +} +``` + +## Metadata Generation + +### Automatic Metadata + +The macro automatically generates metadata for UI integration: + +```rust +use pvq_program::program; + +/// Calculate the sum of account balances for multiple assets +/// +/// # Parameters +/// - `accounts`: List of account IDs to check +/// - `asset_ids`: List of assets to include in calculation +/// +/// # Returns +/// Total balance across all accounts and assets +#[program( + name = "multi_asset_balance", + version = "1.0.0", + author = "Developer Name" +)] +fn calculate_multi_asset_balance( + accounts: Vec<[u8; 32]>, + asset_ids: Vec +) -> u128 { + // Implementation + 0 +} +``` + +This generates metadata that can be used by UIs to provide documentation and type information. + +### Custom Metadata + +Add custom metadata fields: + +```rust +use pvq_program::program; + +#[program( + name = "advanced_query", + description = "Performs advanced portfolio analysis", + version = "2.1.0", + author = "Portfolio Team", + tags = ["finance", "portfolio", "analysis"], + complexity = "high", + estimated_gas = 50000 +)] +fn advanced_portfolio_query() -> String { + "Advanced analysis result".to_string() +} +``` + +## Build Integration + +### Cargo Configuration + +Add to your `Cargo.toml`: + +```toml +[dependencies] +pvq-program = { version = "0.1.0", default-features = false } +pvq-extension-core = { version = "0.1.0", default-features = false } +pvq-extension-fungibles = { version = "0.1.0", default-features = false } +pvq-primitives = { version = "0.1.0", default-features = false } +serde = { version = "1.0", default-features = false, features = ["derive"] } + +[features] +default = [] +std = [ + "pvq-program/std", + "pvq-primitives/std", + "serde/std", +] +``` + +### Build Script Integration + +For metadata generation, add `build.rs`: + +```rust +use std::env; +use std::path::PathBuf; +use std::process::Command; + +fn main() { + // Generate program metadata + let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); + + let status = Command::new("pvq-program-metadata-gen") + .arg("--crate-path") + .arg(env::current_dir().unwrap()) + .arg("--output-dir") + .arg(&out_dir) + .status() + .expect("Failed to run metadata generator"); + + if !status.success() { + panic!("Metadata generation failed"); + } + + println!("cargo:rerun-if-changed=src/"); +} +``` + +## Advanced Usage + +### Custom Serialization + +For performance-critical programs, implement custom serialization: + +```rust +use pvq_program::{program, custom_serde}; + +#[derive(Clone)] +pub struct CustomData { + pub values: Vec, + pub flags: u32, +} + +impl custom_serde::Serialize for CustomData { + fn serialize(&self) -> Vec { + let mut bytes = Vec::new(); + bytes.extend((self.values.len() as u32).to_le_bytes()); + for &val in &self.values { + bytes.extend(val.to_le_bytes()); + } + bytes.extend(self.flags.to_le_bytes()); + bytes + } +} + +impl custom_serde::Deserialize for CustomData { + fn deserialize(bytes: &[u8]) -> Result { + if bytes.len() < 8 { + return Err("Insufficient data".to_string()); + } + + let len = u32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]) as usize; + let mut values = Vec::with_capacity(len); + + for i in 0..len { + let start = 4 + i * 8; + if start + 8 > bytes.len() { + return Err("Truncated data".to_string()); + } + let val = u64::from_le_bytes([ + bytes[start], bytes[start + 1], bytes[start + 2], bytes[start + 3], + bytes[start + 4], bytes[start + 5], bytes[start + 6], bytes[start + 7], + ]); + values.push(val); + } + + let flags_start = 4 + len * 8; + if flags_start + 4 > bytes.len() { + return Err("No flags data".to_string()); + } + + let flags = u32::from_le_bytes([ + bytes[flags_start], bytes[flags_start + 1], + bytes[flags_start + 2], bytes[flags_start + 3], + ]); + + Ok(CustomData { values, flags }) + } +} + +#[program] +fn process_custom_data(data: CustomData) -> u64 { + data.values.iter().sum::() + data.flags as u64 +} +``` + +### Multi-Function Programs + +Create programs with multiple entry points: + +```rust +use pvq_program::{program, program_module}; + +#[program_module] +mod my_programs { + use super::*; + + #[program] + pub fn query_balance(account: [u8; 32]) -> u128 { + // Implementation + 0 + } + + #[program] + pub fn query_metadata(asset_id: u32) -> String { + // Implementation + "Metadata".to_string() + } + + #[program] + pub fn complex_analysis(params: AnalysisParams) -> AnalysisResult { + // Implementation + AnalysisResult::default() + } +} +``` + +## Performance Optimization + +### Size Optimization + +For size-constrained environments: + +```rust +use pvq_program::program; + +// Minimize code size +#[program(optimize = "size", inline_threshold = 0)] +fn minimal_program() -> u32 { + 42 +} + +// Avoid large dependencies +#[program] +fn efficient_program(input: u32) -> u32 { + // Use simple operations instead of complex libraries + input.wrapping_mul(2).wrapping_add(1) +} +``` + +### Speed Optimization + +For performance-critical programs: + +```rust +use pvq_program::program; + +// Optimize for speed +#[program(optimize = "speed", inline_aggressive = true)] +fn fast_program(data: Vec) -> u64 { + data.into_iter() + .map(|x| x as u64) + .sum() +} + +// Use efficient algorithms +#[program] +fn optimized_search(haystack: Vec, needle: u32) -> Option { + // Binary search for sorted data + haystack.binary_search(&needle).ok() +} +``` + +## Troubleshooting + +### Common Issues + +**Program won't compile:** +```bash +# Check for syntax errors in macro usage +cargo check + +# Enable detailed error reporting +RUST_BACKTRACE=1 cargo build +``` + +**Serialization errors:** +```rust +// Ensure all types implement required traits +#[derive(serde::Serialize, serde::Deserialize)] +pub struct MyType { + pub field: u32, +} +``` + +**Runtime errors:** +```rust +// Add error handling +#[program] +fn safe_program(input: u32) -> Result { + if input == 0 { + Err("Zero input not allowed".to_string()) + } else { + Ok(input * 2) + } +} +``` + +### Debug Mode + +Enable debug output: + +```bash +# Build with debug info +RUST_LOG=debug cargo build + +# Run test with debug output +RUST_LOG=pvq_program=debug cargo test +``` + +## Related Components + +- [PVQ Executor](../pvq-executor/) - Program execution environment +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Program Metadata Gen](../pvq-program-metadata-gen/) - Metadata generation +- [PVQ Test Runner](../pvq-test-runner/) - Testing utilities + +--- + +*PVQ Program provides the tools to create efficient, type-safe guest programs for the PVQ system.* \ No newline at end of file diff --git a/pvq-runtime-api/README.md b/pvq-runtime-api/README.md new file mode 100644 index 0000000..fa3b21c --- /dev/null +++ b/pvq-runtime-api/README.md @@ -0,0 +1,648 @@ +# PVQ Runtime API + + + +The Substrate runtime API definition for PVQ (PolkaVM Query) that provides the interface between external clients and the PVQ execution system. This crate defines the `PvqApi` trait that runtime implementations must provide to support PVQ functionality. + +## Overview + +The PVQ Runtime API serves as the bridge between external clients (like UIs, CLIs, or other applications) and the PVQ execution system within Substrate runtimes. It provides a standardized interface for submitting queries, retrieving results, and accessing system metadata. + +## Features + +- 🌐 **Standard Interface**: Consistent API across different runtime implementations +- πŸ” **Query Execution**: Submit and execute PVQ programs with arguments +- πŸ“Š **Metadata Access**: Retrieve information about available extensions and capabilities +- ⚑ **Async Support**: Compatible with Substrate's async runtime API system +- πŸ›‘οΈ **Type Safety**: Strongly-typed interface with compile-time guarantees +- πŸ“ˆ **Performance Metrics**: Built-in execution statistics and monitoring + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Runtime API β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ PvqApi β”‚ β”‚ Client Side β”‚ β”‚ Runtime β”‚ β”‚ +β”‚ β”‚ Trait β”‚ β”‚ β”‚ β”‚ Side β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ β”œβ”€ Query β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ execute β”‚ β”‚ β”‚ Submission β”‚ β”‚ β”œβ”€ Executor β”‚ β”‚ +β”‚ β”‚ β”‚ _query β”‚ β”‚ β”œβ”€ Result β”‚ β”‚ β”‚ Instance β”‚ β”‚ +β”‚ β”‚ β”œβ”€ metadata β”‚ β”‚ β”‚ Processing β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ +β”‚ β”‚ β”œβ”€ capabilities β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ β”‚ Registry β”‚ β”‚ +β”‚ β”‚ └─ version β”‚ β”‚ β”‚ Handling β”‚ β”‚ └─ Context β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ └─ Async β”‚ β”‚ Management β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Support β”‚ β”‚ β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## API Definition + +### Core Trait + +The main runtime API trait that must be implemented: + +```rust +use sp_api::decl_runtime_apis; +use pvq_primitives::{PvqResult, QueryMetadata}; + +decl_runtime_apis! { + /// PVQ Runtime API for executing queries and managing the system + pub trait PvqApi { + /// Execute a PVQ program with given arguments + /// + /// # Parameters + /// - `program`: Compiled PVQ program blob (PolkaVM bytecode) + /// - `args`: Serialized arguments for the program + /// + /// # Returns + /// Serialized result of program execution + fn execute_query(program: Vec, args: Vec) -> Vec; + + /// Get metadata about the PVQ system and available extensions + /// + /// # Returns + /// Serialized system metadata including extension information + fn metadata() -> Vec; + + /// Get system capabilities and configuration + /// + /// # Returns + /// Serialized capability information + fn capabilities() -> Vec; + + /// Get the PVQ system version information + /// + /// # Returns + /// Version string + fn version() -> String; + } +} +``` + +## Runtime Implementation + +### Basic Implementation + +Implementing the API in your runtime: + +```rust +use sp_api::impl_runtime_apis; +use pvq_executor::{PvqExecutor, PvqExecutorContext}; +use pvq_extension::ExtensionsExecutor; +use pvq_extension_core::ExtensionCore; +use pvq_extension_fungibles::ExtensionFungibles; +use pvq_extension_swap::ExtensionSwap; + +impl_runtime_apis! { + impl pvq_runtime_api::PvqApi for Runtime { + fn execute_query(program: Vec, args: Vec) -> Vec { + // Create extension executor with available extensions + let mut extensions = ExtensionsExecutor::new(); + extensions.register::(); + extensions.register::(); + extensions.register::(); + + // Create execution context + let context = PvqExecutorContext::new(extensions); + + // Create and configure executor + let executor = PvqExecutor::new(context); + + // Execute program + match executor.execute(&program, &args) { + Ok(result) => result, + Err(error) => { + // Return error as serialized response + format!("Error: {}", error).into_bytes() + } + } + } + + fn metadata() -> Vec { + let metadata = SystemMetadata { + version: "1.0.0".to_string(), + extensions: vec![ + "core".to_string(), + "fungibles".to_string(), + "swap".to_string(), + ], + max_execution_time_ms: 30000, + max_memory_bytes: 64 * 1024 * 1024, + }; + + serde_json::to_vec(&metadata).unwrap_or_default() + } + + fn capabilities() -> Vec { + let capabilities = SystemCapabilities { + supported_program_versions: vec!["1.0".to_string()], + max_program_size_bytes: 1024 * 1024, + max_args_size_bytes: 1024 * 1024, + max_result_size_bytes: 10 * 1024 * 1024, + execution_timeout_ms: 30000, + }; + + serde_json::to_vec(&capabilities).unwrap_or_default() + } + + fn version() -> String { + "PVQ Runtime API v1.0.0".to_string() + } + } +} +``` + +### Advanced Implementation + +More sophisticated implementation with error handling and monitoring: + +```rust +use pvq_primitives::{PvqResponse, QueryMetadata, PvqError}; +use std::time::Instant; + +impl_runtime_apis! { + impl pvq_runtime_api::PvqApi for Runtime { + fn execute_query(program: Vec, args: Vec) -> Vec { + let start_time = Instant::now(); + let mut query_metadata = QueryMetadata::new(); + + // Validate input sizes + if program.len() > MAX_PROGRAM_SIZE { + let error = PvqError::ResourceLimitExceeded { + resource: "program_size".to_string(), + limit: MAX_PROGRAM_SIZE as u64, + requested: program.len() as u64, + }; + return Self::create_error_response(error, query_metadata); + } + + // Set up extensions with runtime context + let mut extensions = ExtensionsExecutor::new(); + Self::configure_extensions(&mut extensions); + + // Create execution context with monitoring + let context = PvqExecutorContext::builder() + .with_extensions(extensions) + .with_block_number(Self::current_block_number()) + .with_monitoring(true) + .build(); + + // Execute with timeout and resource limits + let executor = PvqExecutor::builder() + .with_context(context) + .with_timeout(Duration::from_secs(30)) + .with_memory_limit(64 * 1024 * 1024) + .build(); + + match executor.execute(&program, &args) { + Ok(result) => { + query_metadata.execution_time_ms = start_time.elapsed().as_millis() as u64; + query_metadata.memory_used_bytes = executor.memory_usage(); + query_metadata.extension_calls = executor.extension_call_count(); + query_metadata.block_number = Self::current_block_number(); + + let response = PvqResponse { + data: result, + metadata: query_metadata, + success: true, + }; + + serde_json::to_vec(&response).unwrap_or_default() + } + Err(error) => { + query_metadata.execution_time_ms = start_time.elapsed().as_millis() as u64; + query_metadata.error_message = Some(error.to_string()); + Self::create_error_response(error, query_metadata) + } + } + } + + fn metadata() -> Vec { + let extensions_metadata = Self::get_extensions_metadata(); + + let metadata = SystemMetadata { + version: env!("CARGO_PKG_VERSION").to_string(), + runtime_version: Self::runtime_version().to_string(), + extensions: extensions_metadata, + limits: SystemLimits { + max_execution_time_ms: MAX_EXECUTION_TIME_MS, + max_memory_bytes: MAX_MEMORY_BYTES, + max_program_size: MAX_PROGRAM_SIZE, + max_args_size: MAX_ARGS_SIZE, + }, + features: SystemFeatures { + async_execution: true, + permission_control: true, + performance_monitoring: true, + }, + }; + + serde_json::to_vec(&metadata).unwrap_or_default() + } + + fn capabilities() -> Vec { + let capabilities = SystemCapabilities::builder() + .with_program_versions(vec!["1.0".to_string()]) + .with_serialization_formats(vec!["json".to_string(), "scale".to_string()]) + .with_extension_discovery(true) + .with_batch_execution(false) // Future feature + .build(); + + serde_json::to_vec(&capabilities).unwrap_or_default() + } + + fn version() -> String { + format!("PVQ Runtime API v{}", env!("CARGO_PKG_VERSION")) + } + } +} + +impl Runtime { + fn configure_extensions(executor: &mut ExtensionsExecutor) { + // Register core extensions + executor.register::(); + + // Register fungibles extension if assets pallet is available + if Self::has_assets_pallet() { + executor.register::(); + } + + // Register swap extension if DEX pallet is available + if Self::has_dex_pallet() { + executor.register::(); + } + + // Register custom runtime-specific extensions + Self::register_custom_extensions(executor); + } + + fn create_error_response(error: PvqError, metadata: QueryMetadata) -> Vec { + let response = PvqResponse { + data: Vec::new(), + metadata, + success: false, + }; + + serde_json::to_vec(&response).unwrap_or_else(|_| { + format!(r#"{{"error": "{}"}}"#, error).into_bytes() + }) + } +} +``` + +## Client Usage + +### Direct API Calls + +Using the API from client code: + +```rust +use sp_api::ProvideRuntimeApi; +use pvq_runtime_api::PvqApi; + +// In your client code +async fn execute_pvq_program( + client: &Client, + program: Vec, + args: Vec, +) -> Result, Box> +where + Client: ProvideRuntimeApi, + Client::Api: PvqApi, +{ + let at = client.info().best_hash; + + let result = client + .runtime_api() + .execute_query(at, program, args)?; + + Ok(result) +} + +async fn get_system_metadata( + client: &Client, +) -> Result> +where + Client: ProvideRuntimeApi, + Client::Api: PvqApi, +{ + let at = client.info().best_hash; + + let metadata_bytes = client + .runtime_api() + .metadata(at)?; + + let metadata: SystemMetadata = serde_json::from_slice(&metadata_bytes)?; + Ok(metadata) +} +``` + +### RPC Integration + +Wrapping the API for RPC access: + +```rust +use jsonrpsee::{core::RpcResult, proc_macros::rpc}; +use sp_api::ProvideRuntimeApi; +use pvq_runtime_api::PvqApi; + +#[rpc(client, server)] +pub trait PvqRpcApi { + /// Execute a PVQ program + #[method(name = "pvq_executeQuery")] + async fn execute_query( + &self, + program: String, // Hex-encoded program + args: String, // Hex-encoded arguments + ) -> RpcResult; // Hex-encoded result + + /// Get system metadata + #[method(name = "pvq_metadata")] + async fn metadata(&self) -> RpcResult; + + /// Get system capabilities + #[method(name = "pvq_capabilities")] + async fn capabilities(&self) -> RpcResult; + + /// Get version information + #[method(name = "pvq_version")] + async fn version(&self) -> RpcResult; +} + +pub struct PvqRpc { + client: Arc, +} + +impl PvqRpc { + pub fn new(client: Arc) -> Self { + Self { client } + } +} + +impl PvqRpcApiServer for PvqRpc +where + Client: ProvideRuntimeApi + Send + Sync + 'static, + Client::Api: PvqApi, +{ + async fn execute_query( + &self, + program_hex: String, + args_hex: String, + ) -> RpcResult { + let program = hex::decode(program_hex) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Invalid program hex: {}", e)))?; + + let args = hex::decode(args_hex) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Invalid args hex: {}", e)))?; + + let at = self.client.info().best_hash; + + let result = self.client + .runtime_api() + .execute_query(at, program, args) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Execution failed: {}", e)))?; + + Ok(hex::encode(result)) + } + + async fn metadata(&self) -> RpcResult { + let at = self.client.info().best_hash; + + let metadata_bytes = self.client + .runtime_api() + .metadata(at) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get metadata: {}", e)))?; + + let metadata: serde_json::Value = serde_json::from_slice(&metadata_bytes) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to parse metadata: {}", e)))?; + + Ok(metadata) + } + + async fn capabilities(&self) -> RpcResult { + let at = self.client.info().best_hash; + + let capabilities_bytes = self.client + .runtime_api() + .capabilities(at) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get capabilities: {}", e)))?; + + let capabilities: serde_json::Value = serde_json::from_slice(&capabilities_bytes) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to parse capabilities: {}", e)))?; + + Ok(capabilities) + } + + async fn version(&self) -> RpcResult { + let at = self.client.info().best_hash; + + let version = self.client + .runtime_api() + .version(at) + .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get version: {}", e)))?; + + Ok(version) + } +} +``` + +## Data Types + +### System Metadata + +Information about the PVQ system: + +```rust +use serde::{Serialize, Deserialize}; + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct SystemMetadata { + pub version: String, + pub runtime_version: String, + pub extensions: Vec, + pub limits: SystemLimits, + pub features: SystemFeatures, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct ExtensionInfo { + pub name: String, + pub version: String, + pub description: String, + pub functions: Vec, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct SystemLimits { + pub max_execution_time_ms: u64, + pub max_memory_bytes: u64, + pub max_program_size: usize, + pub max_args_size: usize, +} + +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct SystemFeatures { + pub async_execution: bool, + pub permission_control: bool, + pub performance_monitoring: bool, +} +``` + +### System Capabilities + +Runtime capability information: + +```rust +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct SystemCapabilities { + pub supported_program_versions: Vec, + pub serialization_formats: Vec, + pub max_program_size_bytes: usize, + pub max_args_size_bytes: usize, + pub max_result_size_bytes: usize, + pub execution_timeout_ms: u64, + pub extension_discovery: bool, + pub batch_execution: bool, +} +``` + +## Configuration + +### Runtime Configuration + +Configure API behavior in your runtime: + +```rust +parameter_types! { + pub const MaxProgramSize: u32 = 1024 * 1024; // 1MB + pub const MaxArgsSize: u32 = 1024 * 1024; // 1MB + pub const MaxResultSize: u32 = 10 * 1024 * 1024; // 10MB + pub const ExecutionTimeout: u64 = 30000; // 30 seconds +} + +impl pvq_runtime_api::Config for Runtime { + type MaxProgramSize = MaxProgramSize; + type MaxArgsSize = MaxArgsSize; + type MaxResultSize = MaxResultSize; + type ExecutionTimeout = ExecutionTimeout; +} +``` + +### Feature Gates + +Control API features: + +```rust +#[cfg(feature = "pvq-api")] +impl_runtime_apis! { + impl pvq_runtime_api::PvqApi for Runtime { + // Implementation here + } +} +``` + +## Development + +### Building + +```bash +# Build runtime API +cargo build -p pvq-runtime-api + +# Build with all features +cargo build -p pvq-runtime-api --all-features + +# Generate documentation +cargo doc -p pvq-runtime-api --open +``` + +### Testing + +```bash +# Unit tests +cargo test -p pvq-runtime-api + +# Integration tests with runtime +cargo test -p pvq-runtime-api --test runtime_integration + +# Test RPC functionality +cargo test -p pvq-runtime-api --test rpc_tests +``` + +## Performance Considerations + +### Optimization Tips + +1. **Cache Extension Registry**: Avoid recreating extensions on each call +2. **Limit Result Size**: Implement size limits to prevent resource exhaustion +3. **Async Execution**: Use async where possible for better concurrency +4. **Memory Management**: Monitor and limit memory usage during execution + +### Example Optimizations + +```rust +impl Runtime { + // Cache extension registry + thread_local! { + static EXTENSION_CACHE: RefCell> = RefCell::new(None); + } + + fn get_cached_extensions() -> ExtensionsExecutor { + Self::EXTENSION_CACHE.with(|cache| { + let mut cache = cache.borrow_mut(); + if cache.is_none() { + let mut extensions = ExtensionsExecutor::new(); + Self::configure_extensions(&mut extensions); + *cache = Some(extensions); + } + cache.as_ref().unwrap().clone() + }) + } +} +``` + +## Security Considerations + +- Validate all input data before processing +- Implement proper resource limits and timeouts +- Use sandboxed execution environment +- Log and monitor API usage +- Rate limit API calls if exposed publicly + +## Error Handling + +Handle API errors gracefully: + +```rust +impl Runtime { + fn handle_api_error(error: PvqError) -> Vec { + let response = match error { + PvqError::Timeout { duration_ms } => { + ApiResponse::timeout(duration_ms) + } + PvqError::ResourceLimitExceeded { resource, .. } => { + ApiResponse::resource_exhausted(resource) + } + _ => { + ApiResponse::general_error(error.to_string()) + } + }; + + serde_json::to_vec(&response).unwrap_or_default() + } +} +``` + +## Related Components + +- [PVQ Executor](../pvq-executor/) - Execution environment +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Primitives](../pvq-primitives/) - Core types +- [PoC Runtime](../poc/runtime/) - Example implementation + +--- + +*The PVQ Runtime API provides the standard interface for integrating PVQ functionality into Substrate runtimes.* \ No newline at end of file diff --git a/pvq-test-runner/README.md b/pvq-test-runner/README.md new file mode 100644 index 0000000..27eb989 --- /dev/null +++ b/pvq-test-runner/README.md @@ -0,0 +1,675 @@ +# PVQ Test Runner + + + +A comprehensive testing and development tool for PVQ (PolkaVM Query) programs. The test runner provides a simulated environment for executing guest programs, validating results, and debugging query logic without requiring a full runtime deployment. + +## Overview + +The PVQ Test Runner serves as an essential development tool that enables developers to test their PVQ programs locally, validate extension interactions, and debug program logic in a controlled environment. It simulates the complete PVQ execution environment including extensions, permissions, and resource limits. + +## Features + +- πŸ§ͺ **Local Testing**: Execute PVQ programs without a full runtime environment +- 🎯 **Extension Simulation**: Mock extension calls and responses for testing +- πŸ” **Detailed Logging**: Comprehensive execution tracing and debugging information +- ⚑ **Performance Profiling**: Measure execution time, memory usage, and resource consumption +- πŸ›‘οΈ **Error Simulation**: Test error handling and edge cases +- πŸ“Š **Result Validation**: Compare actual vs. expected results with detailed diffs +- πŸ”§ **Configuration Options**: Flexible testing scenarios and environment setup + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ PVQ Test Runner β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Test Runner β”‚ β”‚ Simulation β”‚ β”‚ Reporting β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Environment β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Program β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ β”‚ Loading β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ β”œβ”€ Results β”‚ β”‚ +β”‚ β”‚ β”œβ”€ Execution β”‚ β”‚ β”‚ Mocking β”‚ β”‚ β”‚ Analysis β”‚ β”‚ +β”‚ β”‚ β”‚ Management β”‚ β”‚ β”œβ”€ State β”‚ β”‚ β”œβ”€ Performanceβ”‚ β”‚ +β”‚ β”‚ β”œβ”€ Result β”‚ β”‚ β”‚ Management β”‚ β”‚ β”‚ Metrics β”‚ β”‚ +β”‚ β”‚ β”‚ Validation β”‚ β”‚ β”œβ”€ Permission β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ +β”‚ β”‚ └─ Configurationβ”‚ β”‚ β”‚ Control β”‚ β”‚ β”‚ Reports β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ └─ Resource β”‚ β”‚ └─ Debug β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ Limits β”‚ β”‚ Output β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Installation + +### From Source + +```bash +# Clone the PVQ repository +git clone --recursive https://github.com/open-web3-stack/PVQ.git +cd PVQ + +# Build the test runner +cargo build -p pvq-test-runner --release + +# The binary will be available at target/release/pvq-test-runner +``` + +### Global Installation + +```bash +# Install globally via cargo +cargo install --path pvq-test-runner + +# Or install from crates.io (when published) +cargo install pvq-test-runner +``` + +## Usage + +### Basic Usage + +```bash +# Run a compiled PVQ program +pvq-test-runner --program output/my-program.polkavm + +# Run with custom arguments +pvq-test-runner --program output/my-program.polkavm --args "arg1,arg2,arg3" + +# Run with hex-encoded arguments +pvq-test-runner --program output/my-program.polkavm --args-hex "0x010203040506" +``` + +### Command Line Options + +```bash +pvq-test-runner [OPTIONS] --program + +Options: + -p, --program Path to the compiled PVQ program + -a, --args Comma-separated program arguments + --args-hex Hex-encoded program arguments + -c, --config Path to configuration file + -o, --output Output format (json, yaml, pretty) + -v, --verbose Enable verbose logging + --debug Enable debug mode with detailed tracing + --profile Enable performance profiling + --timeout Execution timeout in seconds [default: 30] + --memory-limit Memory limit in MB [default: 64] + --extensions Comma-separated list of enabled extensions + --mock-file Path to extension mock configuration + --validate Path to expected result file for validation + --no-color Disable colored output + -h, --help Print help information + -V, --version Print version information +``` + +## Configuration + +### Configuration File + +Create a `test-config.toml` for complex testing scenarios: + +```toml +[program] +path = "output/my-program.polkavm" +timeout_seconds = 30 +memory_limit_mb = 64 + +[execution] +enable_debugging = true +enable_profiling = true +trace_extension_calls = true +max_call_depth = 256 + +[extensions] +enabled = ["core", "fungibles", "swap"] + +[extensions.fungibles] +mock_balance_responses = true +default_balance = 1000 + +[extensions.swap] +mock_pool_data = true +default_pool_reserves = [10000, 20000] + +[logging] +level = "debug" +format = "pretty" +include_timestamps = true + +[validation] +expected_result_file = "tests/expected_results.json" +tolerance = 0.001 # For floating point comparisons +``` + +### Extension Mocking + +Create mock configurations for testing: + +```json +{ + "extension_mocks": { + "fungibles": { + "balance": { + "default_response": 1000, + "specific_responses": { + "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]": 5000, + "[2, [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]]": 2500 + } + }, + "total_supply": { + "default_response": 1000000, + "specific_responses": { + "[1]": 500000, + "[2]": 750000 + } + } + }, + "swap": { + "pool_reserves": { + "default_response": [10000, 20000], + "specific_responses": { + "[{\"asset_0\": 1, \"asset_1\": 2}]": [15000, 25000] + } + } + } + } +} +``` + +## Testing Scenarios + +### Basic Program Testing + +```bash +# Test a simple balance query program +pvq-test-runner --program output/balance-query.polkavm \ + --args "1,0x0101010101010101010101010101010101010101010101010101010101010101" \ + --extensions "core,fungibles" \ + --verbose +``` + +### Complex Integration Testing + +```bash +# Test with comprehensive configuration +pvq-test-runner \ + --config test-scenarios/portfolio-analysis.toml \ + --mock-file test-data/market-data-mocks.json \ + --validate test-data/expected-portfolio-result.json \ + --profile \ + --output json +``` + +### Performance Testing + +```bash +# Performance benchmark +pvq-test-runner --program output/performance-test.polkavm \ + --profile \ + --timeout 60 \ + --memory-limit 128 \ + --debug +``` + +## Development Integration + +### Automated Testing + +Integrate with your development workflow: + +```bash +#!/bin/bash +# test-all-programs.sh + +set -e + +echo "Building all guest programs..." +make guests + +echo "Running tests..." + +for program in output/*.polkavm; do + echo "Testing $(basename $program)..." + pvq-test-runner --program "$program" \ + --config "tests/$(basename $program .polkavm).toml" \ + --validate "tests/expected/$(basename $program .polkavm).json" +done + +echo "All tests passed!" +``` + +### CI/CD Integration + +GitHub Actions example: + +```yaml +name: PVQ Program Tests + +on: [push, pull_request] + +jobs: + test-programs: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + with: + submodules: recursive + + - name: Install Rust + uses: actions-rs/toolchain@v1 + with: + toolchain: stable + + - name: Install tools + run: make tools + + - name: Build guest programs + run: make guests + + - name: Run PVQ tests + run: | + for program in output/*.polkavm; do + cargo run -p pvq-test-runner -- \ + --program "$program" \ + --config "tests/$(basename $program .polkavm).toml" + done +``` + +## API Reference + +### Programmatic Usage + +Use the test runner as a library: + +```rust +use pvq_test_runner::{TestRunner, TestConfig, ExtensionMock}; +use std::collections::HashMap; + +#[tokio::main] +async fn main() -> Result<(), Box> { + // Create test configuration + let config = TestConfig::builder() + .with_timeout(Duration::from_secs(30)) + .with_memory_limit(64 * 1024 * 1024) + .with_extensions(vec!["core", "fungibles"]) + .enable_profiling(true) + .enable_debugging(true) + .build(); + + // Set up extension mocks + let mut mocks = HashMap::new(); + mocks.insert( + "fungibles.balance".to_string(), + ExtensionMock::simple_response(serde_json::json!(1000)) + ); + + // Create and configure test runner + let mut runner = TestRunner::new(config); + runner.set_extension_mocks(mocks); + + // Load and execute program + let program = std::fs::read("output/my-program.polkavm")?; + let args = serde_json::to_vec(&(1u32, [0u8; 32]))?; + + let result = runner.execute_program(&program, &args).await?; + + // Analyze results + println!("Execution successful: {}", result.success); + println!("Result: {:?}", result.output); + println!("Execution time: {}ms", result.execution_time_ms); + println!("Memory used: {} bytes", result.memory_used_bytes); + + if let Some(profile) = result.profile { + println!("Performance profile:"); + for (operation, duration) in profile.operation_times { + println!(" {}: {}ms", operation, duration.as_millis()); + } + } + + Ok(()) +} +``` + +### Test Utilities + +Helper functions for common testing patterns: + +```rust +use pvq_test_runner::testing::*; + +#[tokio::test] +async fn test_balance_query() { + let mut tester = create_default_tester().await; + + // Mock balance response + tester.mock_extension_call( + "fungibles", + "balance", + vec![json!(1), json!([0u8; 32])], + json!(5000) + ); + + // Execute test + let result = tester.execute_program_file( + "output/balance-query.polkavm", + &(1u32, [0u8; 32]) + ).await.unwrap(); + + // Validate result + assert!(result.success); + let balance: u128 = serde_json::from_slice(&result.output).unwrap(); + assert_eq!(balance, 5000); +} + +#[tokio::test] +async fn test_error_handling() { + let mut tester = create_default_tester().await; + + // Mock extension error + tester.mock_extension_error( + "fungibles", + "balance", + "Asset not found" + ); + + let result = tester.execute_program_file( + "output/balance-query.polkavm", + &(999u32, [0u8; 32]) + ).await.unwrap(); + + // Should handle error gracefully + assert!(result.success); // Program should handle error + let response: String = serde_json::from_slice(&result.output).unwrap(); + assert!(response.contains("Asset not found")); +} +``` + +## Output Formats + +### Pretty Output (Default) + +``` +PVQ Test Runner v0.1.0 +====================== + +Program: output/balance-query.polkavm +Arguments: [1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]] + +Execution Results: +β”œβ”€ Status: βœ“ Success +β”œβ”€ Output: 1000 +β”œβ”€ Execution Time: 15ms +β”œβ”€ Memory Used: 2.1MB +β”œβ”€ Extension Calls: 3 +β”‚ β”œβ”€ core.current_block_number: 1 call +β”‚ └─ fungibles.balance: 1 call +└─ Block Number: 12345 + +Performance Profile: +β”œβ”€ Program Loading: 2ms +β”œβ”€ Extension Setup: 1ms +β”œβ”€ Program Execution: 10ms +└─ Result Serialization: 2ms +``` + +### JSON Output + +```json +{ + "success": true, + "program": "output/balance-query.polkavm", + "arguments": "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]", + "output": "1000", + "execution_time_ms": 15, + "memory_used_bytes": 2097152, + "extension_calls": { + "total": 2, + "by_extension": { + "core": 1, + "fungibles": 1 + } + }, + "block_number": 12345, + "profile": { + "program_loading_ms": 2, + "extension_setup_ms": 1, + "program_execution_ms": 10, + "result_serialization_ms": 2 + }, + "error": null +} +``` + +### YAML Output + +```yaml +success: true +program: output/balance-query.polkavm +arguments: "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]" +output: "1000" +execution_time_ms: 15 +memory_used_bytes: 2097152 +extension_calls: + total: 2 + by_extension: + core: 1 + fungibles: 1 +block_number: 12345 +profile: + program_loading_ms: 2 + extension_setup_ms: 1 + program_execution_ms: 10 + result_serialization_ms: 2 +error: null +``` + +## Advanced Features + +### Custom Validators + +Create custom result validation: + +```rust +use pvq_test_runner::{TestRunner, ValidationRule, ValidationResult}; + +struct BalanceRangeValidator { + min: u128, + max: u128, +} + +impl ValidationRule for BalanceRangeValidator { + fn validate(&self, output: &[u8]) -> ValidationResult { + match serde_json::from_slice::(output) { + Ok(balance) => { + if balance >= self.min && balance <= self.max { + ValidationResult::success() + } else { + ValidationResult::failure(format!( + "Balance {} outside expected range [{}, {}]", + balance, self.min, self.max + )) + } + } + Err(e) => ValidationResult::error(format!("Failed to parse balance: {}", e)), + } + } +} + +// Usage +let mut runner = TestRunner::new(config); +runner.add_validator(Box::new(BalanceRangeValidator { + min: 100, + max: 10000 +})); +``` + +### Extension Development Testing + +Test custom extensions: + +```rust +use pvq_test_runner::extension_testing::ExtensionTester; + +#[tokio::test] +async fn test_custom_extension() { + let mut tester = ExtensionTester::new(); + + // Register custom extension + tester.register_extension::(); + + // Test extension directly + let result = tester.call_extension( + "my_custom_extension", + "my_function", + vec![json!("test_input")] + ).await.unwrap(); + + assert_eq!(result, json!("expected_output")); + + // Test in program context + let program_result = tester.execute_program_with_extension( + "output/custom-extension-test.polkavm", + &("test_input",) + ).await.unwrap(); + + assert!(program_result.success); +} +``` + +## Debugging + +### Debug Mode + +Enable comprehensive debugging: + +```bash +# Enable all debug output +pvq-test-runner --program output/my-program.polkavm --debug + +# Specific debug categories +RUST_LOG=pvq_test_runner=debug,pvq_executor=trace pvq-test-runner \ + --program output/my-program.polkavm +``` + +### Trace Output + +Example trace output: + +``` +[DEBUG pvq_test_runner] Loading program: output/balance-query.polkavm +[TRACE pvq_executor] Creating PolkaVM instance +[DEBUG pvq_extension] Registering extension: core +[DEBUG pvq_extension] Registering extension: fungibles +[TRACE pvq_executor] Program loaded, size: 4096 bytes +[DEBUG pvq_test_runner] Executing with args: [1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]] +[TRACE pvq_executor] Starting program execution +[DEBUG pvq_extension] Extension call: core.current_block_number() -> 12345 +[DEBUG pvq_extension] Extension call: fungibles.balance(1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]) -> 1000 +[TRACE pvq_executor] Program execution completed +[DEBUG pvq_test_runner] Result: 1000 +``` + +## Performance Analysis + +### Profiling Reports + +Detailed performance analysis: + +``` +Performance Analysis Report +=========================== + +Program: output/complex-analysis.polkavm +Total Execution Time: 245ms + +Breakdown: +β”œβ”€ Program Loading: 5ms (2.0%) +β”œβ”€ Extension Setup: 3ms (1.2%) +β”œβ”€ Program Execution: 230ms (93.9%) +β”‚ β”œβ”€ Extension Calls: 45ms (18.4%) +β”‚ β”‚ β”œβ”€ fungibles.list_assets: 5ms +β”‚ β”‚ β”œβ”€ fungibles.balance (Γ—10): 25ms +β”‚ β”‚ └─ swap.pool_info (Γ—5): 15ms +β”‚ └─ Program Logic: 185ms (75.5%) +└─ Result Serialization: 7ms (2.9%) + +Memory Usage: +β”œβ”€ Peak Usage: 12.5MB +β”œβ”€ Average Usage: 8.2MB +└─ Final Usage: 1.1MB + +Extension Call Statistics: +β”œβ”€ Total Calls: 16 +β”œβ”€ Average Call Time: 2.8ms +β”œβ”€ Slowest Call: swap.pool_info (4.2ms) +└─ Most Frequent: fungibles.balance (10 calls) +``` + +### Optimization Suggestions + +The test runner provides optimization hints: + +``` +Optimization Suggestions: +======================== + +πŸ” Performance Issues Detected: +β”œβ”€ High extension call overhead (18.4% of execution time) +β”‚ └─ Consider batching balance queries into single operation +β”œβ”€ Memory fragmentation detected +β”‚ └─ Consider using Vec::with_capacity for known-size collections +└─ Serialization overhead significant (2.9%) + └─ Consider using more efficient serialization format + +βœ… Good Practices Observed: +β”œβ”€ Efficient extension usage patterns +β”œβ”€ Proper error handling +└─ Minimal memory allocations in hot paths +``` + +## Troubleshooting + +### Common Issues + +**Program won't load:** +```bash +# Check program format +file output/my-program.polkavm + +# Verify program was built correctly +ls -la output/my-program.polkavm + +# Try with minimal configuration +pvq-test-runner --program output/my-program.polkavm --extensions core +``` + +**Extension errors:** +```bash +# Enable extension debug logging +RUST_LOG=pvq_extension=debug pvq-test-runner --program output/my-program.polkavm + +# Check available extensions +pvq-test-runner --program output/my-program.polkavm --extensions core --debug +``` + +**Memory issues:** +```bash +# Increase memory limit +pvq-test-runner --program output/my-program.polkavm --memory-limit 128 + +# Enable memory profiling +pvq-test-runner --program output/my-program.polkavm --profile --debug +``` + +## Related Components + +- [PVQ Program](../pvq-program/) - Program development tools +- [PVQ Executor](../pvq-executor/) - Execution environment +- [PVQ Extension System](../pvq-extension/) - Extension framework +- [PVQ Program Metadata Gen](../pvq-program-metadata-gen/) - Metadata generation + +--- + +*The PVQ Test Runner provides comprehensive testing and development support for PVQ programs.* \ No newline at end of file From 7005c244608d873ac4ebb5620050acc030e1cbed Mon Sep 17 00:00:00 2001 From: Bryan Chen Date: Thu, 7 Aug 2025 16:33:35 +1200 Subject: [PATCH 2/4] update docs --- .../sum-balance-percent/src/main.rs | 21 ++++ guest-examples/sum-balance/src/main.rs | 31 ++++++ guest-examples/swap-info/src/main.rs | 99 +++++++++++++++++++ guest-examples/total-supply/src/main.rs | 20 ++++ pvq-executor/src/context.rs | 5 + pvq-executor/src/error.rs | 11 ++- pvq-executor/src/executor.rs | 20 ++++ pvq-executor/src/lib.rs | 1 + pvq-extension-core/src/lib.rs | 13 +++ pvq-extension-fungibles/src/lib.rs | 70 +++++++++++++ pvq-extension-swap/src/lib.rs | 59 +++++++++++ pvq-extension/procedural/src/lib.rs | 3 + pvq-extension/src/calldata.rs | 20 ++-- pvq-extension/src/context.rs | 13 +-- pvq-extension/src/error.rs | 15 +-- pvq-extension/src/executor.rs | 19 ++-- pvq-extension/src/macros.rs | 11 ++- pvq-extension/src/metadata.rs | 28 +++--- pvq-extension/src/perm_controller.rs | 25 ++--- pvq-primitives/src/lib.rs | 12 +++ pvq-program-metadata-gen/src/lib.rs | 1 + pvq-program/procedural/src/lib.rs | 7 +- pvq-program/src/lib.rs | 1 + pvq-runtime-api/src/lib.rs | 18 +++- pvq-test-runner/src/lib.rs | 11 +++ 25 files changed, 467 insertions(+), 67 deletions(-) diff --git a/guest-examples/sum-balance-percent/src/main.rs b/guest-examples/sum-balance-percent/src/main.rs index 1468c9c..a04dab1 100644 --- a/guest-examples/sum-balance-percent/src/main.rs +++ b/guest-examples/sum-balance-percent/src/main.rs @@ -1,16 +1,37 @@ +//! A guest program that sums the balances of a set of accounts for a given asset and returns the percentage of the total supply. #![no_std] #![no_main] +/// The primary module for the sum-balance-percent guest program. +/// +/// This module defines the necessary types, extension functions, and the main entrypoint +/// for calculating the percentage of total supply represented by the sum of balances +/// for a given set of accounts. #[pvq_program::program] mod sum_balance_percent { + /// A type alias for the asset identifier. type AssetId = u32; + /// A type alias for the account identifier. type AccountId = [u8; 32]; + /// A type alias for the balance of an account. type Balance = u64; + + /// Retrieves the balance of a specific account for a given asset. + /// + /// This is an extension function that calls into the runtime. #[program::extension_fn(extension_id = 1248491991627109725u64, fn_index = 6)] fn balance(asset: AssetId, who: AccountId) -> Balance {} + + /// Retrieves the total supply of a given asset. + /// + /// This is an extension function that calls into the runtime. #[program::extension_fn(extension_id = 1248491991627109725u64, fn_index = 5)] fn total_supply(asset: AssetId) -> Balance {} + /// The entrypoint of the program. + /// + /// It takes an asset ID and a vector of account IDs, calculates the sum of their balances, + /// and returns the percentage of this sum with respect to the total supply of the asset. #[program::entrypoint] fn sum_balance(asset: AssetId, accounts: alloc::vec::Vec) -> Balance { let mut sum_balance = 0; diff --git a/guest-examples/sum-balance/src/main.rs b/guest-examples/sum-balance/src/main.rs index 2780a58..737c1a0 100644 --- a/guest-examples/sum-balance/src/main.rs +++ b/guest-examples/sum-balance/src/main.rs @@ -1,28 +1,59 @@ +//! A guest program that sums the balance of a given asset for a list of accounts. #![no_std] #![no_main] +/// A guest program that sums the balance of a given asset for a list of accounts. #[pvq_program::program] mod sum_balance { cfg_if::cfg_if! { if #[cfg(feature = "option_version_1")] { + /// Represents a unique identifier for an account. type AccountId = [u8; 64]; + /// Represents a unique identifier for an asset. type AssetId = u64; + /// Represents the balance of an asset. type Balance = u128; } else if #[cfg(feature = "option_version_2")] { + /// Represents a unique identifier for an account. type AccountId = [u8; 32]; + /// Represents a unique identifier for an asset. type AssetId = u32; + /// Represents the balance of an asset. type Balance = u64; } else { + /// Represents a unique identifier for an account. type AccountId = [u8; 32]; + /// Represents a unique identifier for an asset. type AssetId = u32; + /// Represents the balance of an asset. type Balance = u64; } } + /// Get the balance of a given asset for a specific account. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// * `who`: The account identifier. + /// + /// # Returns + /// + /// The balance of the asset for the specified account. #[program::extension_fn(extension_id = 1248491991627109725u64, fn_index = 6)] fn balance(asset: AssetId, who: AccountId) -> Balance {} + /// Sums the balance of a given asset for a list of accounts. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// * `accounts`: A list of account identifiers. + /// + /// # Returns + /// + /// The total balance of the asset for all the specified accounts. #[program::entrypoint] fn sum_balance(asset: AssetId, accounts: alloc::vec::Vec) -> Balance { let mut sum = 0; diff --git a/guest-examples/swap-info/src/main.rs b/guest-examples/swap-info/src/main.rs index 10538d9..4689878 100644 --- a/guest-examples/swap-info/src/main.rs +++ b/guest-examples/swap-info/src/main.rs @@ -1,21 +1,42 @@ +//! A guest program that provides information about asset swaps. #![no_std] #![no_main] +/// A guest program that provides information about asset swaps. #[pvq_program::program] mod swap_info { + /// Represents a unique identifier for an asset. type AssetId = alloc::vec::Vec; + /// Represents the balance of an asset. type Balance = u128; + /// Represents information about an asset. #[derive( Debug, Clone, PartialEq, Eq, parity_scale_codec::Encode, parity_scale_codec::Decode, scale_info::TypeInfo, )] pub struct AssetInfo { + /// The unique identifier of the asset. pub asset_id: AssetId, + /// The name of the asset. pub name: alloc::vec::Vec, + /// The symbol of the asset. pub symbol: alloc::vec::Vec, + /// The number of decimals the asset has. pub decimals: u8, } + /// Get the quote price of `asset1` in terms of `asset2` for a given amount of `asset2`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to be quoted. + /// * `asset2`: The identifier of the asset to quote against. + /// * `amount`: The amount of `asset2`. + /// * `include_fee`: Whether to include the fee in the quote. + /// + /// # Returns + /// + /// The quote price of `asset1` in terms of `asset2`, or `None` if the quote is not available. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 0)] fn quote_price_tokens_for_exact_tokens( asset1: AssetId, @@ -25,6 +46,18 @@ mod swap_info { ) -> Option { } + /// Get the quote price of `asset2` in terms of `asset1` for a given amount of `asset1`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to quote against. + /// * `asset2`: The identifier of the asset to be quoted. + /// * `amount`: The amount of `asset1`. + /// * `include_fee`: Whether to include the fee in the quote. + /// + /// # Returns + /// + /// The quote price of `asset2` in terms of `asset1`, or `None` if the quote is not available. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 1)] fn quote_price_exact_tokens_for_tokens( asset1: AssetId, @@ -34,18 +67,58 @@ mod swap_info { ) -> Option { } + /// Get the liquidity pool of two assets. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the first asset. + /// * `asset2`: The identifier of the second asset. + /// + /// # Returns + /// + /// A tuple containing the balance of each asset in the liquidity pool, or `None` if the pool does not exist. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 2)] fn get_liquidity_pool(asset1: AssetId, asset2: AssetId) -> Option<(Balance, Balance)> {} + /// List all available liquidity pools. + /// + /// # Returns + /// + /// A list of tuples, where each tuple represents a liquidity pool and contains the identifiers of the two assets in the pool. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 3)] fn list_pools() -> alloc::vec::Vec<(AssetId, AssetId)> {} + /// Get information about a specific asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// Information about the asset, or `None` if the asset does not exist. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 4)] fn asset_info(asset: AssetId) -> Option {} + /// Get information about all assets. + /// + /// # Returns + /// + /// A map of asset identifiers to asset information. #[program::extension_fn(extension_id = 15900548380266538526u64, fn_index = 5)] fn assets_info() -> alloc::collections::BTreeMap {} + /// Get the quote price of `asset2` in terms of `asset1` for a given amount of `asset1`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to quote against. + /// * `asset2`: The identifier of the asset to be quoted. + /// * `amount`: The amount of `asset1`. + /// + /// # Returns + /// + /// The quote price of `asset2` in terms of `asset1`, or `None` if the quote is not available. #[program::entrypoint] fn entrypoint_quote_price_exact_tokens_for_tokens( asset1: AssetId, @@ -55,6 +128,17 @@ mod swap_info { quote_price_exact_tokens_for_tokens(asset1, asset2, amount, true) } + /// Get the quote price of `asset1` in terms of `asset2` for a given amount of `asset2`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to be quoted. + /// * `asset2`: The identifier of the asset to quote against. + /// * `amount`: The amount of `asset2`. + /// + /// # Returns + /// + /// The quote price of `asset1` in terms of `asset2`, or `None` if the quote is not available. #[program::entrypoint] fn entrypoint_quote_price_tokens_for_exact_tokens( asset1: AssetId, @@ -64,11 +148,26 @@ mod swap_info { quote_price_tokens_for_exact_tokens(asset1, asset2, amount, true) } + /// Get the liquidity pool of two assets. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the first asset. + /// * `asset2`: The identifier of the second asset. + /// + /// # Returns + /// + /// A tuple containing the balance of each asset in the liquidity pool, or `None` if the pool does not exist. #[program::entrypoint] fn entrypoint_get_liquidity_pool(asset1: AssetId, asset2: AssetId) -> Option<(Balance, Balance)> { get_liquidity_pool(asset1, asset2) } + /// List all available liquidity pools with their asset information. + /// + /// # Returns + /// + /// A list of tuples, where each tuple represents a liquidity pool and contains the information of the two assets in the pool. #[program::entrypoint] fn entrypoint_list_pools() -> alloc::vec::Vec<(AssetInfo, AssetInfo)> { let pools = list_pools(); diff --git a/guest-examples/total-supply/src/main.rs b/guest-examples/total-supply/src/main.rs index cdda439..56d095f 100644 --- a/guest-examples/total-supply/src/main.rs +++ b/guest-examples/total-supply/src/main.rs @@ -1,11 +1,31 @@ +//! A guest program that queries the total supply of a given asset. #![no_std] #![no_main] +/// A guest program that queries the total supply of a given asset. #[pvq_program::program] mod query_total_supply { + /// Get the total supply of a given asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The total supply of the asset. #[program::extension_fn(extension_id = 4071833530116166512u64, fn_index = 0)] fn total_supply(asset: u32) -> u64 {} + /// Get the total supply of a given asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The total supply of the asset. #[program::entrypoint] fn get_total_supply(asset: u32) -> u64 { total_supply(asset) diff --git a/pvq-executor/src/context.rs b/pvq-executor/src/context.rs index 8c37c94..aba7713 100644 --- a/pvq-executor/src/context.rs +++ b/pvq-executor/src/context.rs @@ -1,8 +1,13 @@ use polkavm::Linker; +/// The context for the PVQ executor. pub trait PvqExecutorContext { + /// The user data. type UserData; + /// The user error. type UserError; + /// Registers the host functions. fn register_host_functions(&mut self, linker: &mut Linker); + /// Returns a mutable reference to the user data. fn data(&mut self) -> &mut Self::UserData; } diff --git a/pvq-executor/src/error.rs b/pvq-executor/src/error.rs index 8aba798..d1cb633 100644 --- a/pvq-executor/src/error.rs +++ b/pvq-executor/src/error.rs @@ -1,20 +1,23 @@ use pvq_primitives::PvqError; +/// The error type for the PVQ executor. #[derive(Debug, thiserror::Error)] pub enum PvqExecutorError { + /// The program format is invalid. #[error("Invalid PVQ program format")] InvalidProgramFormat, + /// A memory access error occurred. #[error("Memory access error: {0}")] MemoryAccessError(polkavm::MemoryAccessError), - // Extract from the PVM CallError + /// A trap occurred during execution. #[error("Trap")] Trap, - // Extract from the PVM CallError + /// Not enough gas to execute the program. #[error("Not enough gas")] NotEnoughGas, - // Usually a custom error type from the extension system definition + /// A user-defined error occurred. #[error("User error: {0}")] User(UserError), - // Other errors directly from the PVM + /// An other error from the PolkaVM occurred. #[error("Other PVM error: {0}")] OtherPvmError(polkavm::Error), } diff --git a/pvq-executor/src/executor.rs b/pvq-executor/src/executor.rs index d2f1862..6c6a605 100644 --- a/pvq-executor/src/executor.rs +++ b/pvq-executor/src/executor.rs @@ -4,9 +4,12 @@ use polkavm::{Config, Engine, Linker, Module, ModuleConfig, ProgramBlob}; use crate::context::PvqExecutorContext; use crate::error::PvqExecutorError; +/// The result of a PVQ execution. type PvqExecutorResult = Result, PvqExecutorError>; +/// The gas limit for a PVQ execution. type GasLimit = Option; +/// The PVQ executor. pub struct PvqExecutor { engine: Engine, linker: Linker, @@ -14,6 +17,12 @@ pub struct PvqExecutor { } impl PvqExecutor { + /// Creates a new PVQ executor. + /// + /// # Arguments + /// + /// * `config`: The PolkaVM configuration. + /// * `context`: The PVQ executor context. pub fn new(config: Config, mut context: Ctx) -> Self { let engine = Engine::new(&config).unwrap(); let mut linker = Linker::::new(); @@ -26,6 +35,17 @@ impl PvqExecutor { } } + /// Executes a PVQ program. + /// + /// # Arguments + /// + /// * `program`: The PVQ program to execute. + /// * `args`: The arguments to pass to the program. + /// * `gas_limit`: The gas limit for the execution. + /// + /// # Returns + /// + /// A tuple containing the result of the execution and the remaining gas. pub fn execute( &mut self, program: &[u8], diff --git a/pvq-executor/src/lib.rs b/pvq-executor/src/lib.rs index 8092ad9..f938e62 100644 --- a/pvq-executor/src/lib.rs +++ b/pvq-executor/src/lib.rs @@ -1,3 +1,4 @@ +//! The PVQ executor. #![cfg_attr(not(feature = "std"), no_std)] extern crate alloc; diff --git a/pvq-extension-core/src/lib.rs b/pvq-extension-core/src/lib.rs index 4a5c039..75b58be 100644 --- a/pvq-extension-core/src/lib.rs +++ b/pvq-extension-core/src/lib.rs @@ -1,12 +1,25 @@ +//! The core PVQ extension. #![cfg_attr(not(feature = "std"), no_std)] use pvq_extension::extension_decl; +/// The core PVQ extension. #[extension_decl] pub mod extension { + /// The core PVQ extension trait. #[extension_decl::extension] pub trait ExtensionCore { + /// The extension identifier type. type ExtensionId; + /// Check if an extension is enabled. + /// + /// # Arguments + /// + /// * `id`: The identifier of the extension. + /// + /// # Returns + /// + /// `true` if the extension is enabled, `false` otherwise. fn has_extension(id: Self::ExtensionId) -> bool; // crypto functions // fn blake2_64(data: Vec) -> [u8; 8]; diff --git a/pvq-extension-fungibles/src/lib.rs b/pvq-extension-fungibles/src/lib.rs index 7e104ca..2686d51 100644 --- a/pvq-extension-fungibles/src/lib.rs +++ b/pvq-extension-fungibles/src/lib.rs @@ -1,20 +1,90 @@ +//! The fungibles PVQ extension. #![cfg_attr(not(feature = "std"), no_std)] use pvq_extension::extension_decl; +/// The fungibles PVQ extension. #[extension_decl] pub mod extension { use scale_info::prelude::vec::Vec; + /// The fungibles PVQ extension trait. #[extension_decl::extension] pub trait ExtensionFungibles { + /// The asset identifier type. type AssetId; + /// The balance type. type Balance; + /// The account identifier type. type AccountId; + /// Check if an asset exists. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// `true` if the asset exists, `false` otherwise. fn asset_exists(asset: Self::AssetId) -> bool; + /// Get the name of an asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The name of the asset. fn name(asset: Self::AssetId) -> Vec; + /// Get the symbol of an asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The symbol of the asset. fn symbol(asset: Self::AssetId) -> Vec; + /// Get the decimals of an asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The decimals of the asset. fn decimals(asset: Self::AssetId) -> u8; + /// Get the minimum balance of an asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The minimum balance of the asset. fn minimum_balance(asset: Self::AssetId) -> Self::Balance; + /// Get the total supply of an asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// The total supply of the asset. fn total_supply(asset: Self::AssetId) -> Self::Balance; + /// Get the balance of an asset for a specific account. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// * `who`: The account identifier. + /// + /// # Returns + /// + /// The balance of the asset for the specified account. fn balance(asset: Self::AssetId, who: Self::AccountId) -> Self::Balance; } } diff --git a/pvq-extension-swap/src/lib.rs b/pvq-extension-swap/src/lib.rs index 1b46977..28cf030 100644 --- a/pvq-extension-swap/src/lib.rs +++ b/pvq-extension-swap/src/lib.rs @@ -1,19 +1,37 @@ +//! The swap PVQ extension. #![cfg_attr(not(feature = "std"), no_std)] extern crate alloc; use pvq_extension::extension_decl; +/// The swap PVQ extension. #[extension_decl] pub mod extension { use alloc::collections::BTreeMap; use alloc::vec::Vec; + /// The swap PVQ extension trait. #[extension_decl::extension] pub trait ExtensionSwap { + /// The asset identifier type. type AssetId; + /// The balance type. type Balance; + /// The asset info type. type AssetInfo; + /// Get the quote price of `asset1` in terms of `asset2` for a given amount of `asset2`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to be quoted. + /// * `asset2`: The identifier of the asset to quote against. + /// * `amount`: The amount of `asset2`. + /// * `include_fee`: Whether to include the fee in the quote. + /// + /// # Returns + /// + /// The quote price of `asset1` in terms of `asset2`, or `None` if the quote is not available. fn quote_price_tokens_for_exact_tokens( asset1: Self::AssetId, asset2: Self::AssetId, @@ -21,6 +39,18 @@ pub mod extension { include_fee: bool, ) -> Option; + /// Get the quote price of `asset2` in terms of `asset1` for a given amount of `asset1`. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the asset to quote against. + /// * `asset2`: The identifier of the asset to be quoted. + /// * `amount`: The amount of `asset1`. + /// * `include_fee`: Whether to include the fee in the quote. + /// + /// # Returns + /// + /// The quote price of `asset2` in terms of `asset1`, or `None` if the quote is not available. fn quote_price_exact_tokens_for_tokens( asset1: Self::AssetId, asset2: Self::AssetId, @@ -28,12 +58,41 @@ pub mod extension { include_fee: bool, ) -> Option; + /// Get the liquidity pool of two assets. + /// + /// # Arguments + /// + /// * `asset1`: The identifier of the first asset. + /// * `asset2`: The identifier of the second asset. + /// + /// # Returns + /// + /// A tuple containing the balance of each asset in the liquidity pool, or `None` if the pool does not exist. fn get_liquidity_pool(asset1: Self::AssetId, asset2: Self::AssetId) -> Option<(Self::Balance, Self::Balance)>; + /// List all available liquidity pools. + /// + /// # Returns + /// + /// A list of tuples, where each tuple represents a liquidity pool and contains the identifiers of the two assets in the pool. fn list_pools() -> Vec<(Self::AssetId, Self::AssetId)>; + /// Get information about a specific asset. + /// + /// # Arguments + /// + /// * `asset`: The identifier of the asset. + /// + /// # Returns + /// + /// Information about the asset, or `None` if the asset does not exist. fn asset_info(asset: Self::AssetId) -> Option; + /// Get information about all assets. + /// + /// # Returns + /// + /// A map of asset identifiers to asset information. fn assets_info() -> BTreeMap; } } diff --git a/pvq-extension/procedural/src/lib.rs b/pvq-extension/procedural/src/lib.rs index 9bc82dd..a5a13b9 100644 --- a/pvq-extension/procedural/src/lib.rs +++ b/pvq-extension/procedural/src/lib.rs @@ -1,13 +1,16 @@ +//! This crate provides procedural macros for the PVQ extension system. use proc_macro::TokenStream; mod extension_decl; mod extensions_impl; pub(crate) mod utils; +/// A procedural macro for declaring an extension. #[proc_macro_attribute] pub fn extension_decl(attr: TokenStream, item: TokenStream) -> TokenStream { extension_decl::extension_decl(attr, item) } +/// A procedural macro for implementing extensions. #[proc_macro_attribute] pub fn extensions_impl(attr: TokenStream, item: TokenStream) -> TokenStream { extensions_impl::extensions_impl(attr, item) diff --git a/pvq-extension/src/calldata.rs b/pvq-extension/src/calldata.rs index 3810f25..6e73c3c 100644 --- a/pvq-extension/src/calldata.rs +++ b/pvq-extension/src/calldata.rs @@ -1,31 +1,35 @@ +//! This module defines the traits and types for handling extension call data. use parity_scale_codec::Decode; use scale_info::prelude::vec::Vec; -/// Type for extension IDs +/// The type for extension identifiers. pub type ExtensionIdTy = u64; -/// Trait for identifying extensions +/// A trait for identifying extensions. pub trait ExtensionId { + /// The unique identifier of the extension. const EXTENSION_ID: ExtensionIdTy; } -/// Trait for dispatching extension calls +/// A trait for dispatching extension calls. pub trait Dispatchable { + /// Dispatches the extension call. fn dispatch(self) -> Result, DispatchError>; } -/// Error type for dispatch operations +/// The error type for dispatch operations. #[derive(Debug, thiserror::Error)] pub enum DispatchError { + /// A phantom data error. #[error("PhantomData")] PhantomData, } -/// Trait for extension call data +/// A trait for extension call data. /// /// This trait combines several traits that are required for extension call data: -/// - `Dispatchable`: Allows dispatching calls to the extension functions -/// - `ExtensionId`: Identifies the extension -/// - `Decode`: Allows decoding the call data +/// - `Dispatchable`: Allows dispatching calls to the extension functions. +/// - `ExtensionId`: Identifies the extension. +/// - `Decode`: Allows decoding the call data. pub trait CallData: Dispatchable + ExtensionId + Decode {} impl CallData for T where T: Dispatchable + ExtensionId + Decode {} diff --git a/pvq-extension/src/context.rs b/pvq-extension/src/context.rs index 976c0c6..4e0d22d 100644 --- a/pvq-extension/src/context.rs +++ b/pvq-extension/src/context.rs @@ -1,3 +1,4 @@ +//! This module defines the execution context for extensions. use pvq_executor::{Caller, Linker, PvqExecutorContext}; use scale_info::prelude::marker::PhantomData; @@ -7,25 +8,25 @@ use crate::{ CallDataTuple, }; -/// Execution context for extensions +/// The execution context for extensions. /// /// This struct provides the context for executing extensions. /// It includes the invoke source and user data. pub struct Context { - /// The source of the invocation + /// The source of the invocation. invoke_source: InvokeSource, - /// User data for the context + /// User data for the context. user_data: (), - /// Marker for the generic parameters + /// A marker for the generic parameters. _marker: PhantomData<(C, P)>, } impl Context { - /// Create a new context + /// Creates a new context. /// /// # Arguments /// - /// * `invoke_source` - The source of the invocation + /// * `invoke_source`: The source of the invocation. pub fn new(invoke_source: InvokeSource) -> Self { Self { invoke_source, diff --git a/pvq-extension/src/error.rs b/pvq-extension/src/error.rs index d3ac67f..89401f1 100644 --- a/pvq-extension/src/error.rs +++ b/pvq-extension/src/error.rs @@ -1,32 +1,33 @@ +//! This module defines the error types for the extension system. // TODO: contain source error use crate::DispatchError; use parity_scale_codec::Error as CodecError; -/// Errors that can occur when working with extensions +/// The error type for the extension system. // Typically will be used as a UserError #[derive(Debug, thiserror::Error)] pub enum ExtensionError { - /// Permission denied for the requested operation + /// Permission to perform the requested operation was denied. #[error("Permission denied")] PermissionError, - /// Failed to allocate memory + /// Failed to allocate memory. #[error("Failed to allocate memory")] MemoryAllocationError, - /// Error accessing memory + /// An error occurred while accessing memory. #[error("Memory access error: {0}")] MemoryAccessError(polkavm::MemoryAccessError), - /// Error decoding data + /// An error occurred while decoding data. #[error("Decode error: {0}")] DecodeError(CodecError), - /// Error dispatching a call + /// An error occurred while dispatching a call. #[error("Dispatch error: {0:?}")] DispatchError(#[from] DispatchError), - /// The requested extension is not supported + /// The requested extension is not supported. #[error("Unsupported extension")] UnsupportedExtension, } diff --git a/pvq-extension/src/executor.rs b/pvq-extension/src/executor.rs index f83fd03..3d68829 100644 --- a/pvq-extension/src/executor.rs +++ b/pvq-extension/src/executor.rs @@ -1,3 +1,4 @@ +//! This module defines the executor for extensions. use pvq_executor::PvqExecutor; use pvq_primitives::{PvqError, PvqResult}; @@ -6,36 +7,38 @@ use crate::{ CallDataTuple, Context, }; -/// Executor for extensions +/// The executor for extensions. /// /// This struct provides an executor for running extension code. -/// It wraps a PvqExecutor with a Context for extensions. +/// It wraps a `PvqExecutor` with a `Context` for extensions. pub struct ExtensionsExecutor { - /// The underlying PVQ executor + /// The underlying PVQ executor. executor: PvqExecutor>, } impl ExtensionsExecutor { - /// Create a new extensions executor + /// Creates a new extensions executor. /// /// # Arguments /// - /// * `source` - The source of the invocation + /// * `source`: The source of the invocation. pub fn new(source: InvokeSource) -> Self { let context = Context::::new(source); let executor = PvqExecutor::new(Default::default(), context); Self { executor } } + /// Executes a program with the given arguments and gas limit. /// /// # Arguments /// - /// * `program` - The program data - /// * `args` - The input data + /// * `program`: The program to execute. + /// * `args`: The arguments to pass to the program. + /// * `gas_limit`: The gas limit for the execution. /// /// # Returns /// - /// The result of the execution or an error + /// A tuple containing the result of the execution and the remaining gas. pub fn execute(&mut self, program: &[u8], args: &[u8], gas_limit: Option) -> (PvqResult, Option) { let (result, gas_remaining) = self.executor.execute(program, args, gas_limit); tracing::info!("result: {:?}", result); diff --git a/pvq-extension/src/macros.rs b/pvq-extension/src/macros.rs index 447640a..b4dff2d 100644 --- a/pvq-extension/src/macros.rs +++ b/pvq-extension/src/macros.rs @@ -1,19 +1,20 @@ +//! This module defines the `CallDataTuple` trait, which is used to dispatch calls to extensions. use crate::{CallData, ExtensionError, ExtensionIdTy}; use fortuples::fortuples; use scale_info::prelude::vec::Vec; -/// Trait for a tuple of extension call data types +/// A trait for a tuple of extension call data types. pub trait CallDataTuple { - /// Dispatch a call to an extension + /// Dispatches a call to an extension. /// /// # Arguments /// - /// * `extension_id` - The ID of the extension to call - /// * `data` - The encoded call data + /// * `extension_id`: The identifier of the extension to call. + /// * `data`: The encoded call data. /// /// # Returns /// - /// The encoded response data or an error + /// The encoded response data or an error. fn dispatch(extension_id: ExtensionIdTy, data: &[u8]) -> Result, ExtensionError>; } diff --git a/pvq-extension/src/metadata.rs b/pvq-extension/src/metadata.rs index 3237302..036dd05 100644 --- a/pvq-extension/src/metadata.rs +++ b/pvq-extension/src/metadata.rs @@ -1,8 +1,10 @@ +//! This module defines the metadata structures for extensions. use crate::ExtensionIdTy; use scale_info::prelude::string::{String, ToString}; -// This trait is for ExtensionImpl +/// A trait for retrieving extension implementation metadata. pub trait ExtensionImplMetadata { + /// Returns the metadata for a given extension. fn extension_metadata(extension_id: ExtensionIdTy) -> ExtensionMetadata; } @@ -14,15 +16,17 @@ use scale_info::{ IntoPortable, PortableRegistry, Registry, }; use serde::Serialize; -/// Metadata of extensions +/// The metadata of all extensions. #[derive(Clone, PartialEq, Eq, Encode, Debug, Serialize)] pub struct Metadata { + /// The portable type registry. pub types: PortableRegistry, - // Use String to prevent loss of precision in frontend codes + /// A map of extension identifiers to their metadata. pub extensions: BTreeMap>, } impl Metadata { + /// Creates a new `Metadata` instance. pub fn new(extensions: BTreeMap) -> Self { let mut registry = Registry::new(); let extensions = extensions @@ -36,10 +40,12 @@ impl Metadata { } } -/// Metadata of an extension. +/// The metadata of an extension. #[derive(Clone, PartialEq, Eq, Encode, Debug)] pub struct ExtensionMetadata { + /// The name of the extension. pub name: T::String, + /// The functions of the extension. pub functions: Vec>, } @@ -67,14 +73,14 @@ impl Serialize for ExtensionMetadata { } } -/// Metadata of a runtime function. +/// The metadata of a runtime function. #[derive(Clone, PartialEq, Eq, Encode, Debug)] pub struct FunctionMetadata { - /// Method name. + /// The name of the function. pub name: T::String, - /// Method parameters. + /// The parameters of the function. pub inputs: Vec>, - /// Method output. + /// The output of the function. pub output: T::Type, } @@ -104,12 +110,12 @@ impl Serialize for FunctionMetadata { } } -/// Metadata of a runtime method parameter. +/// The metadata of a runtime function parameter. #[derive(Clone, PartialEq, Eq, Encode, Debug)] pub struct FunctionParamMetadata { - /// Parameter name. + /// The name of the parameter. pub name: T::String, - /// Parameter type. + /// The type of the parameter. pub ty: T::Type, } diff --git a/pvq-extension/src/perm_controller.rs b/pvq-extension/src/perm_controller.rs index 8a98ef4..5a4f3ab 100644 --- a/pvq-extension/src/perm_controller.rs +++ b/pvq-extension/src/perm_controller.rs @@ -1,41 +1,42 @@ +//! This module defines the permission controller for extensions. use crate::ExtensionIdTy; -/// Source of an extension invocation +/// The source of an extension invocation. #[derive(Copy, Clone, Debug, PartialEq, Eq)] pub enum InvokeSource { - /// Invoked from a runtime API + /// The invocation is from a runtime API. RuntimeAPI, - /// Invoked from XCM (Cross-Consensus Message) + /// The invocation is from an XCM (Cross-Consensus Message). XCM, - /// Invoked from an extrinsic + /// The invocation is from an extrinsic. Extrinsic, - /// Invoked from the runtime inside + /// The invocation is from the runtime itself. Runtime, } -/// Controller for extension permissions +/// A controller for extension permissions. /// /// This trait is used to control access to extensions based on the extension ID, /// call data, and invocation source. pub trait PermissionController { - /// Check if a call to an extension is allowed + /// Checks if a call to an extension is allowed. /// /// # Arguments /// - /// * `extension_id` - The ID of the extension - /// * `call` - The encoded call data - /// * `source` - The source of the invocation + /// * `extension_id`: The identifier of the extension. + /// * `call`: The encoded call data. + /// * `source`: The source of the invocation. /// /// # Returns /// - /// `true` if the call is allowed, `false` otherwise + /// `true` if the call is allowed, `false` otherwise. fn is_allowed(extension_id: ExtensionIdTy, call: &[u8], source: InvokeSource) -> bool; } -/// Default permission controller that allows everything +/// A default permission controller that allows all calls. impl PermissionController for () { fn is_allowed(_extension_id: ExtensionIdTy, _call: &[u8], _source: InvokeSource) -> bool { true diff --git a/pvq-primitives/src/lib.rs b/pvq-primitives/src/lib.rs index 7f4f93d..e1aaf10 100644 --- a/pvq-primitives/src/lib.rs +++ b/pvq-primitives/src/lib.rs @@ -1,3 +1,4 @@ +//! This crate defines the primitive types for the PVQ system. #![cfg_attr(not(feature = "std"), no_std)] extern crate alloc; @@ -8,21 +9,32 @@ use parity_scale_codec::{Decode, Encode}; #[cfg(not(feature = "std"))] use scale_info::TypeInfo; +/// The result of a PVQ query. pub type PvqResult = Result; +/// The response of a PVQ query. pub type PvqResponse = Vec; +/// The error of a PVQ query. #[cfg(feature = "std")] pub type PvqError = String; +/// The error of a PVQ query. #[cfg(not(feature = "std"))] #[derive(Debug, Clone, PartialEq, Eq, Encode, Decode, TypeInfo)] pub enum PvqError { + /// Failed to decode the query. FailedToDecode, + /// The PVQ program format is invalid. InvalidPvqProgramFormat, + /// The query exceeds the weight limit. QueryExceedsWeightLimit, + /// A trap occurred during execution. Trap, + /// A memory access error occurred. MemoryAccessError, + /// A host call error occurred. HostCallError, + /// An other error occurred. Other, } diff --git a/pvq-program-metadata-gen/src/lib.rs b/pvq-program-metadata-gen/src/lib.rs index 82961fe..6be6c4d 100644 --- a/pvq-program-metadata-gen/src/lib.rs +++ b/pvq-program-metadata-gen/src/lib.rs @@ -1,3 +1,4 @@ +//! This crate provides a tool for generating metadata for PVQ programs. pub type ExtensionId = u64; pub type FnIndex = u8; mod features; diff --git a/pvq-program/procedural/src/lib.rs b/pvq-program/procedural/src/lib.rs index 37babc0..c617b2e 100644 --- a/pvq-program/procedural/src/lib.rs +++ b/pvq-program/procedural/src/lib.rs @@ -1,4 +1,7 @@ -/// Declare the calls used in PVQ program +//! This crate provides the `program` procedural macro for creating PVQ programs. +/// +/// # Usage +/// /// ```ignore /// #[program] /// mod query_fungibles { @@ -20,10 +23,10 @@ /// } /// } /// ``` -/// mod program; use proc_macro::TokenStream; +/// A procedural macro for creating PVQ programs. #[proc_macro_attribute] pub fn program(attr: TokenStream, item: TokenStream) -> TokenStream { program::program(attr, item) diff --git a/pvq-program/src/lib.rs b/pvq-program/src/lib.rs index dd2d335..cae8057 100644 --- a/pvq-program/src/lib.rs +++ b/pvq-program/src/lib.rs @@ -1,2 +1,3 @@ +//! This crate provides the `program` macro for creating PVQ programs. #![cfg_attr(not(feature = "std"), no_std)] pub use pvq_program_procedural::program; diff --git a/pvq-runtime-api/src/lib.rs b/pvq-runtime-api/src/lib.rs index af4110d..7a69861 100644 --- a/pvq-runtime-api/src/lib.rs +++ b/pvq-runtime-api/src/lib.rs @@ -1,3 +1,4 @@ +//! This crate defines the runtime API for the PVQ module. #![cfg_attr(not(feature = "std"), no_std)] extern crate alloc; @@ -6,13 +7,22 @@ use alloc::vec::Vec; use pvq_primitives::PvqResult; use sp_api::decl_runtime_apis; -// The runtime API for the PVQ module. -// - `program`: PVQ binary. -// - `args`: Query arguments that is SCALE-encoded. -// - `gas_limit`: Optional gas limit for query execution. When set to `None`, execution is constrained by the default time boundary. decl_runtime_apis! { + /// The runtime API for the PVQ module. pub trait PvqApi { + /// Executes a PVQ query. + /// + /// # Arguments + /// + /// * `program`: The PVQ program binary. + /// * `args`: The SCALE-encoded query arguments. + /// * `gas_limit`: An optional gas limit for the query execution. If `None`, the execution is constrained by the default time boundary. + /// + /// # Returns + /// + /// The result of the PVQ query. fn execute_query(program: Vec, args: Vec, gas_limit: Option) -> PvqResult; + /// Returns the metadata of the PVQ extensions. fn metadata() -> Vec; } } diff --git a/pvq-test-runner/src/lib.rs b/pvq-test-runner/src/lib.rs index 106de3a..c7743d3 100644 --- a/pvq-test-runner/src/lib.rs +++ b/pvq-test-runner/src/lib.rs @@ -1,3 +1,4 @@ +//! This crate provides a test runner for PVQ programs. use parity_scale_codec::Encode; use pvq_extension::{extensions_impl, ExtensionsExecutor, InvokeSource}; use sp_core::crypto::{AccountId32, Ss58Codec}; @@ -5,22 +6,27 @@ use sp_core::hexdisplay::HexDisplay; use xcm::v5::Junction::{GeneralIndex, PalletInstance}; use xcm::v5::Location; +/// The functions of the fungibles extension. #[derive(Encode)] #[allow(non_camel_case_types)] #[allow(dead_code)] pub enum ExtensionFungiblesFunctions { + /// The `total_supply` function. #[codec(index = 0)] total_supply { asset: u32 }, + /// The `balance` function. #[codec(index = 1)] balance { asset: u32, who: [u8; 32] }, } +/// The extensions implementation for the test runner. #[extensions_impl] pub mod extensions { use std::collections::BTreeMap; use parity_scale_codec::Decode; + /// The extensions implementation struct. #[extensions_impl::impl_struct] pub struct ExtensionsImpl; @@ -102,17 +108,20 @@ pub mod extensions { } } +/// The test runner for PVQ programs. pub struct TestRunner { executor: ExtensionsExecutor, } impl TestRunner { + /// Creates a new test runner. pub fn new() -> Self { Self { executor: ExtensionsExecutor::new(InvokeSource::RuntimeAPI), } } + /// Prepares the input data for a test. pub fn prepare_input_data(program_path: &str, chain: &str) -> Vec { let mut input_data = Vec::new(); @@ -146,6 +155,7 @@ impl TestRunner { input_data } + /// Returns the expected result of a test. pub fn expected_result(program_path: &str, chain: &str, entrypoint_idx: u8) -> Vec { // TODO: add more entrypoints if program_path.contains("sum-balance") && chain == "poc" && entrypoint_idx == 0 { @@ -159,6 +169,7 @@ impl TestRunner { Vec::new() } + /// Executes a PVQ program. pub fn execute_program(&mut self, program_blob: &[u8], input_data: &[u8]) -> pvq_primitives::PvqResult { let (result, _) = self.executor.execute(program_blob, input_data, None); result From d47f27a96e7c2371929145789253ccf7c0d4d157 Mon Sep 17 00:00:00 2001 From: Bryan Chen Date: Thu, 7 Aug 2025 16:38:52 +1200 Subject: [PATCH 3/4] update readme --- README.md | 45 +------------------------------------------ poc/runtime/README.md | 15 +++------------ 2 files changed, 4 insertions(+), 56 deletions(-) diff --git a/README.md b/README.md index 345005e..e13455d 100644 --- a/README.md +++ b/README.md @@ -146,7 +146,7 @@ pub trait MyCustomExtension { ### Component Documentation - πŸ“š **[Development Guide](docs/development.md)** - Comprehensive development setup -- πŸ”§ **[Extension Development](docs/extensions.md)** - Creating custom extensions +- πŸ”§ **[Extension Development](docs/extensions.md)** - Creating custom extensions - πŸ—οΈ **[Architecture Deep Dive](docs/architecture.md)** - System design and internals - πŸ“‹ **[API Reference](docs/api.md)** - Complete API documentation @@ -200,46 +200,3 @@ cargo test -p pvq-executor # Run example programs make test-guests ``` - -## 🀝 Contributing - -We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. - -### Development Workflow - -1. Fork the repository -2. Create a feature branch (`git checkout -b feature/amazing-feature`) -3. Make your changes and add tests -4. Ensure all tests pass (`cargo test --all`) -5. Run formatting (`cargo fmt`) and linting (`cargo clippy`) -6. Commit your changes (`git commit -am 'Add amazing feature'`) -7. Push to the branch (`git push origin feature/amazing-feature`) -8. Open a Pull Request - -### Code Style - -- Follow Rust standard formatting (`cargo fmt`) -- Ensure clippy passes (`cargo clippy`) -- Add comprehensive tests for new features -- Document public APIs with rustdoc comments - -## πŸ“„ License - -This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. - -## πŸ™ Acknowledgments - -- [PolkaVM](https://github.com/koute/polkavm) - The underlying virtual machine -- [Substrate](https://substrate.io/) - The blockchain development framework -- [Polkadot](https://polkadot.network/) - The multi-chain protocol - -## πŸ”— Links - -- 🌐 **Homepage**: [https://acala.network](https://acala.network) -- πŸ“– **Documentation**: [https://docs.pvq.dev](https://docs.pvq.dev) -- πŸ’¬ **Discord**: [Join our community](https://discord.gg/acala) -- 🐦 **Twitter**: [@AcalaNetwork](https://twitter.com/AcalaNetwork) - ---- - -**Built with ❀️ by the Acala team** \ No newline at end of file diff --git a/poc/runtime/README.md b/poc/runtime/README.md index 1558ccb..42ed9a7 100644 --- a/poc/runtime/README.md +++ b/poc/runtime/README.md @@ -16,7 +16,7 @@ This PoC runtime showcases how to: - πŸ—οΈ **Complete Runtime**: Full Substrate runtime with PVQ integration - πŸ”Œ **Extension Support**: Pre-configured with core PVQ extensions -- πŸ§ͺ **Testing Ready**: Optimized for development and testing workflows +- πŸ§ͺ **Testing Ready**: Optimized for development and testing workflows - πŸ“‘ **Chopsticks Compatible**: Works with Chopsticks for local development - βš™οΈ **Configurable**: Easy to modify and extend for custom use cases @@ -163,7 +163,7 @@ pub const EXTENSIONS: &[&str] = &["core", "fungibles", "swap"]; 2. **Configure in PVQ setup:** ```rust use my_pvq_extension::ExtensionMyCustom; - + // Add to extensions list ``` @@ -240,15 +240,6 @@ Enable debug logging: RUST_LOG=debug cargo run -p pvq-test-runner -- --program output/guest-example ``` -## Contributing - -When modifying the PoC runtime: - -1. Maintain compatibility with existing PVQ programs -2. Update tests for any configuration changes -3. Document any new pallets or extensions -4. Test with both Chopsticks and the test runner - ## Related Components - [PVQ Executor](../pvq-executor/) - Core execution engine @@ -258,4 +249,4 @@ When modifying the PoC runtime: --- -*This PoC runtime demonstrates PVQ integration patterns and serves as a foundation for production implementations.* \ No newline at end of file +*This PoC runtime demonstrates PVQ integration patterns and serves as a foundation for production implementations.* From acdc089996bce152a85a000a3602b5464235503c Mon Sep 17 00:00:00 2001 From: indirection42 Date: Mon, 11 Aug 2025 09:21:08 +0800 Subject: [PATCH 4/4] rewrite docs with hallucinations --- poc/runtime/README.md | 250 +---------- pvq-extension-core/README.md | 384 ---------------- pvq-extension-fungibles/README.md | 513 ---------------------- pvq-extension/README.md | 557 +----------------------- pvq-primitives/README.md | 700 +----------------------------- pvq-program/README.md | 696 +++-------------------------- pvq-runtime-api/README.md | 656 ++-------------------------- pvq-test-runner/README.md | 673 +--------------------------- 8 files changed, 157 insertions(+), 4272 deletions(-) delete mode 100644 pvq-extension-core/README.md delete mode 100644 pvq-extension-fungibles/README.md diff --git a/poc/runtime/README.md b/poc/runtime/README.md index 42ed9a7..8168391 100644 --- a/poc/runtime/README.md +++ b/poc/runtime/README.md @@ -1,252 +1,20 @@ # PVQ PoC Runtime - +FRAME runtime demonstrating PVQ integration used in this repository. -A proof-of-concept Substrate runtime demonstrating PVQ (PolkaVM Query) system integration. This runtime serves as a testing environment and reference implementation for integrating PVQ capabilities into Substrate-based blockchains. +### What it exposes -## Overview +- Runtime API `PvqApi` with two methods: + - `execute_query(...)`: runs a PVQ program with input bytes and an optional gas limit; if not provided, a default ~2s limit is applied. + - `metadata()`: returns serialized extension metadata bytes (portable type registry + extension functions). -This PoC runtime showcases how to: -- Integrate the `PvqApi` runtime API into a Substrate runtime -- Configure PVQ extensions for runtime interaction -- Provide a development environment for testing PVQ queries -- Demonstrate best practices for PVQ integration +### How it’s wired -## Features +- Uses the generated `extensions` module to connect extension traits to runtime pallets. +- Executes programs via `ExtensionsExecutor` with invocation source set to runtime API. -- πŸ—οΈ **Complete Runtime**: Full Substrate runtime with PVQ integration -- πŸ”Œ **Extension Support**: Pre-configured with core PVQ extensions -- πŸ§ͺ **Testing Ready**: Optimized for development and testing workflows -- πŸ“‘ **Chopsticks Compatible**: Works with Chopsticks for local development -- βš™οΈ **Configurable**: Easy to modify and extend for custom use cases - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PoC Runtime β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ Standard Pallets β”‚ PVQ Integration β”‚ -β”‚ β”œβ”€ Balances β”‚ β”œβ”€ PvqApi β”‚ -β”‚ β”œβ”€ Assets β”‚ β”œβ”€ Extensions β”‚ -β”‚ β”œβ”€ Timestamp β”‚ └─ Executor β”‚ -β”‚ β”œβ”€ Transaction Payment β”‚ β”‚ -β”‚ └─ Sudo β”‚ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## Runtime Components - -### Standard Pallets - -| Pallet | Purpose | -|--------|---------| -| **Balances** | Native token balance management | -| **Assets** | Multi-asset support for fungible tokens | -| **Timestamp** | Block timestamp functionality | -| **Transaction Payment** | Fee calculation and payment | -| **Sudo** | Privileged operations for testing | - -### PVQ Integration - -| Component | Purpose | -|-----------|---------| -| **PvqApi** | Runtime API for executing PVQ queries | -| **Extensions** | Configured extensions (core, fungibles, swap) | -| **Executor** | PVQ program execution environment | - -## Getting Started - -### Prerequisites - -- Rust toolchain (stable) -- PVQ project dependencies -- Chopsticks (for local development) - -### Building the Runtime - -```bash -# Build the runtime -cargo build -p poc-runtime --release - -# The compiled WASM runtime will be available in target/release/wbuild/ -``` - -### Running with Chopsticks - -1. **Start the local development chain:** - ```bash - make run - ``` - -2. **The runtime will be available at:** - - HTTP RPC: `http://localhost:8000` - - WebSocket: `ws://localhost:8000` - -### Testing PVQ Integration - -1. **Build guest programs:** - ```bash - make guests - ``` - -2. **Test with PVQ runner:** - ```bash - cargo run -p pvq-test-runner -- --program output/guest-total-supply - ``` - -3. **Use with Polkadot JS UI:** - - Connect to `ws://localhost:8000` - - Navigate to Developer β†’ Extrinsics β†’ pvq β†’ executeQuery - - Upload program blob and arguments - -## Configuration - -### Chopsticks Configuration - -The runtime includes a `chopsticks.yml` configuration file: - -```yaml -endpoint: wss://acala-rpc-0.aca-api.network -block: latest -runtime-log-level: 5 -``` - -### Runtime Parameters - -Key runtime configurations: - -```rust -// Maximum execution time for PVQ queries -pub const MAX_EXECUTION_TIME: u64 = 10_000; // 10 seconds - -// Maximum memory allocation for guest programs -pub const MAX_MEMORY: u32 = 16 * 1024 * 1024; // 16MB - -// Supported extensions -pub const EXTENSIONS: &[&str] = &["core", "fungibles", "swap"]; -``` - -## Development Workflow - -### Adding New Pallets - -1. **Add dependency to `Cargo.toml`:** - ```toml - my-custom-pallet = { version = "1.0.0", default-features = false } - ``` - -2. **Configure in `lib.rs`:** - ```rust - impl my_custom_pallet::Config for Runtime { - // Configuration here - } - ``` - -3. **Add to runtime construction:** - ```rust - construct_runtime!( - pub struct Runtime { - // ... existing pallets - MyCustomPallet: my_custom_pallet, - } - ); - ``` - -### Adding PVQ Extensions - -1. **Add extension dependency:** - ```toml - my-pvq-extension = { path = "../my-pvq-extension" } - ``` - -2. **Configure in PVQ setup:** - ```rust - use my_pvq_extension::ExtensionMyCustom; - - // Add to extensions list - ``` - -### Testing Changes +### Build ```bash -# Build and test -cargo build -p poc-runtime -make run - -# Test with guest programs -cargo run -p pvq-test-runner -- --program output/guest-example -``` - -## API Reference - -### PVQ Runtime API - -The runtime exposes the following PVQ API methods: - -#### `execute_query(program: Vec, args: Vec) -> Vec` - -Executes a PVQ program with the provided arguments. - -**Parameters:** -- `program`: Compiled PVQ program blob -- `args`: Serialized input arguments - -**Returns:** Serialized query results - -#### `metadata() -> Vec` - -Returns metadata about available PVQ extensions and capabilities. - -**Returns:** Serialized metadata information - -### Extension APIs - -All standard PVQ extensions are available: - -- **Core Extension**: Basic functionality and extension discovery -- **Fungibles Extension**: Asset queries and balance information -- **Swap Extension**: DEX interactions and liquidity data - -## Troubleshooting - -### Common Issues - -**Runtime doesn't build:** -```bash -# Clean and rebuild -cargo clean cargo build -p poc-runtime --release ``` - -**Chopsticks connection fails:** -```bash -# Check if the endpoint is accessible -curl -X POST -H "Content-Type: application/json" \ - --data '{"jsonrpc":"2.0","method":"system_name","id":1}' \ - http://localhost:8000 -``` - -**PVQ queries fail:** -- Ensure guest programs are compiled correctly -- Check that required extensions are configured -- Verify argument serialization format - -### Debug Mode - -Enable debug logging: - -```bash -RUST_LOG=debug cargo run -p pvq-test-runner -- --program output/guest-example -``` - -## Related Components - -- [PVQ Executor](../pvq-executor/) - Core execution engine -- [PVQ Runtime API](../pvq-runtime-api/) - API definitions -- [PVQ Extensions](../pvq-extension/) - Extension system -- [Guest Examples](../guest-examples/) - Example programs - ---- - -*This PoC runtime demonstrates PVQ integration patterns and serves as a foundation for production implementations.* diff --git a/pvq-extension-core/README.md b/pvq-extension-core/README.md deleted file mode 100644 index 3e12ab0..0000000 --- a/pvq-extension-core/README.md +++ /dev/null @@ -1,384 +0,0 @@ -# PVQ Extension Core - - - -The foundational extension for the PVQ (PolkaVM Query) system, providing essential functionality and serving as the base for all other extensions. Every PVQ program has access to core extension features by default. - -## Overview - -The Core Extension provides fundamental capabilities that all PVQ guest programs need, including extension discovery, system information, and basic utility functions. It serves as the foundation layer that other extensions build upon. - -## Features - -- πŸ” **Extension Discovery**: Query available extensions at runtime -- ℹ️ **System Information**: Access runtime and execution environment details -- πŸ› οΈ **Utility Functions**: Common operations needed by guest programs -- πŸ”’ **Security Primitives**: Basic security and validation functions -- πŸ“Š **Metadata Access**: Extension and system metadata queries - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ ExtensionCore β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Discovery β”‚ β”‚ Utilities β”‚ β”‚ Metadata β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ β”œβ”€ Hashing β”‚ β”‚ β”œβ”€ System β”‚ β”‚ -β”‚ β”‚ β”‚ Enumeration β”‚ β”‚ β”œβ”€ Encoding β”‚ β”‚ β”‚ Info β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Capability β”‚ β”‚ β”œβ”€ Validation β”‚ β”‚ β”œβ”€ Runtime β”‚ β”‚ -β”‚ β”‚ β”‚ Checking β”‚ β”‚ └─ Conversion β”‚ β”‚ β”‚ Version β”‚ β”‚ -β”‚ β”‚ └─ Version β”‚ β”‚ β”‚ β”‚ └─ Extension β”‚ β”‚ -β”‚ β”‚ Matching β”‚ β”‚ β”‚ β”‚ Metadata β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## API Reference - -### Extension Discovery - -#### `extension_exists(name: &str) -> bool` - -Checks if a specific extension is available in the current execution environment. - -```rust -use pvq_extension_core::ExtensionCore; - -// Check if fungibles extension is available -if ExtensionCore::extension_exists("fungibles") { - // Use fungibles extension -} else { - // Fallback logic -} -``` - -#### `list_extensions() -> Vec` - -Returns a list of all available extensions in the current environment. - -```rust -use pvq_extension_core::ExtensionCore; - -let available_extensions = ExtensionCore::list_extensions(); -for extension in available_extensions { - println!("Available: {}", extension); -} -``` - -#### `extension_version(name: &str) -> Option` - -Gets the version of a specific extension. - -```rust -use pvq_extension_core::ExtensionCore; - -if let Some(version) = ExtensionCore::extension_version("fungibles") { - println!("Fungibles extension version: {}", version); -} -``` - -### System Information - -#### `runtime_version() -> String` - -Returns the runtime version information. - -```rust -use pvq_extension_core::ExtensionCore; - -let version = ExtensionCore::runtime_version(); -println!("Runtime version: {}", version); -``` - -#### `execution_info() -> ExecutionInfo` - -Provides information about the current execution environment. - -```rust -use pvq_extension_core::{ExtensionCore, ExecutionInfo}; - -let info = ExtensionCore::execution_info(); -println!("Max memory: {} bytes", info.max_memory); -println!("Execution timeout: {:?}", info.timeout); -``` - -### Utility Functions - -#### `hash_blake2_256(data: &[u8]) -> [u8; 32]` - -Computes Blake2 256-bit hash of the input data. - -```rust -use pvq_extension_core::ExtensionCore; - -let data = b"Hello, PVQ!"; -let hash = ExtensionCore::hash_blake2_256(data); -println!("Hash: {:?}", hash); -``` - -#### `validate_account_id(account: &[u8]) -> bool` - -Validates whether the provided bytes represent a valid account ID. - -```rust -use pvq_extension_core::ExtensionCore; - -let account = [1u8; 32]; -if ExtensionCore::validate_account_id(&account) { - println!("Valid account ID"); -} -``` - -#### `current_block_number() -> u32` - -Returns the current block number. - -```rust -use pvq_extension_core::ExtensionCore; - -let block_number = ExtensionCore::current_block_number(); -println!("Current block: {}", block_number); -``` - -## Data Types - -### ExecutionInfo - -Information about the current execution environment: - -```rust -pub struct ExecutionInfo { - pub max_memory: u64, - pub timeout: Duration, - pub runtime_version: String, - pub extensions_count: usize, -} -``` - -### ExtensionMetadata - -Metadata for an extension: - -```rust -pub struct ExtensionMetadata { - pub name: String, - pub version: String, - pub description: String, - pub functions: Vec, -} -``` - -## Usage Examples - -### Basic Extension Check - -```rust -use pvq_extension_core::ExtensionCore; - -#[pvq_program::program] -fn my_query() -> String { - if ExtensionCore::extension_exists("fungibles") { - "Fungibles extension is available".to_string() - } else { - "Fungibles extension not found".to_string() - } -} -``` - -### System Information Query - -```rust -use pvq_extension_core::ExtensionCore; - -#[pvq_program::program] -fn system_info() -> String { - let runtime_version = ExtensionCore::runtime_version(); - let block_number = ExtensionCore::current_block_number(); - let extensions = ExtensionCore::list_extensions(); - - format!( - "Runtime: {}, Block: {}, Extensions: {:?}", - runtime_version, block_number, extensions - ) -} -``` - -### Conditional Feature Usage - -```rust -use pvq_extension_core::ExtensionCore; -use pvq_extension_fungibles::ExtensionFungibles; -use pvq_extension_swap::ExtensionSwap; - -#[pvq_program::program] -fn advanced_query(asset_id: u32) -> String { - let mut result = String::new(); - - // Always available - core functionality - let block = ExtensionCore::current_block_number(); - result.push_str(&format!("Block: {}\n", block)); - - // Check for fungibles extension - if ExtensionCore::extension_exists("fungibles") { - let supply = ExtensionFungibles::total_supply(asset_id); - result.push_str(&format!("Total Supply: {}\n", supply)); - } - - // Check for swap extension - if ExtensionCore::extension_exists("swap") { - let pools = ExtensionSwap::list_pools(); - result.push_str(&format!("Available Pools: {}\n", pools.len())); - } - - result -} -``` - -## Integration - -### With Other Extensions - -Core extension is automatically available and should be used by other extensions: - -```rust -// In custom extension implementation -use pvq_extension_core::ExtensionCore; - -impl MyExtension for MyExtensionImpl { - fn my_function() -> bool { - // Use core functionality - let block = ExtensionCore::current_block_number(); - - // Extension-specific logic - block > 1000 - } -} -``` - -### Extension Development - -When creating new extensions, leverage core utilities: - -```rust -use pvq_extension_core::ExtensionCore; -use pvq_extension::extension_decl; - -#[extension_decl] -pub trait MyCustomExtension { - fn custom_query() -> String { - // Use core hashing - let data = b"custom data"; - let hash = ExtensionCore::hash_blake2_256(data); - - // Use system info - let version = ExtensionCore::runtime_version(); - - format!("Hash: {:?}, Version: {}", hash, version) - } -} -``` - -## Configuration - -### Runtime Configuration - -Configure core extension behavior: - -```rust -// In runtime configuration -impl pvq_extension_core::Config for Runtime { - type MaxExtensionNameLength = ConstU32<32>; - type MaxExtensionsCount = ConstU32<64>; - type EnableDebugInfo = ConstBool; -} -``` - -### Extension Registry - -Register core extension in executor: - -```rust -use pvq_extension::ExtensionsExecutor; -use pvq_extension_core::ExtensionCore; - -let mut executor = ExtensionsExecutor::new(); -executor.register::(); -``` - -## Development - -### Building - -```bash -# Build core extension -cargo build -p pvq-extension-core - -# Run tests -cargo test -p pvq-extension-core - -# Check documentation -cargo doc -p pvq-extension-core --open -``` - -### Testing - -```bash -# Unit tests -cargo test -p pvq-extension-core - -# Integration tests with executor -cargo test -p pvq-extension-core --test integration - -# Test with guest programs -cargo run -p pvq-test-runner -- --program test-core-extension -``` - -## Security Considerations - -- All core functions are safe and do not expose sensitive runtime internals -- Extension discovery is read-only and cannot modify system state -- Hash functions use cryptographically secure algorithms -- Account ID validation prevents invalid account access attempts - -## Performance - -### Optimization Tips - -1. **Cache Extension List**: Extension discovery results can be cached -2. **Batch Queries**: Combine multiple core queries when possible -3. **Selective Checks**: Only check for extensions you actually need - -### Benchmarks - -```bash -# Run core extension benchmarks -cargo bench -p pvq-extension-core - -# Profile extension discovery -cargo run --example profile_discovery --release -``` - -## Error Handling - -Core extension functions are designed to be safe and rarely fail: - -```rust -// Most functions return safe defaults -let extensions = ExtensionCore::list_extensions(); // Never fails - -// Optional results for queries that might not have data -let version = ExtensionCore::extension_version("nonexistent"); // Returns None -``` - -## Related Components - -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Fungibles Extension](../pvq-extension-fungibles/) - Asset functionality -- [PVQ Swap Extension](../pvq-extension-swap/) - DEX functionality -- [PVQ Executor](../pvq-executor/) - Execution environment - ---- - -*The Core Extension provides the essential foundation for all PVQ program functionality.* \ No newline at end of file diff --git a/pvq-extension-fungibles/README.md b/pvq-extension-fungibles/README.md deleted file mode 100644 index 37306fd..0000000 --- a/pvq-extension-fungibles/README.md +++ /dev/null @@ -1,513 +0,0 @@ -# PVQ Extension Fungibles - - - -A powerful extension for the PVQ (PolkaVM Query) system that enables guest programs to query fungible asset information from the runtime. This extension provides comprehensive access to asset balances, metadata, and supply information. - -## Overview - -The Fungibles Extension integrates with Substrate's assets pallet and native balance systems to provide PVQ guest programs with read-only access to fungible token data. It supports both native tokens and custom assets created through the assets pallet. - -## Features - -- πŸ’° **Balance Queries**: Get account balances for any asset -- πŸ“Š **Supply Information**: Query total supply and supply metrics -- ℹ️ **Asset Metadata**: Access asset names, symbols, and decimals -- πŸ” **Asset Discovery**: List and enumerate available assets -- 🏦 **Multi-Asset Support**: Native tokens and custom assets -- ⚑ **Efficient Queries**: Optimized for high-performance data access - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ ExtensionFungibles β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Balances β”‚ β”‚ Assets β”‚ β”‚ Metadata β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Account β”‚ β”‚ β”œβ”€ Total Supply β”‚ β”‚ β”œβ”€ Name β”‚ β”‚ -β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Minimum β”‚ β”‚ β”œβ”€ Symbol β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Free β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Decimals β”‚ β”‚ -β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”œβ”€ Asset Exists β”‚ β”‚ └─ Custom β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Reserved β”‚ β”‚ └─ Asset List β”‚ β”‚ Fields β”‚ β”‚ -β”‚ β”‚ β”‚ Balance β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ └─ Locked β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ Balance β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## API Reference - -### Balance Operations - -#### `balance(asset_id: u32, account: &[u8; 32]) -> u128` - -Get the total balance (free + reserved) of an account for a specific asset. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn get_account_balance(asset_id: u32, account: [u8; 32]) -> u128 { - ExtensionFungibles::balance(asset_id, &account) -} -``` - -#### `balance_free(asset_id: u32, account: &[u8; 32]) -> u128` - -Get the free (transferable) balance of an account. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -let free_balance = ExtensionFungibles::balance_free(asset_id, &account); -println!("Free balance: {}", free_balance); -``` - -#### `balance_reserved(asset_id: u32, account: &[u8; 32]) -> u128` - -Get the reserved (non-transferable) balance of an account. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -let reserved_balance = ExtensionFungibles::balance_reserved(asset_id, &account); -println!("Reserved balance: {}", reserved_balance); -``` - -### Asset Information - -#### `total_supply(asset_id: u32) -> u128` - -Get the total supply of an asset. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn asset_total_supply(asset_id: u32) -> u128 { - ExtensionFungibles::total_supply(asset_id) -} -``` - -#### `minimum_balance(asset_id: u32) -> u128` - -Get the minimum balance required to maintain an account. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -let min_balance = ExtensionFungibles::minimum_balance(asset_id); -println!("Minimum balance: {}", min_balance); -``` - -#### `asset_exists(asset_id: u32) -> bool` - -Check if an asset exists in the runtime. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -if ExtensionFungibles::asset_exists(asset_id) { - // Asset is valid - let supply = ExtensionFungibles::total_supply(asset_id); -} else { - // Asset doesn't exist -} -``` - -### Asset Discovery - -#### `list_assets() -> Vec` - -Get a list of all available asset IDs. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn available_assets() -> Vec { - ExtensionFungibles::list_assets() -} -``` - -#### `asset_count() -> u32` - -Get the total number of assets in the system. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -let count = ExtensionFungibles::asset_count(); -println!("Total assets: {}", count); -``` - -### Metadata Operations - -#### `asset_metadata(asset_id: u32) -> Option` - -Get complete metadata for an asset. - -```rust -use pvq_extension_fungibles::{ExtensionFungibles, AssetMetadata}; - -if let Some(metadata) = ExtensionFungibles::asset_metadata(asset_id) { - println!("Name: {}", metadata.name); - println!("Symbol: {}", metadata.symbol); - println!("Decimals: {}", metadata.decimals); -} -``` - -#### `asset_name(asset_id: u32) -> Option` - -Get the name of an asset. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -if let Some(name) = ExtensionFungibles::asset_name(asset_id) { - println!("Asset name: {}", name); -} -``` - -#### `asset_symbol(asset_id: u32) -> Option` - -Get the symbol of an asset. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -if let Some(symbol) = ExtensionFungibles::asset_symbol(asset_id) { - println!("Asset symbol: {}", symbol); -} -``` - -#### `asset_decimals(asset_id: u32) -> Option` - -Get the decimal precision of an asset. - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -if let Some(decimals) = ExtensionFungibles::asset_decimals(asset_id) { - println!("Asset decimals: {}", decimals); -} -``` - -## Data Types - -### AssetMetadata - -Complete asset metadata information: - -```rust -pub struct AssetMetadata { - pub name: String, - pub symbol: String, - pub decimals: u8, - pub is_frozen: bool, -} -``` - -### BalanceInfo - -Comprehensive balance information: - -```rust -pub struct BalanceInfo { - pub free: u128, - pub reserved: u128, - pub frozen: u128, -} -``` - -## Usage Examples - -### Basic Balance Query - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn check_balance(asset_id: u32, account: [u8; 32]) -> String { - let balance = ExtensionFungibles::balance(asset_id, &account); - - if balance == 0 { - "Account has no balance".to_string() - } else { - format!("Balance: {}", balance) - } -} -``` - -### Multi-Account Balance Sum - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn sum_balances(asset_id: u32, accounts: Vec<[u8; 32]>) -> u128 { - accounts.iter() - .map(|account| ExtensionFungibles::balance(asset_id, account)) - .sum() -} -``` - -### Asset Analysis - -```rust -use pvq_extension_fungibles::{ExtensionFungibles, AssetMetadata}; - -#[pvq_program::program] -fn analyze_asset(asset_id: u32) -> String { - if !ExtensionFungibles::asset_exists(asset_id) { - return "Asset does not exist".to_string(); - } - - let supply = ExtensionFungibles::total_supply(asset_id); - let min_balance = ExtensionFungibles::minimum_balance(asset_id); - - let metadata = ExtensionFungibles::asset_metadata(asset_id); - - match metadata { - Some(meta) => { - format!( - "Asset {}: {} ({}) - Supply: {}, Min Balance: {}, Decimals: {}", - asset_id, meta.name, meta.symbol, supply, min_balance, meta.decimals - ) - } - None => { - format!( - "Asset {}: No metadata - Supply: {}, Min Balance: {}", - asset_id, supply, min_balance - ) - } - } -} -``` - -### Portfolio Tracker - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn portfolio_value(account: [u8; 32]) -> Vec<(u32, u128)> { - let assets = ExtensionFungibles::list_assets(); - let mut portfolio = Vec::new(); - - for asset_id in assets { - let balance = ExtensionFungibles::balance(asset_id, &account); - if balance > 0 { - portfolio.push((asset_id, balance)); - } - } - - portfolio -} -``` - -### Percentage of Total Supply - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn balance_percentage(asset_id: u32, account: [u8; 32]) -> f64 { - let balance = ExtensionFungibles::balance(asset_id, &account); - let total_supply = ExtensionFungibles::total_supply(asset_id); - - if total_supply == 0 { - 0.0 - } else { - (balance as f64 / total_supply as f64) * 100.0 - } -} -``` - -## Integration - -### With Runtime - -Configure the fungibles extension in your runtime: - -```rust -// In runtime configuration -impl pvq_extension_fungibles::Config for Runtime { - type AssetId = u32; - type Balance = u128; - type AssetRegistry = Assets; - type NativeBalance = Balances; -} -``` - -### With Extension Executor - -Register the fungibles extension: - -```rust -use pvq_extension::ExtensionsExecutor; -use pvq_extension_fungibles::ExtensionFungibles; - -let mut executor = ExtensionsExecutor::new(); -executor.register::(); -``` - -### Error Handling - -Handle potential query failures: - -```rust -use pvq_extension_fungibles::ExtensionFungibles; - -#[pvq_program::program] -fn safe_balance_query(asset_id: u32, account: [u8; 32]) -> String { - if !ExtensionFungibles::asset_exists(asset_id) { - return "Asset not found".to_string(); - } - - let balance = ExtensionFungibles::balance(asset_id, &account); - format!("Balance: {}", balance) -} -``` - -## Configuration - -### Asset ID Types - -Support different asset ID types: - -```rust -impl pvq_extension_fungibles::Config for Runtime { - // For simple numeric IDs - type AssetId = u32; - - // For complex asset identifiers - type AssetId = AssetIdOf; -} -``` - -### Balance Types - -Configure balance precision: - -```rust -impl pvq_extension_fungibles::Config for Runtime { - // Standard 128-bit balance - type Balance = u128; - - // Custom balance type - type Balance = BalanceOf; -} -``` - -## Development - -### Building - -```bash -# Build fungibles extension -cargo build -p pvq-extension-fungibles - -# Run tests -cargo test -p pvq-extension-fungibles - -# Generate documentation -cargo doc -p pvq-extension-fungibles --open -``` - -### Testing - -```bash -# Unit tests -cargo test -p pvq-extension-fungibles - -# Integration tests with runtime -cargo test -p pvq-extension-fungibles --test runtime_integration - -# Test with guest programs -cargo run -p pvq-test-runner -- --program test-fungibles -``` - -### Benchmarking - -```bash -# Run performance benchmarks -cargo bench -p pvq-extension-fungibles - -# Profile balance queries -cargo run --example profile_queries --release -``` - -## Performance Considerations - -### Optimization Tips - -1. **Batch Queries**: Query multiple balances in a single program -2. **Asset Filtering**: Use `asset_exists()` before expensive operations -3. **Cache Metadata**: Store frequently accessed metadata -4. **Selective Fields**: Only query the specific data you need - -### Query Patterns - -```rust -// Efficient: Single program, multiple accounts -#[pvq_program::program] -fn efficient_multi_balance(asset_id: u32, accounts: Vec<[u8; 32]>) -> Vec { - accounts.iter() - .map(|account| ExtensionFungibles::balance(asset_id, account)) - .collect() -} - -// Less efficient: Multiple program calls -// Don't do this - call the program once with all accounts -``` - -## Security Considerations - -- All functions are read-only and cannot modify balances -- Account data access is controlled by the runtime -- Asset existence checks prevent invalid asset queries -- No sensitive runtime internals are exposed - -## Common Use Cases - -### DeFi Applications - -```rust -// Calculate liquidity pool ratios -#[pvq_program::program] -fn pool_ratio(token_a: u32, token_b: u32, pool_account: [u8; 32]) -> (u128, u128) { - let balance_a = ExtensionFungibles::balance(token_a, &pool_account); - let balance_b = ExtensionFungibles::balance(token_b, &pool_account); - (balance_a, balance_b) -} -``` - -### Analytics - -```rust -// Top holders analysis -#[pvq_program::program] -fn top_holders_share(asset_id: u32, holders: Vec<[u8; 32]>) -> f64 { - let total_supply = ExtensionFungibles::total_supply(asset_id); - let holders_balance: u128 = holders.iter() - .map(|account| ExtensionFungibles::balance(asset_id, account)) - .sum(); - - (holders_balance as f64 / total_supply as f64) * 100.0 -} -``` - -## Related Components - -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Extension Core](../pvq-extension-core/) - Core functionality -- [PVQ Extension Swap](../pvq-extension-swap/) - DEX functionality -- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration - ---- - -*The Fungibles Extension provides comprehensive access to asset and balance data for PVQ programs.* \ No newline at end of file diff --git a/pvq-extension/README.md b/pvq-extension/README.md index 7b6bdfb..abc11cd 100644 --- a/pvq-extension/README.md +++ b/pvq-extension/README.md @@ -1,552 +1,27 @@ -# PVQ Extension System +# PVQ Extension - +Extension framework for PVQ with declarative interfaces, metadata generation, execution, and permissions. -The foundational extension framework for PVQ (PolkaVM Query) that enables modular and secure interaction between guest programs and runtime functionality. This system provides the architecture for creating, registering, and executing extensions. +### What it provides -## Overview +- Declarative extension traits and implementations via macros +- Generated dispatch and metadata for extensions +- Execution wrapper to run PVQ programs against your extensions +- Pluggable permission controller (allow‑all by default) -The PVQ Extension System serves as the bridge between sandboxed guest programs and the Substrate runtime. It provides a secure, permission-controlled mechanism for exposing runtime functionality to PVQ programs through a modular extension architecture. +### Typical use -## Features +1. Declare an extension trait and implement it for your runtime. +2. Generate an `extensions` module (via the macros) and expose its `metadata()`. +3. Initialize `ExtensionsExecutor` with your generated `extensions` and an invocation source. +4. Optionally implement a custom `PermissionController`. -- 🧩 **Modular Architecture**: Plugin-based extension system -- πŸ”’ **Security Controls**: Fine-grained permission management -- πŸ“ **Procedural Macros**: Simplified extension development with `#[extension_decl]` and `#[extensions_impl]` -- ⚑ **High Performance**: Optimized call dispatch and execution -- πŸ›‘οΈ **Type Safety**: Compile-time validation of extension interfaces -- πŸ”Œ **Hot-Pluggable**: Runtime extension registration and discovery +### Metadata -## Architecture +- A portable type registry plus per‑extension function signatures (names, inputs, output). Useful to serve from your runtime’s `metadata()`. -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PVQ Extension System β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Extension β”‚ β”‚ Extensions β”‚ β”‚ Permission β”‚ β”‚ -β”‚ β”‚ Declaration β”‚ β”‚ Executor β”‚ β”‚ Controller β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Trait Def β”‚ β”‚ β”œβ”€ Registry β”‚ β”‚ β”œβ”€ Access β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Metadata β”‚ β”‚ β”œβ”€ Dispatch β”‚ β”‚ β”‚ Control β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Function β”‚ β”‚ β”œβ”€ Context β”‚ β”‚ β”œβ”€ Policy β”‚ β”‚ -β”‚ β”‚ β”‚ Signatures β”‚ β”‚ β”‚ Management β”‚ β”‚ β”‚ Engine β”‚ β”‚ -β”‚ β”‚ └─ Validation β”‚ β”‚ └─ Error β”‚ β”‚ └─ Audit β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Handling β”‚ β”‚ Trail β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## Core Components - -### Extension Declaration (`#[extension_decl]`) - -Define new extensions using the procedural macro: - -```rust -use pvq_extension::extension_decl; - -#[extension_decl] -pub trait MyExtension { - /// Get system information - fn get_info() -> String; - - /// Process data with validation - fn process_data(input: Vec) -> Result, String>; - - /// Query with parameters - fn query_with_params(id: u32, filter: Option) -> Vec; -} -``` - -### Extension Implementation (`#[extensions_impl]`) - -Implement extensions for runtime integration: - -```rust -use pvq_extension::extensions_impl; - -pub struct MyExtensionImpl; - -#[extensions_impl] -impl MyExtension for MyExtensionImpl { - fn get_info() -> String { - "System v1.0".to_string() - } - - fn process_data(input: Vec) -> Result, String> { - if input.is_empty() { - Err("Empty input".to_string()) - } else { - Ok(input.into_iter().map(|b| b.wrapping_add(1)).collect()) - } - } - - fn query_with_params(id: u32, filter: Option) -> Vec { - // Implementation logic - vec![id] - } -} -``` - -### Extensions Executor - -Manages extension registration and execution: - -```rust -use pvq_extension::ExtensionsExecutor; - -let mut executor = ExtensionsExecutor::new(); - -// Register extensions -executor.register::(); -executor.register::(); - -// Execute extension calls from guest programs -let result = executor.execute_call("MyExtension", "get_info", &[])?; -``` - -### Permission Controller - -Controls access to extensions and functions: - -```rust -use pvq_extension::{PermissionController, Permission}; - -let mut controller = PermissionController::new(); - -// Grant specific permissions -controller.grant_permission( - "guest_program_id", - Permission::Extension("MyExtension".to_string()) -); - -// Check permissions before execution -if controller.check_permission("guest_program_id", &permission) { - // Execute extension call -} -``` - -## Usage Guide - -### Creating a New Extension - -1. **Define the Extension Trait:** - -```rust -use pvq_extension::extension_decl; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize)] -pub struct CustomData { - pub value: u64, - pub name: String, -} - -#[extension_decl] -pub trait CustomExtension { - /// Get custom data by ID - fn get_data(id: u32) -> Option; - - /// List all available data IDs - fn list_data_ids() -> Vec; - - /// Validate data format - fn validate_data(data: CustomData) -> bool; -} -``` - -2. **Implement the Extension:** - -```rust -use pvq_extension::extensions_impl; - -pub struct CustomExtensionImpl { - _phantom: std::marker::PhantomData, -} - -#[extensions_impl] -impl CustomExtension for CustomExtensionImpl { - fn get_data(id: u32) -> Option { - // Access runtime storage - T::Storage::get_custom_data(id) - } - - fn list_data_ids() -> Vec { - T::Storage::list_custom_data_ids() - } - - fn validate_data(data: CustomData) -> bool { - !data.name.is_empty() && data.value > 0 - } -} -``` - -3. **Register in Runtime:** - -```rust -use pvq_extension::ExtensionsExecutor; - -impl pvq_runtime_api::PvqApi for Runtime { - fn execute_query(&self, program: Vec, args: Vec) -> Vec { - let mut extensions = ExtensionsExecutor::new(); - - // Register standard extensions - extensions.register::(); - extensions.register::(); - - // Register custom extension - extensions.register::(); - - // Execute program with extensions - let context = PvqExecutorContext::new(extensions); - let executor = PvqExecutor::new(context); - - executor.execute(&program, &args).unwrap_or_default() - } -} -``` - -### Using Extensions in Guest Programs - -```rust -use pvq_program::program; -use custom_extension::CustomExtension; - -#[program] -fn query_custom_data(id: u32) -> String { - match CustomExtension::get_data(id) { - Some(data) => { - format!("Data: {} = {}", data.name, data.value) - } - None => { - "Data not found".to_string() - } - } -} - -#[program] -fn validate_and_list() -> Vec { - let all_ids = CustomExtension::list_data_ids(); - - all_ids.into_iter() - .filter(|&id| { - if let Some(data) = CustomExtension::get_data(id) { - CustomExtension::validate_data(data) - } else { - false - } - }) - .collect() -} -``` - -## Advanced Features - -### Permission Management - -Configure fine-grained access control: - -```rust -use pvq_extension::{PermissionController, Permission, PermissionPolicy}; - -let mut controller = PermissionController::new(); - -// Create permission policy -let policy = PermissionPolicy::new() - .allow_extension("core") - .allow_extension("fungibles") - .deny_function("fungibles", "admin_function") - .with_rate_limit(100, Duration::from_secs(60)); - -// Apply policy to program -controller.set_policy("program_id", policy); -``` - -### Extension Metadata - -Generate and access extension metadata: - -```rust -use pvq_extension::{ExtensionMetadata, FunctionMetadata}; - -// Metadata is automatically generated from extension declarations -let metadata = ExtensionsExecutor::get_extension_metadata("CustomExtension"); - -match metadata { - Some(meta) => { - println!("Extension: {}", meta.name); - println!("Version: {}", meta.version); - - for function in meta.functions { - println!("Function: {}", function.name); - println!("Parameters: {:?}", function.parameters); - println!("Return Type: {:?}", function.return_type); - } - } - None => println!("Extension not found"), -} -``` - -### Error Handling - -Handle extension errors gracefully: - -```rust -use pvq_extension::{ExtensionError, ExtensionResult}; - -#[extension_decl] -pub trait SafeExtension { - fn safe_operation(input: u32) -> ExtensionResult; -} - -#[extensions_impl] -impl SafeExtension for SafeExtensionImpl { - fn safe_operation(input: u32) -> ExtensionResult { - if input == 0 { - Err(ExtensionError::InvalidInput("Input cannot be zero".to_string())) - } else if input > 1000 { - Err(ExtensionError::OutOfRange("Input too large".to_string())) - } else { - Ok(format!("Processed: {}", input)) - } - } -} -``` - -## Configuration - -### Runtime Configuration - -Configure the extension system in your runtime: - -```rust -impl pvq_extension::Config for Runtime { - type MaxExtensions = ConstU32<64>; - type MaxFunctionsPerExtension = ConstU32<32>; - type EnablePermissionControl = ConstBool; - type EnableMetadataGeneration = ConstBool; -} -``` - -### Extension Registration - -Register extensions with configuration: - -```rust -use pvq_extension::{ExtensionsExecutor, ExtensionConfig}; - -let mut executor = ExtensionsExecutor::new(); - -let config = ExtensionConfig { - enable_caching: true, - max_call_depth: 10, - timeout: Duration::from_secs(30), -}; - -executor.register_with_config::(config); -``` - -## Development Tools - -### Testing Extensions - -```rust -#[cfg(test)] -mod tests { - use super::*; - use pvq_extension::testing::ExtensionTester; - - #[test] - fn test_extension_functionality() { - let tester = ExtensionTester::new(); - tester.register::(); - - // Test extension calls - let result = tester.call("CustomExtension", "get_data", &[1u32]); - assert!(result.is_ok()); - } - - #[test] - fn test_permission_enforcement() { - let mut tester = ExtensionTester::new(); - tester.set_permission_policy(restrictive_policy()); - - // This should fail due to permissions - let result = tester.call("CustomExtension", "admin_function", &[]); - assert!(matches!(result, Err(ExtensionError::PermissionDenied(_)))); - } -} -``` - -### Debugging - -Enable detailed logging: - -```rust -use log::debug; - -#[extensions_impl] -impl CustomExtension for CustomExtensionImpl { - fn debug_function(input: String) -> String { - debug!("CustomExtension::debug_function called with: {}", input); - - let result = process_input(&input); - - debug!("CustomExtension::debug_function returning: {}", result); - result - } -} -``` - -Run with debug logging: +### Build ```bash -RUST_LOG=pvq_extension=debug,custom_extension=debug cargo test +cargo build -p pvq-extension ``` - -## Performance Considerations - -### Optimization Tips - -1. **Minimize Allocations**: Use stack allocation when possible -2. **Batch Operations**: Group related calls into single functions -3. **Cache Results**: Cache expensive computations within extensions -4. **Limit Recursion**: Avoid deep call chains between extensions - -### Benchmarking - -```bash -# Benchmark extension performance -cargo bench -p pvq-extension - -# Profile extension calls -cargo run --example profile_extensions --release -``` - -### Example Optimizations - -```rust -// Efficient: Batch operation -#[extension_decl] -pub trait OptimizedExtension { - fn batch_process(items: Vec) -> Vec; -} - -// Less efficient: Individual calls -#[extension_decl] -pub trait UnoptimizedExtension { - fn process_single(item: ProcessItem) -> ProcessResult; -} -``` - -## Security Best Practices - -### Input Validation - -Always validate inputs in extension implementations: - -```rust -#[extensions_impl] -impl SecureExtension for SecureExtensionImpl { - fn secure_function(user_input: String) -> Result { - // Validate input length - if user_input.len() > 1000 { - return Err("Input too long".to_string()); - } - - // Validate input format - if !user_input.chars().all(|c| c.is_alphanumeric()) { - return Err("Invalid characters".to_string()); - } - - // Safe processing - Ok(process_validated_input(&user_input)) - } -} -``` - -### Permission Boundaries - -Implement proper access controls: - -```rust -use pvq_extension::{Permission, PermissionController}; - -fn setup_permissions() -> PermissionController { - let mut controller = PermissionController::new(); - - // Public functions - accessible to all programs - controller.grant_global_permission(Permission::Function( - "PublicExtension".to_string(), - "public_query".to_string() - )); - - // Admin functions - restricted access - controller.grant_permission( - "admin_program_id", - Permission::Function( - "AdminExtension".to_string(), - "admin_function".to_string() - ) - ); - - controller -} -``` - -## Migration Guide - -### Upgrading Extensions - -When updating extension interfaces: - -1. **Version your extensions:** - -```rust -#[extension_decl] -pub trait MyExtensionV2 { - // Keep old functions for compatibility - fn old_function() -> String; - - // Add new functions - fn new_function() -> EnhancedResult; -} -``` - -2. **Provide migration path:** - -```rust -// Implement both versions -impl MyExtension for MyExtensionImpl { /* old implementation */ } -impl MyExtensionV2 for MyExtensionImpl { /* new implementation */ } -``` - -### Breaking Changes - -Handle breaking changes gracefully: - -```rust -#[extension_decl] -pub trait VersionedExtension { - fn get_version() -> u32 { 2 } - - fn function_v2(input: NewInputType) -> NewOutputType; - - // Deprecated but maintained for compatibility - #[deprecated] - fn function_v1(input: OldInputType) -> OldOutputType; -} -``` - -## Related Components - -- [PVQ Extension Core](../pvq-extension-core/) - Core extension functionality -- [PVQ Extension Fungibles](../pvq-extension-fungibles/) - Asset functionality -- [PVQ Extension Swap](../pvq-extension-swap/) - DEX functionality -- [PVQ Executor](../pvq-executor/) - Execution environment -- [PVQ Program](../pvq-program/) - Program development tools - ---- - -*The PVQ Extension System provides the foundation for modular, secure runtime interaction in PVQ programs.* \ No newline at end of file diff --git a/pvq-primitives/README.md b/pvq-primitives/README.md index ebb2d5e..4203475 100644 --- a/pvq-primitives/README.md +++ b/pvq-primitives/README.md @@ -1,703 +1,17 @@ # PVQ Primitives - +Lightweight types shared across PVQ crates. -Core primitive types and utilities for the PVQ (PolkaVM Query) ecosystem. This crate provides fundamental data structures, error types, and serialization utilities that ensure interoperability across all PVQ components. +- PvqResponse: raw bytes returned from a PVQ program +- PvqResult: success bytes or error +- PvqError: in `std` it is a string message; in `no_std` it is a compact enum -## Overview +### Behavior -PVQ Primitives serves as the foundational layer for all PVQ components, providing standardized types and utilities that enable seamless communication between guest programs, extensions, executors, and runtime APIs. +- No helpers or codecs are provided here; this crate only defines the basic types used by executors, extensions, and runtime APIs. -## Features - -- πŸ“¦ **Core Data Types**: Fundamental types used across the PVQ ecosystem -- πŸ”„ **Serialization Support**: Efficient encoding/decoding with multiple formats -- ❌ **Error Handling**: Comprehensive error types with detailed context -- πŸ”— **Interoperability**: Consistent types across all PVQ components -- πŸ“ **Size Optimization**: Compact representations for performance -- πŸ›‘οΈ **Type Safety**: Strong typing to prevent runtime errors - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PVQ Primitives β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Core Types β”‚ β”‚ Error Types β”‚ β”‚ Utilities β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ PvqResult β”‚ β”‚ β”œβ”€ PvqError β”‚ β”‚ β”œβ”€ Codec β”‚ β”‚ -β”‚ β”‚ β”œβ”€ PvqResponse β”‚ β”‚ β”œβ”€ ErrorCode β”‚ β”‚ β”œβ”€ Hashing β”‚ β”‚ -β”‚ β”‚ β”œβ”€ QueryData β”‚ β”‚ β”œβ”€ ErrorContext β”‚ β”‚ β”œβ”€ Validationβ”‚ β”‚ -β”‚ β”‚ β”œβ”€ Metadata β”‚ β”‚ └─ ErrorChain β”‚ β”‚ └─ Constants β”‚ β”‚ -β”‚ β”‚ └─ Context β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## Core Types - -### PvqResult - -The standard result type for PVQ operations: - -```rust -use pvq_primitives::{PvqResult, PvqError}; - -pub type PvqResult = Result; - -// Usage example -fn example_function() -> PvqResult { - Ok("Success".to_string()) -} - -// Error handling -match example_function() { - Ok(value) => println!("Success: {}", value), - Err(error) => println!("Error: {}", error), -} -``` - -### PvqResponse - -Standard response wrapper for query results: - -```rust -use pvq_primitives::{PvqResponse, QueryMetadata}; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize)] -pub struct PvqResponse { - pub data: T, - pub metadata: QueryMetadata, - pub success: bool, -} - -impl PvqResponse { - pub fn success(data: T) -> Self { - Self { - data, - metadata: QueryMetadata::default(), - success: true, - } - } - - pub fn error(data: T, error: String) -> Self { - Self { - data, - metadata: QueryMetadata::with_error(error), - success: false, - } - } -} - -// Usage in guest programs -#[pvq_program::program] -fn my_query() -> PvqResponse { - PvqResponse::success("Query completed".to_string()) -} -``` - -### QueryData - -Standardized input/output data format: - -```rust -use pvq_primitives::QueryData; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize)] -pub struct QueryData { - pub payload: T, - pub version: u32, - pub timestamp: u64, -} - -impl QueryData { - pub fn new(payload: T) -> Self { - Self { - payload, - version: 1, - timestamp: current_timestamp(), - } - } -} -``` - -## Error Handling - -### PvqError - -Comprehensive error type covering all failure scenarios: - -```rust -use pvq_primitives::PvqError; -use thiserror::Error; - -#[derive(Error, Debug, Clone, PartialEq)] -pub enum PvqError { - #[error("Program execution failed: {message}")] - ExecutionError { - message: String, - code: u32, - }, - - #[error("Extension call failed: {extension}.{function}")] - ExtensionError { - extension: String, - function: String, - reason: String, - }, - - #[error("Serialization error: {0}")] - SerializationError(String), - - #[error("Permission denied: {operation}")] - PermissionDenied { - operation: String - }, - - #[error("Resource limit exceeded: {resource}")] - ResourceLimitExceeded { - resource: String, - limit: u64, - requested: u64, - }, - - #[error("Invalid input: {field}")] - InvalidInput { - field: String, - expected: String, - received: String, - }, - - #[error("Timeout after {duration_ms}ms")] - Timeout { - duration_ms: u64 - }, - - #[error("Internal error: {0}")] - Internal(String), -} -``` - -### Error Construction Utilities - -Helper functions for creating common errors: - -```rust -use pvq_primitives::PvqError; - -impl PvqError { - pub fn execution_failed(message: impl Into) -> Self { - Self::ExecutionError { - message: message.into(), - code: 1001, - } - } - - pub fn extension_failed( - extension: impl Into, - function: impl Into, - reason: impl Into, - ) -> Self { - Self::ExtensionError { - extension: extension.into(), - function: function.into(), - reason: reason.into(), - } - } - - pub fn invalid_input( - field: impl Into, - expected: impl Into, - received: impl Into, - ) -> Self { - Self::InvalidInput { - field: field.into(), - expected: expected.into(), - received: received.into(), - } - } -} - -// Usage -fn validate_input(value: u32) -> PvqResult { - if value == 0 { - Err(PvqError::invalid_input( - "value", - "non-zero number", - "0" - )) - } else { - Ok(value) - } -} -``` - -## Metadata Types - -### QueryMetadata - -Metadata about query execution: - -```rust -use pvq_primitives::QueryMetadata; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct QueryMetadata { - pub execution_time_ms: u64, - pub memory_used_bytes: u64, - pub extension_calls: u32, - pub block_number: u32, - pub error_message: Option, -} - -impl QueryMetadata { - pub fn new() -> Self { - Self { - execution_time_ms: 0, - memory_used_bytes: 0, - extension_calls: 0, - block_number: 0, - error_message: None, - } - } - - pub fn with_error(error: String) -> Self { - Self { - error_message: Some(error), - ..Self::new() - } - } -} -``` - -### ExtensionMetadata - -Metadata for extension functions: - -```rust -use pvq_primitives::ExtensionMetadata; - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct ExtensionMetadata { - pub name: String, - pub version: String, - pub description: String, - pub functions: Vec, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct FunctionMetadata { - pub name: String, - pub parameters: Vec, - pub return_type: TypeInfo, - pub description: String, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct ParameterInfo { - pub name: String, - pub type_info: TypeInfo, - pub optional: bool, - pub description: String, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct TypeInfo { - pub name: String, - pub size_hint: Option, - pub is_collection: bool, -} -``` - -## Serialization Utilities - -### Codec Support - -Support for multiple serialization formats: - -```rust -use pvq_primitives::codec::{Encode, Decode, Codec}; -use serde::{Serialize, Deserialize}; - -// Types that work with both SCALE and Serde -#[derive(Encode, Decode, Serialize, Deserialize)] -pub struct MyData { - pub value: u64, - pub name: String, -} - -// Codec utilities -pub fn encode_data(data: &T) -> Vec { - data.encode() -} - -pub fn decode_data(bytes: &[u8]) -> PvqResult { - T::decode(&mut &bytes[..]) - .map_err(|e| PvqError::SerializationError(e.to_string())) -} - -// JSON utilities -pub fn to_json(data: &T) -> PvqResult { - serde_json::to_string(data) - .map_err(|e| PvqError::SerializationError(e.to_string())) -} - -pub fn from_json Deserialize<'de>>(json: &str) -> PvqResult { - serde_json::from_str(json) - .map_err(|e| PvqError::SerializationError(e.to_string())) -} -``` - -### Custom Serialization - -For performance-critical applications: - -```rust -use pvq_primitives::codec::{Encode, Decode, Error}; - -pub struct CompactData { - pub numbers: Vec, - pub flags: u8, -} - -impl Encode for CompactData { - fn encode(&self) -> Vec { - let mut bytes = Vec::new(); - - // Encode length + data - bytes.extend((self.numbers.len() as u32).to_le_bytes()); - for &num in &self.numbers { - bytes.extend(num.to_le_bytes()); - } - bytes.push(self.flags); - - bytes - } -} - -impl Decode for CompactData { - fn decode(input: &mut I) -> Result { - let len = u32::decode(input)? as usize; - let mut numbers = Vec::with_capacity(len); - - for _ in 0..len { - numbers.push(u32::decode(input)?); - } - - let flags = u8::decode(input)?; - - Ok(Self { numbers, flags }) - } -} -``` - -## Constants and Configuration - -### System Constants - -Common constants used across PVQ: - -```rust -use pvq_primitives::constants::*; - -// Execution limits -pub const MAX_EXECUTION_TIME_MS: u64 = 30_000; -pub const MAX_MEMORY_BYTES: u64 = 64 * 1024 * 1024; // 64MB -pub const MAX_CALL_DEPTH: u32 = 256; - -// Data limits -pub const MAX_INPUT_SIZE_BYTES: usize = 1024 * 1024; // 1MB -pub const MAX_OUTPUT_SIZE_BYTES: usize = 10 * 1024 * 1024; // 10MB -pub const MAX_EXTENSION_NAME_LENGTH: usize = 64; - -// Protocol versions -pub const PVQ_PROTOCOL_VERSION: u32 = 1; -pub const MIN_SUPPORTED_VERSION: u32 = 1; -pub const MAX_SUPPORTED_VERSION: u32 = 1; -``` - -### Configuration Types - -Runtime configuration structures: - -```rust -use pvq_primitives::config::*; - -#[derive(Debug, Clone)] -pub struct PvqConfig { - pub max_execution_time_ms: u64, - pub max_memory_bytes: u64, - pub max_extensions: u32, - pub enable_debugging: bool, - pub permission_model: PermissionModel, -} - -impl Default for PvqConfig { - fn default() -> Self { - Self { - max_execution_time_ms: MAX_EXECUTION_TIME_MS, - max_memory_bytes: MAX_MEMORY_BYTES, - max_extensions: 32, - enable_debugging: false, - permission_model: PermissionModel::Strict, - } - } -} - -#[derive(Debug, Clone)] -pub enum PermissionModel { - Open, // All extensions allowed - Strict, // Explicit permission required - Sandbox, // Limited core functionality only -} -``` - -## Validation Utilities - -### Input Validation - -Common validation functions: - -```rust -use pvq_primitives::validation::*; - -pub fn validate_account_id(account: &[u8]) -> PvqResult<()> { - if account.len() != 32 { - return Err(PvqError::invalid_input( - "account_id", - "32 bytes", - &format!("{} bytes", account.len()) - )); - } - Ok(()) -} - -pub fn validate_asset_id(asset_id: u32) -> PvqResult<()> { - if asset_id == 0 { - return Err(PvqError::invalid_input( - "asset_id", - "non-zero value", - "0" - )); - } - Ok(()) -} - -pub fn validate_program_size(program: &[u8]) -> PvqResult<()> { - if program.len() > MAX_INPUT_SIZE_BYTES { - return Err(PvqError::ResourceLimitExceeded { - resource: "program_size".to_string(), - limit: MAX_INPUT_SIZE_BYTES as u64, - requested: program.len() as u64, - }); - } - Ok(()) -} - -// Usage in extensions -#[extension_decl] -pub trait ValidatedExtension { - fn safe_function(account: [u8; 32], asset_id: u32) -> PvqResult { - validate_account_id(&account)?; - validate_asset_id(asset_id)?; - - // Safe to proceed - Ok("Validation passed".to_string()) - } -} -``` - -## Usage Examples - -### Basic Error Handling - -```rust -use pvq_primitives::{PvqResult, PvqError, PvqResponse}; - -#[pvq_program::program] -fn safe_query(input: u32) -> PvqResponse { - match process_input(input) { - Ok(result) => PvqResponse::success(result), - Err(error) => PvqResponse::error( - String::new(), - error.to_string() - ), - } -} - -fn process_input(input: u32) -> PvqResult { - if input == 0 { - return Err(PvqError::invalid_input( - "input", - "positive number", - "0" - )); - } - - Ok(format!("Processed: {}", input)) -} -``` - -### Complex Data Structures - -```rust -use pvq_primitives::{QueryData, QueryMetadata}; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize)] -pub struct AnalysisQuery { - pub assets: Vec, - pub accounts: Vec<[u8; 32]>, - pub include_metadata: bool, -} - -#[derive(Serialize, Deserialize)] -pub struct AnalysisResult { - pub total_value: u128, - pub asset_distribution: Vec<(u32, u128)>, - pub top_holders: Vec<([u8; 32], u128)>, -} - -#[pvq_program::program] -fn complex_analysis(query: QueryData) -> PvqResponse { - let start_time = current_timestamp(); - - // Process query - let result = perform_analysis(&query.payload)?; - - // Create response with metadata - let mut response = PvqResponse::success(result); - response.metadata.execution_time_ms = current_timestamp() - start_time; - - response -} -``` - -## Integration - -### Runtime Integration - -Configure primitives in your runtime: - -```rust -impl pvq_primitives::Config for Runtime { - type MaxQuerySize = ConstU32<1048576>; // 1MB - type MaxResponseSize = ConstU32<10485760>; // 10MB - type ErrorStringLimit = ConstU32<256>; -} -``` - -### Extension Integration - -Use primitives in extensions: - -```rust -use pvq_primitives::{PvqResult, PvqError, ExtensionMetadata}; - -#[extensions_impl] -impl MyExtension for MyExtensionImpl { - fn get_metadata() -> ExtensionMetadata { - ExtensionMetadata { - name: "MyExtension".to_string(), - version: "1.0.0".to_string(), - description: "Custom extension example".to_string(), - functions: vec![ - // Function metadata... - ], - } - } - - fn safe_operation(input: u32) -> PvqResult { - if input > 1000 { - return Err(PvqError::invalid_input( - "input", - "value <= 1000", - &input.to_string() - )); - } - - Ok(format!("Result: {}", input * 2)) - } -} -``` - -## Development - -### Building +### Build ```bash -# Build primitives cargo build -p pvq-primitives - -# Run tests -cargo test -p pvq-primitives - -# Check documentation -cargo doc -p pvq-primitives --open -``` - -### Testing - -```bash -# Unit tests -cargo test -p pvq-primitives - -# Integration tests -cargo test -p pvq-primitives --test integration - -# Test serialization performance -cargo bench -p pvq-primitives ``` - -## Performance Considerations - -### Memory Efficiency - -- Use compact serialization formats when possible -- Implement custom `Encode`/`Decode` for hot paths -- Avoid unnecessary allocations in error paths - -### Error Handling - -- Error construction should be lightweight -- Consider error code enums for frequently occurring errors -- Provide context without excessive string allocation - -```rust -// Efficient error handling -#[derive(Debug, Clone, Copy)] -pub enum ErrorCode { - InvalidInput = 1001, - PermissionDenied = 1002, - ResourceExhausted = 1003, - Timeout = 1004, -} - -impl From for PvqError { - fn from(code: ErrorCode) -> Self { - match code { - ErrorCode::InvalidInput => PvqError::InvalidInput { - field: "unknown".to_string(), - expected: "valid input".to_string(), - received: "invalid".to_string(), - }, - // ... other mappings - } - } -} -``` - -## Related Components - -- [PVQ Executor](../pvq-executor/) - Execution environment -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Program](../pvq-program/) - Program development -- [PVQ Runtime API](../pvq-runtime-api/) - Runtime integration - ---- - -*PVQ Primitives provide the essential building blocks for all PVQ system components.* \ No newline at end of file diff --git a/pvq-program/README.md b/pvq-program/README.md index 4264ea9..119f03f 100644 --- a/pvq-program/README.md +++ b/pvq-program/README.md @@ -1,675 +1,113 @@ # PVQ Program - +Procedural macro for declaring PVQ (PolkaVM Query) guest programs. This crate exposes one attribute, `#[pvq_program::program]`, which turns an inline Rust module into a PolkaVM guest with entrypoints and host extension calls. -Development tools and procedural macros for creating PVQ (PolkaVM Query) guest programs. This crate provides the `#[program]` macro and supporting utilities that simplify writing queries that run inside PolkaVM. +The README reflects the real implementation in this crate and the working examples in `guest-examples/`. -## Overview +## Usage overview -PVQ Program provides the essential tools for developing guest programs that execute within the PVQ system. The `#[program]` macro handles all the boilerplate code required to make your functions executable by the PVQ executor, while additional utilities help with development and debugging. +- Annotate an inline module with `#[pvq_program::program]`. +- Mark extension functions with `#[program::extension_fn(extension_id = , fn_index = )]`. +- Mark one or more entrypoints with `#[program::entrypoint]`. +- Use SCALE-encodable types for all arguments and return values. -## Features +## Simple example -- πŸš€ **`#[program]` Macro**: Automatically generates program entry points and serialization code -- πŸ“¦ **Zero Boilerplate**: Focus on query logic, not infrastructure -- πŸ”„ **Automatic Serialization**: Seamless input/output parameter handling -- ⚑ **Optimized Code Generation**: Efficient PolkaVM-compatible code -- πŸ› οΈ **Development Tools**: Utilities for testing and debugging programs -- πŸ“‹ **Metadata Generation**: Automatic program metadata for UI integration - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PVQ Program β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Program β”‚ β”‚ Code Gen β”‚ β”‚ Dev Tools β”‚ β”‚ -β”‚ β”‚ Macro β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”œβ”€ Entry Point β”‚ β”‚ β”œβ”€ Testing β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Parsing β”‚ β”‚ β”œβ”€ Serializationβ”‚ β”‚ β”œβ”€ Debugging β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Validation β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ β”œβ”€ Profiling β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Transform β”‚ β”‚ β”‚ Handling β”‚ β”‚ └─ Metadata β”‚ β”‚ -β”‚ β”‚ └─ Generation β”‚ β”‚ └─ Export β”‚ β”‚ Generation β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Functions β”‚ β”‚ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## Getting Started - -### Basic Program - -The simplest PVQ program using the `#[program]` macro: - -```rust -use pvq_program::program; - -#[program] -fn hello_world() -> String { - "Hello from PVQ!".to_string() -} -``` - -This generates all the necessary boilerplate to make your function executable as a PVQ program. - -### Program with Parameters - -Programs can accept parameters that are automatically deserialized: - -```rust -use pvq_program::program; - -#[program] -fn add_numbers(a: u32, b: u32) -> u32 { - a + b -} - -#[program] -fn process_account(account_id: [u8; 32], multiplier: u64) -> u64 { - // Your logic here - account_id[0] as u64 * multiplier -} -``` - -### Complex Data Types - -Programs support complex input and output types: - -```rust -use pvq_program::program; -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize)] -pub struct QueryInput { - pub asset_ids: Vec, - pub account: [u8; 32], - pub include_metadata: bool, -} - -#[derive(Serialize, Deserialize)] -pub struct QueryResult { - pub total_balance: u128, - pub asset_balances: Vec<(u32, u128)>, - pub metadata: Option, -} - -#[program] -fn complex_query(input: QueryInput) -> QueryResult { - // Implementation here - QueryResult { - total_balance: 1000, - asset_balances: vec![(1, 500), (2, 500)], - metadata: if input.include_metadata { - Some("Sample metadata".to_string()) - } else { - None - }, - } -} -``` - -## Program Development - -### Using Extensions - -Programs can call extension functions to interact with the runtime: - -```rust -use pvq_program::program; -use pvq_extension_core::ExtensionCore; -use pvq_extension_fungibles::ExtensionFungibles; - -#[program] -fn get_balance_with_info(asset_id: u32, account: [u8; 32]) -> String { - // Check if fungibles extension is available - if !ExtensionCore::extension_exists("fungibles") { - return "Fungibles extension not available".to_string(); - } - - // Get balance using extension - let balance = ExtensionFungibles::balance(asset_id, &account); - let block = ExtensionCore::current_block_number(); - - format!("Balance: {} at block {}", balance, block) -} -``` - -### Error Handling - -Programs can return `Result` types for error handling: +A minimal module with one extension function and one entrypoint: ```rust -use pvq_program::program; -use pvq_primitives::PvqResult; +#![no_std] +#![no_main] -#[program] -fn safe_divide(a: u64, b: u64) -> PvqResult { - if b == 0 { - Err(pvq_primitives::PvqError::invalid_input( - "divisor", - "non-zero value", - "0" - )) - } else { - Ok(a / b) - } -} -``` - -### Advanced Programs - -More complex programs with multiple operations: - -```rust -use pvq_program::program; -use pvq_extension_fungibles::ExtensionFungibles; -use pvq_extension_swap::ExtensionSwap; -use serde::{Serialize, Deserialize}; +#[pvq_program::program] +mod my_program { + type AssetId = u32; + type Balance = u64; -#[derive(Serialize, Deserialize)] -pub struct PortfolioAnalysis { - pub total_value: u128, - pub asset_count: u32, - pub largest_holding: (u32, u128), - pub trading_opportunities: Vec, -} + // Replace extension_id/fn_index with values that match your runtime + #[program::extension_fn(extension_id = 123u64, fn_index = 0)] + fn total_supply(asset: AssetId) -> Balance {} -#[program] -fn analyze_portfolio(account: [u8; 32]) -> PortfolioAnalysis { - let mut analysis = PortfolioAnalysis { - total_value: 0, - asset_count: 0, - largest_holding: (0, 0), - trading_opportunities: Vec::new(), - }; - - // Get all assets - let assets = ExtensionFungibles::list_assets(); - - for asset_id in assets { - let balance = ExtensionFungibles::balance(asset_id, &account); - - if balance > 0 { - analysis.total_value += balance; - analysis.asset_count += 1; - - // Track largest holding - if balance > analysis.largest_holding.1 { - analysis.largest_holding = (asset_id, balance); - } - - // Look for trading opportunities - if balance > 1000 { - let pools = ExtensionSwap::pools_for_asset(asset_id); - if !pools.is_empty() { - analysis.trading_opportunities.push( - format!("Consider trading asset {} (balance: {})", asset_id, balance) - ); - } - } - } + #[program::entrypoint] + fn query(asset: AssetId) -> Balance { + total_supply(asset) } - - analysis } ``` -## Program Attributes - -### Basic Attributes +## Authoring rules -The `#[program]` macro supports various attributes: +- Use on an inline module only: `#[pvq_program::program] mod my_program { ... }`. +- Mark extension functions with `#[program::extension_fn(extension_id = , fn_index = )]`. + - Both attributes are required. + - Functions must be free functions (no `self`/receiver). +- Mark one or more entrypoints with `#[program::entrypoint]`. + - At least one entrypoint is required. + - Entrypoints must be free functions (no receivers). + - Entrypoints’ arguments and return types must be SCALE `Encode`/`Decode`. -```rust -use pvq_program::program; +## Types and encoding -// Basic program -#[program] -fn basic_function() -> u32 { 42 } +All inputs/outputs are SCALE encoded. For custom structs/enums, always derive +`parity_scale_codec::{Encode, Decode}` (for host/guest serialization) and +`scale_info::TypeInfo` (for metadata generation). -// Program with custom name (affects metadata) -#[program(name = "custom_name")] -fn function_with_custom_name() -> String { - "Custom named function".to_string() -} - -// Program with description -#[program(description = "This function demonstrates documentation")] -fn documented_function() -> String { - "Documented function".to_string() -} -``` - -### Performance Attributes +Example: ```rust -use pvq_program::program; +#[derive(parity_scale_codec::Encode, parity_scale_codec::Decode, scale_info::TypeInfo)] +pub struct MyData { pub x: u32, pub y: u64 } -// Optimize for size -#[program(optimize = "size")] -fn size_optimized() -> u32 { 123 } - -// Optimize for speed -#[program(optimize = "speed")] -fn speed_optimized() -> u64 { 456 } - -// Set maximum execution time (milliseconds) -#[program(timeout = 5000)] -fn time_limited() -> String { - "Completes within 5 seconds".to_string() -} +#[program::entrypoint] +fn do_something(d: MyData) -> Option { Some(d.x as u64 + d.y) } ``` -### Memory Attributes +## Building and metadata -```rust -use pvq_program::program; - -// Set maximum memory usage (bytes) -#[program(max_memory = 1048576)] // 1MB -fn memory_limited() -> Vec { - vec![1, 2, 3, 4, 5] -} - -// Stack size configuration -#[program(stack_size = 65536)] // 64KB -fn custom_stack() -> String { - "Custom stack size".to_string() -} -``` - -## Development Tools - -### Testing Programs - -PVQ Program provides utilities for testing programs locally: - -```rust -#[cfg(test)] -mod tests { - use super::*; - use pvq_program::testing::ProgramTester; - - #[test] - fn test_my_program() { - let tester = ProgramTester::new(); - - // Test basic function - let result = tester.call_program(hello_world, ()); - assert_eq!(result, "Hello from PVQ!"); - - // Test function with parameters - let result = tester.call_program(add_numbers, (10, 20)); - assert_eq!(result, 30); - } - - #[test] - fn test_with_extensions() { - let mut tester = ProgramTester::new(); - tester.enable_extension("core"); - tester.enable_extension("fungibles"); - - // Mock extension responses - tester.mock_extension_call( - "fungibles", - "balance", - vec![1u32, [0u8; 32]], - 1000u128 - ); - - let result = tester.call_program(get_balance_with_info, (1u32, [0u8; 32])); - assert!(result.contains("Balance: 1000")); - } -} -``` - -### Debugging Support - -Enable debugging features during development: - -```rust -use pvq_program::{program, debug}; - -#[program] -fn debug_example(input: u32) -> u32 { - debug!("Input value: {}", input); - - let result = input * 2; - debug!("Calculated result: {}", result); - - result -} - -// Run with debugging enabled -// RUST_LOG=pvq_program=debug cargo run -``` - -### Profiling - -Profile program performance: - -```rust -use pvq_program::{program, profile}; - -#[program] -fn performance_example() -> u64 { - let _timer = profile!("main_calculation"); - - let mut result = 0u64; - for i in 0..1000000 { - result += i; - } - - result -} -``` - -## Metadata Generation - -### Automatic Metadata - -The macro automatically generates metadata for UI integration: - -```rust -use pvq_program::program; - -/// Calculate the sum of account balances for multiple assets -/// -/// # Parameters -/// - `accounts`: List of account IDs to check -/// - `asset_ids`: List of assets to include in calculation -/// -/// # Returns -/// Total balance across all accounts and assets -#[program( - name = "multi_asset_balance", - version = "1.0.0", - author = "Developer Name" -)] -fn calculate_multi_asset_balance( - accounts: Vec<[u8; 32]>, - asset_ids: Vec -) -> u128 { - // Implementation - 0 -} -``` - -This generates metadata that can be used by UIs to provide documentation and type information. - -### Custom Metadata - -Add custom metadata fields: - -```rust -use pvq_program::program; - -#[program( - name = "advanced_query", - description = "Performs advanced portfolio analysis", - version = "2.1.0", - author = "Portfolio Team", - tags = ["finance", "portfolio", "analysis"], - complexity = "high", - estimated_gas = 50000 -)] -fn advanced_portfolio_query() -> String { - "Advanced analysis result".to_string() -} -``` - -## Build Integration - -### Cargo Configuration - -Add to your `Cargo.toml`: +Guest programs are `no_std` PolkaVM binaries. Typical Cargo dependencies (see `guest-examples/*/Cargo.toml`): ```toml [dependencies] -pvq-program = { version = "0.1.0", default-features = false } -pvq-extension-core = { version = "0.1.0", default-features = false } -pvq-extension-fungibles = { version = "0.1.0", default-features = false } -pvq-primitives = { version = "0.1.0", default-features = false } -serde = { version = "1.0", default-features = false, features = ["derive"] } - -[features] -default = [] -std = [ - "pvq-program/std", - "pvq-primitives/std", - "serde/std", -] +polkavm-derive = { workspace = true } +pvq-program = { workspace = true } +parity-scale-codec = { workspace = true } +# For richer metadata types: +scale-info = { workspace = true, optional = true } ``` -### Build Script Integration - -For metadata generation, add `build.rs`: +To emit program metadata during build (as used in all examples), use a `build.rs` +that runs the generator and writes to a directory provided by the environment: ```rust -use std::env; -use std::path::PathBuf; -use std::process::Command; +use std::{env, path::PathBuf, process::Command}; fn main() { - // Generate program metadata - let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); - + println!("cargo:rerun-if-changed=src/main.rs"); + let current_dir = env::current_dir().expect("Failed to get current directory"); + let output_dir = PathBuf::from(env::var("METADATA_OUTPUT_DIR").expect("METADATA_OUTPUT_DIR is not set")); + let status = Command::new("pvq-program-metadata-gen") - .arg("--crate-path") - .arg(env::current_dir().unwrap()) - .arg("--output-dir") - .arg(&out_dir) + .arg("--crate-path").arg(¤t_dir) + .arg("--output-dir").arg(&output_dir) + .env("RUST_LOG", "info") .status() - .expect("Failed to run metadata generator"); - - if !status.success() { - panic!("Metadata generation failed"); - } - - println!("cargo:rerun-if-changed=src/"); -} -``` - -## Advanced Usage - -### Custom Serialization - -For performance-critical programs, implement custom serialization: - -```rust -use pvq_program::{program, custom_serde}; - -#[derive(Clone)] -pub struct CustomData { - pub values: Vec, - pub flags: u32, -} - -impl custom_serde::Serialize for CustomData { - fn serialize(&self) -> Vec { - let mut bytes = Vec::new(); - bytes.extend((self.values.len() as u32).to_le_bytes()); - for &val in &self.values { - bytes.extend(val.to_le_bytes()); - } - bytes.extend(self.flags.to_le_bytes()); - bytes - } -} - -impl custom_serde::Deserialize for CustomData { - fn deserialize(bytes: &[u8]) -> Result { - if bytes.len() < 8 { - return Err("Insufficient data".to_string()); - } - - let len = u32::from_le_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]) as usize; - let mut values = Vec::with_capacity(len); - - for i in 0..len { - let start = 4 + i * 8; - if start + 8 > bytes.len() { - return Err("Truncated data".to_string()); - } - let val = u64::from_le_bytes([ - bytes[start], bytes[start + 1], bytes[start + 2], bytes[start + 3], - bytes[start + 4], bytes[start + 5], bytes[start + 6], bytes[start + 7], - ]); - values.push(val); - } - - let flags_start = 4 + len * 8; - if flags_start + 4 > bytes.len() { - return Err("No flags data".to_string()); - } - - let flags = u32::from_le_bytes([ - bytes[flags_start], bytes[flags_start + 1], - bytes[flags_start + 2], bytes[flags_start + 3], - ]); - - Ok(CustomData { values, flags }) - } -} - -#[program] -fn process_custom_data(data: CustomData) -> u64 { - data.values.iter().sum::() + data.flags as u64 + .expect("Failed to execute pvq-program-metadata-gen"); + if !status.success() { panic!("Failed to generate program metadata"); } } ``` -### Multi-Function Programs +## Not supported (by design) -Create programs with multiple entry points: - -```rust -use pvq_program::{program, program_module}; - -#[program_module] -mod my_programs { - use super::*; - - #[program] - pub fn query_balance(account: [u8; 32]) -> u128 { - // Implementation - 0 - } - - #[program] - pub fn query_metadata(asset_id: u32) -> String { - // Implementation - "Metadata".to_string() - } - - #[program] - pub fn complex_analysis(params: AnalysisParams) -> AnalysisResult { - // Implementation - AnalysisResult::default() - } -} -``` - -## Performance Optimization - -### Size Optimization - -For size-constrained environments: - -```rust -use pvq_program::program; - -// Minimize code size -#[program(optimize = "size", inline_threshold = 0)] -fn minimal_program() -> u32 { - 42 -} - -// Avoid large dependencies -#[program] -fn efficient_program(input: u32) -> u32 { - // Use simple operations instead of complex libraries - input.wrapping_mul(2).wrapping_add(1) -} -``` - -### Speed Optimization - -For performance-critical programs: - -```rust -use pvq_program::program; - -// Optimize for speed -#[program(optimize = "speed", inline_aggressive = true)] -fn fast_program(data: Vec) -> u64 { - data.into_iter() - .map(|x| x as u64) - .sum() -} - -// Use efficient algorithms -#[program] -fn optimized_search(haystack: Vec, needle: u32) -> Option { - // Binary search for sorted data - haystack.binary_search(&needle).ok() -} -``` - -## Troubleshooting - -### Common Issues - -**Program won't compile:** -```bash -# Check for syntax errors in macro usage -cargo check - -# Enable detailed error reporting -RUST_BACKTRACE=1 cargo build -``` - -**Serialization errors:** -```rust -// Ensure all types implement required traits -#[derive(serde::Serialize, serde::Deserialize)] -pub struct MyType { - pub field: u32, -} -``` - -**Runtime errors:** -```rust -// Add error handling -#[program] -fn safe_program(input: u32) -> Result { - if input == 0 { - Err("Zero input not allowed".to_string()) - } else { - Ok(input * 2) - } -} -``` - -### Debug Mode - -Enable debug output: - -```bash -# Build with debug info -RUST_LOG=debug cargo build - -# Run test with debug output -RUST_LOG=pvq_program=debug cargo test -``` +- Per-function macro options such as optimization levels, timeouts, custom stack/memory, or built-in testing/debug/profiling helpers are not part of this crate. +- Methods with `self`/`&self` receivers cannot be entrypoints or extension functions. -## Related Components +## See also -- [PVQ Executor](../pvq-executor/) - Program execution environment -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Program Metadata Gen](../pvq-program-metadata-gen/) - Metadata generation -- [PVQ Test Runner](../pvq-test-runner/) - Testing utilities +- `guest-examples/sum-balance` +- `guest-examples/sum-balance-percent` +- `guest-examples/total-supply` +- `guest-examples/swap-info` --- -*PVQ Program provides the tools to create efficient, type-safe guest programs for the PVQ system.* \ No newline at end of file +See `pvq-program/src/lib.rs` for the re-export and `pvq-program/procedural` for internals. diff --git a/pvq-runtime-api/README.md b/pvq-runtime-api/README.md index fa3b21c..16040f9 100644 --- a/pvq-runtime-api/README.md +++ b/pvq-runtime-api/README.md @@ -1,648 +1,86 @@ # PVQ Runtime API - +Substrate runtime API definition for PVQ (PolkaVM Query). This crate exposes a single runtime API trait that runtimes implement to execute PVQ programs and expose PVQ-related metadata. -The Substrate runtime API definition for PVQ (PolkaVM Query) that provides the interface between external clients and the PVQ execution system. This crate defines the `PvqApi` trait that runtime implementations must provide to support PVQ functionality. +## API definition -## Overview - -The PVQ Runtime API serves as the bridge between external clients (like UIs, CLIs, or other applications) and the PVQ execution system within Substrate runtimes. It provides a standardized interface for submitting queries, retrieving results, and accessing system metadata. - -## Features - -- 🌐 **Standard Interface**: Consistent API across different runtime implementations -- πŸ” **Query Execution**: Submit and execute PVQ programs with arguments -- πŸ“Š **Metadata Access**: Retrieve information about available extensions and capabilities -- ⚑ **Async Support**: Compatible with Substrate's async runtime API system -- πŸ›‘οΈ **Type Safety**: Strongly-typed interface with compile-time guarantees -- πŸ“ˆ **Performance Metrics**: Built-in execution statistics and monitoring - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PVQ Runtime API β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ PvqApi β”‚ β”‚ Client Side β”‚ β”‚ Runtime β”‚ β”‚ -β”‚ β”‚ Trait β”‚ β”‚ β”‚ β”‚ Side β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ β”œβ”€ Query β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ execute β”‚ β”‚ β”‚ Submission β”‚ β”‚ β”œβ”€ Executor β”‚ β”‚ -β”‚ β”‚ β”‚ _query β”‚ β”‚ β”œβ”€ Result β”‚ β”‚ β”‚ Instance β”‚ β”‚ -β”‚ β”‚ β”œβ”€ metadata β”‚ β”‚ β”‚ Processing β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ -β”‚ β”‚ β”œβ”€ capabilities β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ β”‚ Registry β”‚ β”‚ -β”‚ β”‚ └─ version β”‚ β”‚ β”‚ Handling β”‚ β”‚ └─ Context β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ └─ Async β”‚ β”‚ Management β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Support β”‚ β”‚ β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## API Definition - -### Core Trait - -The main runtime API trait that must be implemented: +The trait exactly as defined in this crate: ```rust +use alloc::vec::Vec; +use pvq_primitives::PvqResult; use sp_api::decl_runtime_apis; -use pvq_primitives::{PvqResult, QueryMetadata}; decl_runtime_apis! { - /// PVQ Runtime API for executing queries and managing the system + /// The runtime API for the PVQ module. pub trait PvqApi { - /// Execute a PVQ program with given arguments - /// - /// # Parameters - /// - `program`: Compiled PVQ program blob (PolkaVM bytecode) - /// - `args`: Serialized arguments for the program - /// - /// # Returns - /// Serialized result of program execution - fn execute_query(program: Vec, args: Vec) -> Vec; - - /// Get metadata about the PVQ system and available extensions - /// - /// # Returns - /// Serialized system metadata including extension information + /// Execute a PVQ program with arguments. + /// + /// - `program`: PolkaVM bytecode of the guest program + /// - `args`: SCALE-encoded call data + /// - `gas_limit`: Optional execution gas limit; `None` means use the default time boundary + fn execute_query(program: Vec, args: Vec, gas_limit: Option) -> PvqResult; + + /// Return PVQ extensions metadata as an opaque byte blob. fn metadata() -> Vec; - - /// Get system capabilities and configuration - /// - /// # Returns - /// Serialized capability information - fn capabilities() -> Vec; - - /// Get the PVQ system version information - /// - /// # Returns - /// Version string - fn version() -> String; } } ``` -## Runtime Implementation +Notes -### Basic Implementation +- `PvqResult` is defined in `pvq-primitives` as `Result` where `PvqResponse = Vec`. +- The `args` buffer must match the PVQ guest ABI (see `pvq-program`): first byte is the entrypoint index, followed by SCALE-encoded arguments. -Implementing the API in your runtime: +## Minimal runtime implementation ```rust use sp_api::impl_runtime_apis; -use pvq_executor::{PvqExecutor, PvqExecutorContext}; -use pvq_extension::ExtensionsExecutor; -use pvq_extension_core::ExtensionCore; -use pvq_extension_fungibles::ExtensionFungibles; -use pvq_extension_swap::ExtensionSwap; +use pvq_runtime_api::PvqApi; +use pvq_primitives::PvqResult; impl_runtime_apis! { - impl pvq_runtime_api::PvqApi for Runtime { - fn execute_query(program: Vec, args: Vec) -> Vec { - // Create extension executor with available extensions - let mut extensions = ExtensionsExecutor::new(); - extensions.register::(); - extensions.register::(); - extensions.register::(); - - // Create execution context - let context = PvqExecutorContext::new(extensions); - - // Create and configure executor - let executor = PvqExecutor::new(context); - - // Execute program - match executor.execute(&program, &args) { - Ok(result) => result, - Err(error) => { - // Return error as serialized response - format!("Error: {}", error).into_bytes() - } - } - } - - fn metadata() -> Vec { - let metadata = SystemMetadata { - version: "1.0.0".to_string(), - extensions: vec![ - "core".to_string(), - "fungibles".to_string(), - "swap".to_string(), - ], - max_execution_time_ms: 30000, - max_memory_bytes: 64 * 1024 * 1024, - }; - - serde_json::to_vec(&metadata).unwrap_or_default() - } - - fn capabilities() -> Vec { - let capabilities = SystemCapabilities { - supported_program_versions: vec!["1.0".to_string()], - max_program_size_bytes: 1024 * 1024, - max_args_size_bytes: 1024 * 1024, - max_result_size_bytes: 10 * 1024 * 1024, - execution_timeout_ms: 30000, - }; - - serde_json::to_vec(&capabilities).unwrap_or_default() + impl PvqApi for Runtime { + fn execute_query(program: Vec, args: Vec, gas_limit: Option) -> PvqResult { + // Integrate with your PVQ executor here, e.g. pvq-executor + // Ok(result_bytes) or Err(pvq_error) + unimplemented!() } - - fn version() -> String { - "PVQ Runtime API v1.0.0".to_string() - } - } -} -``` - -### Advanced Implementation - -More sophisticated implementation with error handling and monitoring: -```rust -use pvq_primitives::{PvqResponse, QueryMetadata, PvqError}; -use std::time::Instant; - -impl_runtime_apis! { - impl pvq_runtime_api::PvqApi for Runtime { - fn execute_query(program: Vec, args: Vec) -> Vec { - let start_time = Instant::now(); - let mut query_metadata = QueryMetadata::new(); - - // Validate input sizes - if program.len() > MAX_PROGRAM_SIZE { - let error = PvqError::ResourceLimitExceeded { - resource: "program_size".to_string(), - limit: MAX_PROGRAM_SIZE as u64, - requested: program.len() as u64, - }; - return Self::create_error_response(error, query_metadata); - } - - // Set up extensions with runtime context - let mut extensions = ExtensionsExecutor::new(); - Self::configure_extensions(&mut extensions); - - // Create execution context with monitoring - let context = PvqExecutorContext::builder() - .with_extensions(extensions) - .with_block_number(Self::current_block_number()) - .with_monitoring(true) - .build(); - - // Execute with timeout and resource limits - let executor = PvqExecutor::builder() - .with_context(context) - .with_timeout(Duration::from_secs(30)) - .with_memory_limit(64 * 1024 * 1024) - .build(); - - match executor.execute(&program, &args) { - Ok(result) => { - query_metadata.execution_time_ms = start_time.elapsed().as_millis() as u64; - query_metadata.memory_used_bytes = executor.memory_usage(); - query_metadata.extension_calls = executor.extension_call_count(); - query_metadata.block_number = Self::current_block_number(); - - let response = PvqResponse { - data: result, - metadata: query_metadata, - success: true, - }; - - serde_json::to_vec(&response).unwrap_or_default() - } - Err(error) => { - query_metadata.execution_time_ms = start_time.elapsed().as_millis() as u64; - query_metadata.error_message = Some(error.to_string()); - Self::create_error_response(error, query_metadata) - } - } - } - fn metadata() -> Vec { - let extensions_metadata = Self::get_extensions_metadata(); - - let metadata = SystemMetadata { - version: env!("CARGO_PKG_VERSION").to_string(), - runtime_version: Self::runtime_version().to_string(), - extensions: extensions_metadata, - limits: SystemLimits { - max_execution_time_ms: MAX_EXECUTION_TIME_MS, - max_memory_bytes: MAX_MEMORY_BYTES, - max_program_size: MAX_PROGRAM_SIZE, - max_args_size: MAX_ARGS_SIZE, - }, - features: SystemFeatures { - async_execution: true, - permission_control: true, - performance_monitoring: true, - }, - }; - - serde_json::to_vec(&metadata).unwrap_or_default() + // Return extension metadata bytes (format decided by the runtime) + Vec::new() } - - fn capabilities() -> Vec { - let capabilities = SystemCapabilities::builder() - .with_program_versions(vec!["1.0".to_string()]) - .with_serialization_formats(vec!["json".to_string(), "scale".to_string()]) - .with_extension_discovery(true) - .with_batch_execution(false) // Future feature - .build(); - - serde_json::to_vec(&capabilities).unwrap_or_default() - } - - fn version() -> String { - format!("PVQ Runtime API v{}", env!("CARGO_PKG_VERSION")) - } - } -} - -impl Runtime { - fn configure_extensions(executor: &mut ExtensionsExecutor) { - // Register core extensions - executor.register::(); - - // Register fungibles extension if assets pallet is available - if Self::has_assets_pallet() { - executor.register::(); - } - - // Register swap extension if DEX pallet is available - if Self::has_dex_pallet() { - executor.register::(); - } - - // Register custom runtime-specific extensions - Self::register_custom_extensions(executor); - } - - fn create_error_response(error: PvqError, metadata: QueryMetadata) -> Vec { - let response = PvqResponse { - data: Vec::new(), - metadata, - success: false, - }; - - serde_json::to_vec(&response).unwrap_or_else(|_| { - format!(r#"{{"error": "{}"}}"#, error).into_bytes() - }) } } ``` -## Client Usage +## Metadata -### Direct API Calls +`metadata()` returns a byte vector. The encoding and schema are defined by the runtime. Recommended format is either SCALE or JSON of the structure defined in `pvq-extension` (`pvq-extension/src/metadata.rs`). -Using the API from client code: +Shape -```rust -use sp_api::ProvideRuntimeApi; -use pvq_runtime_api::PvqApi; - -// In your client code -async fn execute_pvq_program( - client: &Client, - program: Vec, - args: Vec, -) -> Result, Box> -where - Client: ProvideRuntimeApi, - Client::Api: PvqApi, -{ - let at = client.info().best_hash; - - let result = client - .runtime_api() - .execute_query(at, program, args)?; - - Ok(result) -} - -async fn get_system_metadata( - client: &Client, -) -> Result> -where - Client: ProvideRuntimeApi, - Client::Api: PvqApi, -{ - let at = client.info().best_hash; - - let metadata_bytes = client - .runtime_api() - .metadata(at)?; - - let metadata: SystemMetadata = serde_json::from_slice(&metadata_bytes)?; - Ok(metadata) -} -``` - -### RPC Integration - -Wrapping the API for RPC access: - -```rust -use jsonrpsee::{core::RpcResult, proc_macros::rpc}; -use sp_api::ProvideRuntimeApi; -use pvq_runtime_api::PvqApi; - -#[rpc(client, server)] -pub trait PvqRpcApi { - /// Execute a PVQ program - #[method(name = "pvq_executeQuery")] - async fn execute_query( - &self, - program: String, // Hex-encoded program - args: String, // Hex-encoded arguments - ) -> RpcResult; // Hex-encoded result - - /// Get system metadata - #[method(name = "pvq_metadata")] - async fn metadata(&self) -> RpcResult; - - /// Get system capabilities - #[method(name = "pvq_capabilities")] - async fn capabilities(&self) -> RpcResult; - - /// Get version information - #[method(name = "pvq_version")] - async fn version(&self) -> RpcResult; -} - -pub struct PvqRpc { - client: Arc, -} - -impl PvqRpc { - pub fn new(client: Arc) -> Self { - Self { client } - } -} - -impl PvqRpcApiServer for PvqRpc -where - Client: ProvideRuntimeApi + Send + Sync + 'static, - Client::Api: PvqApi, +```json { - async fn execute_query( - &self, - program_hex: String, - args_hex: String, - ) -> RpcResult { - let program = hex::decode(program_hex) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Invalid program hex: {}", e)))?; - - let args = hex::decode(args_hex) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Invalid args hex: {}", e)))?; - - let at = self.client.info().best_hash; - - let result = self.client - .runtime_api() - .execute_query(at, program, args) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Execution failed: {}", e)))?; - - Ok(hex::encode(result)) - } - - async fn metadata(&self) -> RpcResult { - let at = self.client.info().best_hash; - - let metadata_bytes = self.client - .runtime_api() - .metadata(at) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get metadata: {}", e)))?; - - let metadata: serde_json::Value = serde_json::from_slice(&metadata_bytes) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to parse metadata: {}", e)))?; - - Ok(metadata) - } - - async fn capabilities(&self) -> RpcResult { - let at = self.client.info().best_hash; - - let capabilities_bytes = self.client - .runtime_api() - .capabilities(at) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get capabilities: {}", e)))?; - - let capabilities: serde_json::Value = serde_json::from_slice(&capabilities_bytes) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to parse capabilities: {}", e)))?; - - Ok(capabilities) - } - - async fn version(&self) -> RpcResult { - let at = self.client.info().best_hash; - - let version = self.client - .runtime_api() - .version(at) - .map_err(|e| jsonrpsee::core::Error::Custom(format!("Failed to get version: {}", e)))?; - - Ok(version) - } -} -``` - -## Data Types - -### System Metadata - -Information about the PVQ system: - -```rust -use serde::{Serialize, Deserialize}; - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct SystemMetadata { - pub version: String, - pub runtime_version: String, - pub extensions: Vec, - pub limits: SystemLimits, - pub features: SystemFeatures, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct ExtensionInfo { - pub name: String, - pub version: String, - pub description: String, - pub functions: Vec, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct SystemLimits { - pub max_execution_time_ms: u64, - pub max_memory_bytes: u64, - pub max_program_size: usize, - pub max_args_size: usize, -} - -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct SystemFeatures { - pub async_execution: bool, - pub permission_control: bool, - pub performance_monitoring: bool, -} -``` - -### System Capabilities - -Runtime capability information: - -```rust -#[derive(Serialize, Deserialize, Debug, Clone)] -pub struct SystemCapabilities { - pub supported_program_versions: Vec, - pub serialization_formats: Vec, - pub max_program_size_bytes: usize, - pub max_args_size_bytes: usize, - pub max_result_size_bytes: usize, - pub execution_timeout_ms: u64, - pub extension_discovery: bool, - pub batch_execution: bool, -} -``` - -## Configuration - -### Runtime Configuration - -Configure API behavior in your runtime: - -```rust -parameter_types! { - pub const MaxProgramSize: u32 = 1024 * 1024; // 1MB - pub const MaxArgsSize: u32 = 1024 * 1024; // 1MB - pub const MaxResultSize: u32 = 10 * 1024 * 1024; // 10MB - pub const ExecutionTimeout: u64 = 30000; // 30 seconds -} - -impl pvq_runtime_api::Config for Runtime { - type MaxProgramSize = MaxProgramSize; - type MaxArgsSize = MaxArgsSize; - type MaxResultSize = MaxResultSize; - type ExecutionTimeout = ExecutionTimeout; -} -``` - -### Feature Gates - -Control API features: - -```rust -#[cfg(feature = "pvq-api")] -impl_runtime_apis! { - impl pvq_runtime_api::PvqApi for Runtime { - // Implementation here - } -} -``` - -## Development - -### Building - -```bash -# Build runtime API -cargo build -p pvq-runtime-api - -# Build with all features -cargo build -p pvq-runtime-api --all-features - -# Generate documentation -cargo doc -p pvq-runtime-api --open -``` - -### Testing - -```bash -# Unit tests -cargo test -p pvq-runtime-api - -# Integration tests with runtime -cargo test -p pvq-runtime-api --test runtime_integration - -# Test RPC functionality -cargo test -p pvq-runtime-api --test rpc_tests -``` - -## Performance Considerations - -### Optimization Tips - -1. **Cache Extension Registry**: Avoid recreating extensions on each call -2. **Limit Result Size**: Implement size limits to prevent resource exhaustion -3. **Async Execution**: Use async where possible for better concurrency -4. **Memory Management**: Monitor and limit memory usage during execution - -### Example Optimizations - -```rust -impl Runtime { - // Cache extension registry - thread_local! { - static EXTENSION_CACHE: RefCell> = RefCell::new(None); - } - - fn get_cached_extensions() -> ExtensionsExecutor { - Self::EXTENSION_CACHE.with(|cache| { - let mut cache = cache.borrow_mut(); - if cache.is_none() { - let mut extensions = ExtensionsExecutor::new(); - Self::configure_extensions(&mut extensions); - *cache = Some(extensions); - } - cache.as_ref().unwrap().clone() - }) - } -} -``` - -## Security Considerations - -- Validate all input data before processing -- Implement proper resource limits and timeouts -- Use sandboxed execution environment -- Log and monitor API usage -- Rate limit API calls if exposed publicly - -## Error Handling - -Handle API errors gracefully: - -```rust -impl Runtime { - fn handle_api_error(error: PvqError) -> Vec { - let response = match error { - PvqError::Timeout { duration_ms } => { - ApiResponse::timeout(duration_ms) - } - PvqError::ResourceLimitExceeded { resource, .. } => { - ApiResponse::resource_exhausted(resource) - } - _ => { - ApiResponse::general_error(error.to_string()) - } - }; - - serde_json::to_vec(&response).unwrap_or_default() + "types": { /* scale-info PortableRegistry */ }, + "extensions": { + "": { + "name": "", + "functions": [ + { + "name": "", + "inputs": [ { "name": "", "ty": } ], + "output": + } + ] } + } } ``` -## Related Components - -- [PVQ Executor](../pvq-executor/) - Execution environment -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Primitives](../pvq-primitives/) - Core types -- [PoC Runtime](../poc/runtime/) - Example implementation - ---- +Notes -*The PVQ Runtime API provides the standard interface for integrating PVQ functionality into Substrate runtimes.* \ No newline at end of file +- `types` is a `scale-info` PortableRegistry describing all referenced types. +- `ty` and `output` are numeric type IDs that reference entries inside `types`. +- The `extensions` object is keyed by stringified extension IDs (u64 -> string) and maps to per-extension metadata: name and function signatures. diff --git a/pvq-test-runner/README.md b/pvq-test-runner/README.md index 27eb989..4c39b7f 100644 --- a/pvq-test-runner/README.md +++ b/pvq-test-runner/README.md @@ -1,675 +1,24 @@ # PVQ Test Runner - +CLI to execute PVQ guest programs against the extensions and executor used in this repository. -A comprehensive testing and development tool for PVQ (PolkaVM Query) programs. The test runner provides a simulated environment for executing guest programs, validating results, and debugging query logic without requiring a full runtime deployment. +### Use cases -## Overview +- Prepare input bytes for bundled example programs +- Execute a program and print the raw result +- Inspect generated extension metadata -The PVQ Test Runner serves as an essential development tool that enables developers to test their PVQ programs locally, validate extension interactions, and debug program logic in a controlled environment. It simulates the complete PVQ execution environment including extensions, permissions, and resource limits. - -## Features - -- πŸ§ͺ **Local Testing**: Execute PVQ programs without a full runtime environment -- 🎯 **Extension Simulation**: Mock extension calls and responses for testing -- πŸ” **Detailed Logging**: Comprehensive execution tracing and debugging information -- ⚑ **Performance Profiling**: Measure execution time, memory usage, and resource consumption -- πŸ›‘οΈ **Error Simulation**: Test error handling and edge cases -- πŸ“Š **Result Validation**: Compare actual vs. expected results with detailed diffs -- πŸ”§ **Configuration Options**: Flexible testing scenarios and environment setup - -## Architecture - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ PVQ Test Runner β”‚ -β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ -β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ -β”‚ β”‚ Test Runner β”‚ β”‚ Simulation β”‚ β”‚ Reporting β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Environment β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Program β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ -β”‚ β”‚ β”‚ Loading β”‚ β”‚ β”œβ”€ Extension β”‚ β”‚ β”œβ”€ Results β”‚ β”‚ -β”‚ β”‚ β”œβ”€ Execution β”‚ β”‚ β”‚ Mocking β”‚ β”‚ β”‚ Analysis β”‚ β”‚ -β”‚ β”‚ β”‚ Management β”‚ β”‚ β”œβ”€ State β”‚ β”‚ β”œβ”€ Performanceβ”‚ β”‚ -β”‚ β”‚ β”œβ”€ Result β”‚ β”‚ β”‚ Management β”‚ β”‚ β”‚ Metrics β”‚ β”‚ -β”‚ β”‚ β”‚ Validation β”‚ β”‚ β”œβ”€ Permission β”‚ β”‚ β”œβ”€ Error β”‚ β”‚ -β”‚ β”‚ └─ Configurationβ”‚ β”‚ β”‚ Control β”‚ β”‚ β”‚ Reports β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ └─ Resource β”‚ β”‚ └─ Debug β”‚ β”‚ -β”‚ β”‚ β”‚ β”‚ Limits β”‚ β”‚ Output β”‚ β”‚ -β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -## Installation - -### From Source - -```bash -# Clone the PVQ repository -git clone --recursive https://github.com/open-web3-stack/PVQ.git -cd PVQ - -# Build the test runner -cargo build -p pvq-test-runner --release - -# The binary will be available at target/release/pvq-test-runner -``` - -### Global Installation - -```bash -# Install globally via cargo -cargo install --path pvq-test-runner - -# Or install from crates.io (when published) -cargo install pvq-test-runner -``` - -## Usage - -### Basic Usage - -```bash -# Run a compiled PVQ program -pvq-test-runner --program output/my-program.polkavm - -# Run with custom arguments -pvq-test-runner --program output/my-program.polkavm --args "arg1,arg2,arg3" - -# Run with hex-encoded arguments -pvq-test-runner --program output/my-program.polkavm --args-hex "0x010203040506" -``` - -### Command Line Options - -```bash -pvq-test-runner [OPTIONS] --program - -Options: - -p, --program Path to the compiled PVQ program - -a, --args Comma-separated program arguments - --args-hex Hex-encoded program arguments - -c, --config Path to configuration file - -o, --output Output format (json, yaml, pretty) - -v, --verbose Enable verbose logging - --debug Enable debug mode with detailed tracing - --profile Enable performance profiling - --timeout Execution timeout in seconds [default: 30] - --memory-limit Memory limit in MB [default: 64] - --extensions Comma-separated list of enabled extensions - --mock-file Path to extension mock configuration - --validate Path to expected result file for validation - --no-color Disable colored output - -h, --help Print help information - -V, --version Print version information -``` - -## Configuration - -### Configuration File - -Create a `test-config.toml` for complex testing scenarios: - -```toml -[program] -path = "output/my-program.polkavm" -timeout_seconds = 30 -memory_limit_mb = 64 - -[execution] -enable_debugging = true -enable_profiling = true -trace_extension_calls = true -max_call_depth = 256 - -[extensions] -enabled = ["core", "fungibles", "swap"] - -[extensions.fungibles] -mock_balance_responses = true -default_balance = 1000 - -[extensions.swap] -mock_pool_data = true -default_pool_reserves = [10000, 20000] - -[logging] -level = "debug" -format = "pretty" -include_timestamps = true - -[validation] -expected_result_file = "tests/expected_results.json" -tolerance = 0.001 # For floating point comparisons -``` - -### Extension Mocking - -Create mock configurations for testing: - -```json -{ - "extension_mocks": { - "fungibles": { - "balance": { - "default_response": 1000, - "specific_responses": { - "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]": 5000, - "[2, [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]]": 2500 - } - }, - "total_supply": { - "default_response": 1000000, - "specific_responses": { - "[1]": 500000, - "[2]": 750000 - } - } - }, - "swap": { - "pool_reserves": { - "default_response": [10000, 20000], - "specific_responses": { - "[{\"asset_0\": 1, \"asset_1\": 2}]": [15000, 25000] - } - } - } - } -} -``` - -## Testing Scenarios - -### Basic Program Testing - -```bash -# Test a simple balance query program -pvq-test-runner --program output/balance-query.polkavm \ - --args "1,0x0101010101010101010101010101010101010101010101010101010101010101" \ - --extensions "core,fungibles" \ - --verbose -``` - -### Complex Integration Testing - -```bash -# Test with comprehensive configuration -pvq-test-runner \ - --config test-scenarios/portfolio-analysis.toml \ - --mock-file test-data/market-data-mocks.json \ - --validate test-data/expected-portfolio-result.json \ - --profile \ - --output json -``` - -### Performance Testing - -```bash -# Performance benchmark -pvq-test-runner --program output/performance-test.polkavm \ - --profile \ - --timeout 60 \ - --memory-limit 128 \ - --debug -``` - -## Development Integration - -### Automated Testing - -Integrate with your development workflow: - -```bash -#!/bin/bash -# test-all-programs.sh - -set -e - -echo "Building all guest programs..." -make guests - -echo "Running tests..." - -for program in output/*.polkavm; do - echo "Testing $(basename $program)..." - pvq-test-runner --program "$program" \ - --config "tests/$(basename $program .polkavm).toml" \ - --validate "tests/expected/$(basename $program .polkavm).json" -done - -echo "All tests passed!" -``` - -### CI/CD Integration - -GitHub Actions example: - -```yaml -name: PVQ Program Tests - -on: [push, pull_request] - -jobs: - test-programs: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - with: - submodules: recursive - - - name: Install Rust - uses: actions-rs/toolchain@v1 - with: - toolchain: stable - - - name: Install tools - run: make tools - - - name: Build guest programs - run: make guests - - - name: Run PVQ tests - run: | - for program in output/*.polkavm; do - cargo run -p pvq-test-runner -- \ - --program "$program" \ - --config "tests/$(basename $program .polkavm).toml" - done -``` - -## API Reference - -### Programmatic Usage - -Use the test runner as a library: - -```rust -use pvq_test_runner::{TestRunner, TestConfig, ExtensionMock}; -use std::collections::HashMap; - -#[tokio::main] -async fn main() -> Result<(), Box> { - // Create test configuration - let config = TestConfig::builder() - .with_timeout(Duration::from_secs(30)) - .with_memory_limit(64 * 1024 * 1024) - .with_extensions(vec!["core", "fungibles"]) - .enable_profiling(true) - .enable_debugging(true) - .build(); - - // Set up extension mocks - let mut mocks = HashMap::new(); - mocks.insert( - "fungibles.balance".to_string(), - ExtensionMock::simple_response(serde_json::json!(1000)) - ); - - // Create and configure test runner - let mut runner = TestRunner::new(config); - runner.set_extension_mocks(mocks); - - // Load and execute program - let program = std::fs::read("output/my-program.polkavm")?; - let args = serde_json::to_vec(&(1u32, [0u8; 32]))?; - - let result = runner.execute_program(&program, &args).await?; - - // Analyze results - println!("Execution successful: {}", result.success); - println!("Result: {:?}", result.output); - println!("Execution time: {}ms", result.execution_time_ms); - println!("Memory used: {} bytes", result.memory_used_bytes); - - if let Some(profile) = result.profile { - println!("Performance profile:"); - for (operation, duration) in profile.operation_times { - println!(" {}: {}ms", operation, duration.as_millis()); - } - } - - Ok(()) -} -``` - -### Test Utilities - -Helper functions for common testing patterns: - -```rust -use pvq_test_runner::testing::*; - -#[tokio::test] -async fn test_balance_query() { - let mut tester = create_default_tester().await; - - // Mock balance response - tester.mock_extension_call( - "fungibles", - "balance", - vec![json!(1), json!([0u8; 32])], - json!(5000) - ); - - // Execute test - let result = tester.execute_program_file( - "output/balance-query.polkavm", - &(1u32, [0u8; 32]) - ).await.unwrap(); - - // Validate result - assert!(result.success); - let balance: u128 = serde_json::from_slice(&result.output).unwrap(); - assert_eq!(balance, 5000); -} - -#[tokio::test] -async fn test_error_handling() { - let mut tester = create_default_tester().await; - - // Mock extension error - tester.mock_extension_error( - "fungibles", - "balance", - "Asset not found" - ); - - let result = tester.execute_program_file( - "output/balance-query.polkavm", - &(999u32, [0u8; 32]) - ).await.unwrap(); - - // Should handle error gracefully - assert!(result.success); // Program should handle error - let response: String = serde_json::from_slice(&result.output).unwrap(); - assert!(response.contains("Asset not found")); -} -``` - -## Output Formats - -### Pretty Output (Default) - -``` -PVQ Test Runner v0.1.0 -====================== - -Program: output/balance-query.polkavm -Arguments: [1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]] - -Execution Results: -β”œβ”€ Status: βœ“ Success -β”œβ”€ Output: 1000 -β”œβ”€ Execution Time: 15ms -β”œβ”€ Memory Used: 2.1MB -β”œβ”€ Extension Calls: 3 -β”‚ β”œβ”€ core.current_block_number: 1 call -β”‚ └─ fungibles.balance: 1 call -└─ Block Number: 12345 - -Performance Profile: -β”œβ”€ Program Loading: 2ms -β”œβ”€ Extension Setup: 1ms -β”œβ”€ Program Execution: 10ms -└─ Result Serialization: 2ms -``` - -### JSON Output - -```json -{ - "success": true, - "program": "output/balance-query.polkavm", - "arguments": "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]", - "output": "1000", - "execution_time_ms": 15, - "memory_used_bytes": 2097152, - "extension_calls": { - "total": 2, - "by_extension": { - "core": 1, - "fungibles": 1 - } - }, - "block_number": 12345, - "profile": { - "program_loading_ms": 2, - "extension_setup_ms": 1, - "program_execution_ms": 10, - "result_serialization_ms": 2 - }, - "error": null -} -``` - -### YAML Output - -```yaml -success: true -program: output/balance-query.polkavm -arguments: "[1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]" -output: "1000" -execution_time_ms: 15 -memory_used_bytes: 2097152 -extension_calls: - total: 2 - by_extension: - core: 1 - fungibles: 1 -block_number: 12345 -profile: - program_loading_ms: 2 - extension_setup_ms: 1 - program_execution_ms: 10 - result_serialization_ms: 2 -error: null -``` - -## Advanced Features - -### Custom Validators - -Create custom result validation: - -```rust -use pvq_test_runner::{TestRunner, ValidationRule, ValidationResult}; - -struct BalanceRangeValidator { - min: u128, - max: u128, -} - -impl ValidationRule for BalanceRangeValidator { - fn validate(&self, output: &[u8]) -> ValidationResult { - match serde_json::from_slice::(output) { - Ok(balance) => { - if balance >= self.min && balance <= self.max { - ValidationResult::success() - } else { - ValidationResult::failure(format!( - "Balance {} outside expected range [{}, {}]", - balance, self.min, self.max - )) - } - } - Err(e) => ValidationResult::error(format!("Failed to parse balance: {}", e)), - } - } -} - -// Usage -let mut runner = TestRunner::new(config); -runner.add_validator(Box::new(BalanceRangeValidator { - min: 100, - max: 10000 -})); -``` - -### Extension Development Testing - -Test custom extensions: - -```rust -use pvq_test_runner::extension_testing::ExtensionTester; - -#[tokio::test] -async fn test_custom_extension() { - let mut tester = ExtensionTester::new(); - - // Register custom extension - tester.register_extension::(); - - // Test extension directly - let result = tester.call_extension( - "my_custom_extension", - "my_function", - vec![json!("test_input")] - ).await.unwrap(); - - assert_eq!(result, json!("expected_output")); - - // Test in program context - let program_result = tester.execute_program_with_extension( - "output/custom-extension-test.polkavm", - &("test_input",) - ).await.unwrap(); - - assert!(program_result.success); -} -``` - -## Debugging - -### Debug Mode - -Enable comprehensive debugging: +### CLI usage ```bash -# Enable all debug output -pvq-test-runner --program output/my-program.polkavm --debug - -# Specific debug categories -RUST_LOG=pvq_test_runner=debug,pvq_executor=trace pvq-test-runner \ - --program output/my-program.polkavm -``` - -### Trace Output - -Example trace output: - -``` -[DEBUG pvq_test_runner] Loading program: output/balance-query.polkavm -[TRACE pvq_executor] Creating PolkaVM instance -[DEBUG pvq_extension] Registering extension: core -[DEBUG pvq_extension] Registering extension: fungibles -[TRACE pvq_executor] Program loaded, size: 4096 bytes -[DEBUG pvq_test_runner] Executing with args: [1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]] -[TRACE pvq_executor] Starting program execution -[DEBUG pvq_extension] Extension call: core.current_block_number() -> 12345 -[DEBUG pvq_extension] Extension call: fungibles.balance(1, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]) -> 1000 -[TRACE pvq_executor] Program execution completed -[DEBUG pvq_test_runner] Result: 1000 -``` - -## Performance Analysis - -### Profiling Reports - -Detailed performance analysis: - -``` -Performance Analysis Report -=========================== - -Program: output/complex-analysis.polkavm -Total Execution Time: 245ms - -Breakdown: -β”œβ”€ Program Loading: 5ms (2.0%) -β”œβ”€ Extension Setup: 3ms (1.2%) -β”œβ”€ Program Execution: 230ms (93.9%) -β”‚ β”œβ”€ Extension Calls: 45ms (18.4%) -β”‚ β”‚ β”œβ”€ fungibles.list_assets: 5ms -β”‚ β”‚ β”œβ”€ fungibles.balance (Γ—10): 25ms -β”‚ β”‚ └─ swap.pool_info (Γ—5): 15ms -β”‚ └─ Program Logic: 185ms (75.5%) -└─ Result Serialization: 7ms (2.9%) - -Memory Usage: -β”œβ”€ Peak Usage: 12.5MB -β”œβ”€ Average Usage: 8.2MB -└─ Final Usage: 1.1MB - -Extension Call Statistics: -β”œβ”€ Total Calls: 16 -β”œβ”€ Average Call Time: 2.8ms -β”œβ”€ Slowest Call: swap.pool_info (4.2ms) -└─ Most Frequent: fungibles.balance (10 calls) -``` - -### Optimization Suggestions - -The test runner provides optimization hints: +cargo run -p pvq-test-runner -- --program path/to/program.polkavm --chain poc --entrypoint-idx 0 +# Print only prepared input/expected result +cargo run -p pvq-test-runner -- --program path/to/program.polkavm --chain poc --entrypoint-idx 0 --print-data ``` -Optimization Suggestions: -======================== -πŸ” Performance Issues Detected: -β”œβ”€ High extension call overhead (18.4% of execution time) -β”‚ └─ Consider batching balance queries into single operation -β”œβ”€ Memory fragmentation detected -β”‚ └─ Consider using Vec::with_capacity for known-size collections -└─ Serialization overhead significant (2.9%) - └─ Consider using more efficient serialization format +### Build -βœ… Good Practices Observed: -β”œβ”€ Efficient extension usage patterns -β”œβ”€ Proper error handling -└─ Minimal memory allocations in hot paths -``` - -## Troubleshooting - -### Common Issues - -**Program won't load:** -```bash -# Check program format -file output/my-program.polkavm - -# Verify program was built correctly -ls -la output/my-program.polkavm - -# Try with minimal configuration -pvq-test-runner --program output/my-program.polkavm --extensions core -``` - -**Extension errors:** ```bash -# Enable extension debug logging -RUST_LOG=pvq_extension=debug pvq-test-runner --program output/my-program.polkavm - -# Check available extensions -pvq-test-runner --program output/my-program.polkavm --extensions core --debug -``` - -**Memory issues:** -```bash -# Increase memory limit -pvq-test-runner --program output/my-program.polkavm --memory-limit 128 - -# Enable memory profiling -pvq-test-runner --program output/my-program.polkavm --profile --debug +cargo build -p pvq-test-runner --release ``` - -## Related Components - -- [PVQ Program](../pvq-program/) - Program development tools -- [PVQ Executor](../pvq-executor/) - Execution environment -- [PVQ Extension System](../pvq-extension/) - Extension framework -- [PVQ Program Metadata Gen](../pvq-program-metadata-gen/) - Metadata generation - ---- - -*The PVQ Test Runner provides comprehensive testing and development support for PVQ programs.* \ No newline at end of file