diff --git a/README.md b/README.md index 3cc6e37..0e821e5 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,16 @@ This repository contains examples and guides for how to interact with the Space ## Tutorials -- [Hello World Tutorial](./hello_world_tutorial/HELLO_WORLD_TUTORIAL.md) +- [Hello World Tutorial](./tutorials/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md) +- [Community Tables Tutorial](./tutorials/community_tutorial/COMMUNITY_TABLE_TUTORIAL.md) +- [Public Tables Tutorial](./tutorials/public_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md) ## How-To Guides - [How to convert between address formats](./how_to/HOW_TO_CONVERT_BETWEEN_ADDRESS_FORMATS.md) - [How To fund a wallet with Dreamspace Pay](./how_to/HOW_TO_FUND_A_WALLET_WITH_DSPAY.md) + +## Documentation +- [Tables Overview](./tables/OVERVIEW.md) +- [PublicPermissionless Tables](./tables/PUBLIC_TABLES.md) +- [Community Tables](./tables/COMMUNITY_TABLES.md) \ No newline at end of file diff --git a/tables/COMMUNITY_TABLES.md b/tables/COMMUNITY_TABLES.md new file mode 100644 index 0000000..ad7fc9a --- /dev/null +++ b/tables/COMMUNITY_TABLES.md @@ -0,0 +1,50 @@ +# Community Tables + +Community tables are user-owned tables with **controlled access**. Only the table creator and users who have been granted explicit permission can insert data into these tables, making them ideal for private data storage and scenarios where you need to control who can write to your tables. + +## When to use Community Tables + +- Private application data +- User management systems +- Controlled datasets +- Internal business data +- Scenarios where you need permission-based access control +- Multi-user applications with role-based data submission + +# Example Use Case - User Management System + +In this example we'll discuss a generic user management system built on Community tables that stores basic membership information. Only authorized submitters can insert or modify the user data. To create the system, we will create a namespace and a table, then we'll show how authorized submitters would add user records. + +In this example we'll create the statements as if we are using the wallet address `0xABC8d709C80262965344f5240Ad123f5cBE51123` + +So our namespace will be + +`MYAPP_ABC8d709C80262965344f5240Ad123f5cBE51123` + +And we'll use the following statements: + +```sql +CREATE SCHEMA IF NOT EXISTS MYAPP_ABC8d709C80262965344f5240Ad123f5cBE51123; +CREATE TABLE IF NOT EXISTS MYAPP_ABC8d709C80262965344f5240Ad123f5cBE51123.USERS ( + USER_ID BIGINT NOT NULL, + USERNAME VARCHAR NOT NULL, + EMAIL VARCHAR NOT NULL, + CREATED_AT BIGINT NOT NULL, + PRIMARY_KEY(USER_ID) +); +``` +Which will result in a table like this: + +| `USER_ID` (`BIGINT`) | `USERNAME` (`VARCHAR`) | `EMAIL` (`VARCHAR`) | `CREATED_AT` (`BIGINT`) | +| -------------------- | ---------------------- | ------------------- | ----------------------- | +| 1 | alice | alice@example.com | 1700000000 | +| 2 | bob | bob@example.com | 1700000100 | +| 3 | charlie | charlie@example.com | 1700000200 | + +It's worth noting here that the table schema only includes the columns you define. **Unlike PublicPermissionless tables, Community tables do NOT automatically add a `META_SUBMITTER` column.** + +## Granting Insert Permissions + +By default, only the table creator can insert data. To allow other addresses to insert data, you must explicitly grant them permission using the `permissions.addProxyPermission()` transaction with the `IndexingPallet.SubmitDataForPrivilegedQuorum` permission. + +Scripts, examples, and a walkthrough for creating namespaces, tables, and permissioning users can be found in the [Community Tables Tutorial](../tutorials/community_tutorial/COMMUNITY_TABLE_TUTORIAL.md) diff --git a/tables/OVERVIEW.md b/tables/OVERVIEW.md new file mode 100644 index 0000000..4211b45 --- /dev/null +++ b/tables/OVERVIEW.md @@ -0,0 +1,47 @@ +# Tables on Space and Time + +## Overview + +Space and Time (SXT Chain) is the first trustless database for the EVM, enabling smart contracts to interact with historical, cross-chain, or offchain data as if it were natively accessible onchain. Space and Time (SXT Chain) is optimized for: + +* High-throughput data ingestion from blockchains, consumer apps, and enterprise sources +* Verifiable query execution over large datasets (millions of rows) +* Fast ZK proof generation and EVM-compatible proof verification with minimal gas + +## Using Tables + +SXT Chain is a permissionless L1 blockchain that acts as a decentralized database and enables developers to define, own, and maintain tamperproof tables using standard SQL DDL. These tables form the foundation of Space and Time’s verifiable data model, allowing structured data to be inserted, queried, and cryptographically proven at scale. + +Creating a table on SXT Chain is similar to defining a schema in any relational database, but with added guarantees of verifiability and decentralized consensus. Developers submit to SXT Chain an ECDSA or ED25519-signed Substrate transaction containing CREATE TABLE SQL syntax to define table structure, including column types, and constraints. + +Once created, the table is: + + Assigned a unique table ID + Associated with an initial table owner (the signer of the DDL transaction) + Registered in the SXT Chain under consensus + Bound to a cryptographic commitment representing the table’s state (initially empty) + +All inserts, updates, and deletions from that point forward are tracked by cryptographic commitments stored onchain, allowing for ZK-proven query execution and verifiable data integrity. + +When creating a SXT Chain table, there are several types to choose from depending on the application and access required: + + Public/Permissionless Tables: Any user can submit data (e.g., public content feeds or game leaderboards). + Community Tables: Only the table owner (or a whitelisted set of public keys) can insert data. Useful for oracle publishers or protocol-owned datasets. + +Ownership is defined at table creation via the wallet signature that initiates the DDL transaction. + +## Paying for Tables and Inserts + +Table creation is paid for using compute credits. Compute credits can be acquired through a few different means, mentioned in other parts of this documentation. + +| Action | Cost | +|--------------------|-------------------| +| Table Creation | 20 SXT per Table | +| Namespace Creation | 20 SXT per Schema | +| Row Inserts | ~0.02 SXT per row | + +## More Reading + +* [PublicPermissionless Tables](./PUBLIC_TABLES.MD) +* [Community Tables](./COMMUNITY_TABLES.MD) + diff --git a/tables/PUBLIC_TABLES.md b/tables/PUBLIC_TABLES.md new file mode 100644 index 0000000..c1cff74 --- /dev/null +++ b/tables/PUBLIC_TABLES.md @@ -0,0 +1,43 @@ +# Public/Permissionless Tables + +PublicPermissionless tables allow anyone to insert data without requiring explicit permissions. The system automatically tracks who submitted each row using a `META_SUBMITTER` column, allowing for analytics. + +## When to use PublicPermissionless Tables + +- Open data collection (surveys, feedback, reports) +- Crowdsourced datasets +- Public registries and directories +- Community-driven data contributions +- Scenarios where you want open participation with accountability + +# Example Use Case - Poll System + +In this example we'll discuss a Poll System built on PublicPermissionless tables that allows users to vote on which movie will be the leader in box office earnings each week. To create the system, we will create a namespace and a table, then we'll show how users would 'vote' on the poll by inserting data into the table. + +In this example we'll create the statements as if we are using the wallet address `0xABC8d709C80262965344f5240Ad123f5cBE51123` + +So our namespace will be + +`MOVIES_ABC8d709C80262965344f5240Ad123f5cBE51123` + +And we'll use the following statements: + +```sql +CREATE SCHEMA IF NOT EXISTS MOVIES_ABC8d709C80262965344f5240Ad123f5cBE51123; +CREATE TABLE IF NOT EXISTS MOVIES_ABC8d709C80262965344f5240Ad123f5cBE51123.VOTES ( + MOVIE VARCHAR NOT NULL, + WEEK BIGINT NOT NULL, + PRIMARY_KEY(MOVIE, WEEK) +); +``` +Which will, after being modified by the chain, result in a table like this; + +| `MOVIE` (`VARCHAR`) | `WEEK` (`BIGINT`) | `META_SUBMITTER` (`BINARY`) | +|---------------------|-------------------| --------------------------- | +| Movie A | 2 | 0x000...user1address | +| Movie A | 2 | 0x000...user2address | +| Movie B | 2 | 0x000...user3address | + +It's worth noting here that the table schema and the data submitted **should not include** the `META_SUBMITTER` column. + +You can follow along in the [PublicPermission Tables Tutorial](../tutorials/public_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md) diff --git a/tutorials/community_tutorial/COMMUNITY_TABLE_TUTORIAL.md b/tutorials/community_tutorial/COMMUNITY_TABLE_TUTORIAL.md new file mode 100644 index 0000000..955041a --- /dev/null +++ b/tutorials/community_tutorial/COMMUNITY_TABLE_TUTORIAL.md @@ -0,0 +1,268 @@ +# Community Table Tutorial + +In this tutorial, we will be creating a **Community table** on the Space and Time chain using an ECDSA wallet. + +Community tables are user-owned tables with **controlled access**. Only the table creator and users who have been granted permission can insert data into these tables. This makes them ideal for private data storage, application data, or any scenario where you need to control who can write to your tables. + +The major steps we will walk through are: + +1. Funding a wallet with compute credits. +2. Creating a Community table on the Space and Time chain. +3. Granting insert permissions to other addresses (optional). +4. Inserting data into the new table. + + +## Step 1: Funding a Wallet + +In order to interact with the Space and Time chain, you need compute credits. In this tutorial, we will use the wallet address `0xABC8d709C80262965344f5240Ad123f5cBE51123` and will be funding the wallet with 100 SXT. + +You can fund your wallet using one of two methods: + +### Option A: Manual Funding + +To manually fund your wallet, you can follow the instructions in the [Hello World Tutorial](../../hello_world_tutorial/HELLO_WORLD_TUTORIAL.md) + +### Option B: Dreamspace Pay + +For a simpler funding experience, see the [How to fund a wallet with Dreamspace Pay](../../how_to/HOW_TO_FUND_A_WALLET_WITH_DSPAY.md) guide. + +## Step 2: Clone this repo + +We will be using this repo, which has the scripts that we will be using. So, we clone this repo: + +```bash +git clone git@github.com:spaceandtimefdn/sxt-chain-examples.git +cd sxt-chain-examples +``` + +We will be using [Node.js](https://nodejs.org/en/download/current) which can be installed a variety of ways. + +Then, we need to install the prerequisite npm packages: + +```bash +npm install +``` + +Then, we can make the `community_table_tutorial` directory our working directory. + +```bash +cd community_table_tutorial +``` + +## Step 3: Creating a Community Table (DDL) + +We will write a node script that will create a Community table for us. This script is named `community_create_table.js`. + +For this tutorial, we'll create a user management table called `USERS` with the following schema: + +| `USER_ID` (`BIGINT`) | `USERNAME` (`VARCHAR`) | `EMAIL` (`VARCHAR`) | `CREATED_AT` (`BIGINT`) | +| -------------------- | ---------------------- | ------------------- | ----------------------- | +| 1 | alice | alice@example.com | 1700000000 | +| 2 | bob | bob@example.com | 1700000100 | +| 3 | charlie | charlie@example.com | 1700000200 | + +The first thing we must do in order to create a table is create a connection with a Space and Time RPC node: + +```javascript +const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); +const api = await ApiPromise.create({ provider, noInitWarn: true }); +``` + +Next, we build a transaction to create a new namespace. **Important**: The namespace must end with the wallet address (without the `0x` prefix, uppercase). We will use the namespace `MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345`. + +```javascript +const createNamespaceTX = api.tx.tables.createNamespace( + "MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345", + 0, + "CREATE SCHEMA IF NOT EXISTS MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345", + "Community", + { UserCreated: "My Application Namespace" }, +); +``` + +Then, we build a transaction for the DDL of the new Community table. + +```javascript +const createTablesTX = api.tx.tables.createTables([ + { + ident: { + namespace: "MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345", + name: "USERS", + }, + createStatement: + "CREATE TABLE MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345.USERS (USER_ID BIGINT NOT NULL, USERNAME VARCHAR NOT NULL, EMAIL VARCHAR NOT NULL, CREATED_AT BIGINT NOT NULL)", + tableType: "Community", + commitment: { Empty: { hyperKzg: true } }, + source: { UserCreated: "User management table" }, + }, +]); +``` + +Then, instead of submitting two separate transactions, we opt to batch these into a single transaction: + +```javascript +const batchTX = api.tx.utility.batchAll([createNamespaceTX, createTablesTX]); +``` + +We must sign the transaction. To do this, we add the private key of our `0xDEF...345` wallet as an environment variable. We add it to a `.env` file and `import 'dotenv/config';`. The `.env` file looks like this: + +``` +PRIVATE_KEY=d157███████████████████████ REDACTED ███████████████████████f415 +``` + +We can then create a wallet in our script by using the `ethers` package and the custom `EthEcdsaSigner`: + +```javascript +const wallet = new Wallet(process.env.PRIVATE_KEY); +const signer = new EthEcdsaSigner(wallet, api); +``` + +Finally, we submit the transaction to the Space and Time chain to create the namespace and table. + +```javascript +await batchTX.signAndSend(signer.address, { signer }); +``` + +Now, we run the script with: + +```bash +node community_create_table.js +``` + +**What makes this a Community table?** +- The `tableType` is set to `"Community"` +- Only you (the creator) can insert data initially +- You can grant permission to other specific users to insert data +- No automatic `META_SUBMITTER` column is added + +## Step 4: Granting Insert Permissions (Optional) + +Community tables have **controlled access**, meaning only authorized addresses can insert data. By default, only the table creator (you) can insert data. If you want to allow another wallet address to insert data, you need to explicitly grant them permission. + +**Note**: If you're only inserting data from your own wallet (the table creator), you can skip this step and go directly to Step 5. + +### When to grant permissions + +You should grant insert permissions when: +- You have multiple wallets or services that need to insert data +- You're building an application where different users submit data to your table +- You want to delegate data submission to another party + +### Granting permission to another address + +To grant insert permission to another address, we'll use the `add_insert_permission.js` script. + +First, add the address you want to grant permission to in your `.env` file: + +``` +PRIVATE_KEY=d157███████████████████████ REDACTED ███████████████████████f415 +DATA_SUBMITTER_ADDRESS=0xABC1234567890ABCDEF1234567890ABCDEF67890 +``` + +The script will: +1. Convert the Ethereum address to Substrate AccountId format (32 bytes: 12 zero bytes + 20-byte Ethereum address) +2. Create a permission for `IndexingPallet.SubmitDataForPrivilegedQuorum` for the specific table +3. Submit the transaction using `api.tx.permissions.addProxyPermission()` + +Here's the key part of the permission granting code: + +```javascript +// Convert Ethereum address to Substrate AccountId format +const dataSubmitterAccountId = "0x" + "00".repeat(12) + dataSubmitterAddress.substring(2).toUpperCase(); + +// Create the permission for submitting data to this specific table +const permission = { + IndexingPallet: { + SubmitDataForPrivilegedQuorum: { + namespace: namespace, + name: "USERS" + } + } +}; + +// Build the transaction to add data submitter permission +const addPermissionTx = api.tx.permissions.addProxyPermission( + dataSubmitterAccountId, + permission +); +``` + +Run the script to grant the permission: + +```bash +node add_insert_permission.js +``` + +You should see output like: + +``` +Connected to RPC. +Granting permission to: 0xABC1234567890ABCDEF1234567890ABCDEF67890 +For table: MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345.USERS +Signing and sending transaction... +Permission granted successfully! +``` + +Once the permission is granted, the specified address can now submit data to your Community table. + +## Step 5: Inserting Data (DML) + +Now that we have created the table (and optionally granted permissions to other addresses), we can insert data. The corresponding script is named `community_insert_data.js`. + +**Important**: You can only insert data if you are either: +- The table creator (your wallet created the table in Step 3), or +- An address that has been granted insert permission (via Step 4) + +The majority of the script is identical to the one that created the table. The only difference is how we build the transaction. + +The data is inserted to the chain as an Apache Arrow table. We build the table as follows: + +```javascript +const table = new Table({ + USER_ID: vectorFromArray([1n, 2n, 3n], new Int64()), + USERNAME: vectorFromArray(["alice", "bob", "charlie"], new Utf8()), + EMAIL: vectorFromArray( + ["alice@example.com", "bob@example.com", "charlie@example.com"], + new Utf8() + ), + CREATED_AT: vectorFromArray([1700000000n, 1700000100n, 1700000200n], new Int64()) +}); +``` + +Finally, we build the transaction that will insert the table. We need to specify a unique `batchId` for each insertion. + +```javascript +const batchId = "MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345.USERS.1"; +const insertDataTx = api.tx.indexing.submitData( + { + namespace: "MYAPP_DEF1234567890ABCDEF1234567890ABCDEF12345", + name: "USERS", + }, + batchId, + u8aToHex(tableToIPC(table)), +); +``` + +We run the script with: + +```bash +node community_insert_data.js +``` + +We have now created a Community table and inserted data! + +## Community vs PublicPermissionless Tables + +**Community tables** are best when you need **controlled access**: +- ✅ Private application data +- ✅ User management systems +- ✅ Controlled datasets +- ✅ Data that requires permission to modify + +For **open-access** tables where anyone can insert data, see the [PublicPermissionless Table Tutorial](../public_permissionless_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md). + +## Next Steps + +- Learn about [PublicPermissionless tables](../public_permissionless_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md) for open crowdsourced data +- Explore how to grant permissions to other users (see the [Community Tables How-To Guide](../docs/how-to-create-community-tables.md)) +- Query your data using Proof of SQL diff --git a/tutorials/community_tutorial/add_insert_permission.js b/tutorials/community_tutorial/add_insert_permission.js new file mode 100644 index 0000000..ef57d2c --- /dev/null +++ b/tutorials/community_tutorial/add_insert_permission.js @@ -0,0 +1,75 @@ +import "dotenv/config"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Wallet } from "ethers"; +import { EthEcdsaSigner } from "../lib/ethecdsa_signer.js"; + +async function main() { + console.log("Connecting to RPC..."); + const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); + const api = await ApiPromise.create({ provider, noInitWarn: true }); + console.log("Connected to RPC."); + + // Get wallet address for namespace + const wallet = new Wallet(process.env.PRIVATE_KEY); + const ethAddress = wallet.address.substring(2).toUpperCase(); + const namespace = `MYAPP_${ethAddress}`; + + // The data submitter wallet address to grant permission to + const dataSubmitterAddress = process.env.DATA_SUBMITTER_ADDRESS; + + if (!dataSubmitterAddress) { + console.error("Error: DATA_SUBMITTER_ADDRESS environment variable not set"); + console.log("Add DATA_SUBMITTER_ADDRESS=0x... to your .env file"); + process.exit(1); + } + + // Convert Ethereum address to Substrate AccountId format + // Substrate AccountIds are 32 bytes: 12 zero bytes + 20-byte Ethereum address + const dataSubmitterAccountId = "0x" + "00".repeat(12) + dataSubmitterAddress.substring(2).toUpperCase(); + + console.log("Granting permission to:", dataSubmitterAddress); + console.log("For table:", `${namespace}.USERS`); + + // Create the permission for submitting data to this specific table + const permission = { + IndexingPallet: { + SubmitDataForPrivilegedQuorum: { + namespace: namespace, // <--- Your Namespace + name: "USERS" // <--- Your Table Name + } + } + }; + + // Build the transaction to add data submitter permission + const addPermissionTx = api.tx.permissions.addProxyPermission( + dataSubmitterAccountId, + permission + ); + + const signer = new EthEcdsaSigner(wallet, api); + console.log("Signing and sending transaction..."); + const unsub = await addPermissionTx.signAndSend( + signer.address, + { signer }, + async (status) => { + if (status.isFinalized) { + console.log("Permission granted successfully!"); + console.log("Finalized in block", status.blockNumber.toString()); + console.log(""); + console.log("Details:"); + console.log(" Table:", `${namespace}.USERS`); + console.log(" Data submitter address:", dataSubmitterAddress); + console.log(" Permission: SubmitDataForPrivilegedQuorum"); + console.log(""); + console.log("The data submitter address can now submit data to this table."); + unsub(); + process.exit(0); + } + }, + ); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/tutorials/community_tutorial/community_create_table.js b/tutorials/community_tutorial/community_create_table.js new file mode 100644 index 0000000..aa71abc --- /dev/null +++ b/tutorials/community_tutorial/community_create_table.js @@ -0,0 +1,69 @@ +import "dotenv/config"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Wallet } from "ethers"; +import { EthEcdsaSigner } from "../lib/ethecdsa_signer.js"; + +async function main() { + console.log("Connecting to RPC..."); + const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); + const api = await ApiPromise.create({ provider, noInitWarn: true }); + console.log("Connected to RPC."); + + // Get wallet address for namespace + const wallet = new Wallet(process.env.PRIVATE_KEY); + const ethAddress = wallet.address.substring(2).toUpperCase(); // Remove 0x and uppercase + const namespace = `MYAPP_${ethAddress}`; + + console.log("Creating namespace:", namespace); + + // Step 1: Create namespace + const createNamespaceTX = api.tx.tables.createNamespace( + namespace, + 0, + `CREATE SCHEMA IF NOT EXISTS ${namespace}`, + "Community", + { UserCreated: "My Application Namespace" }, + ); + + const createTablesTX = api.tx.tables.createTables([ + { + ident: { + namespace: namespace, + name: "USERS", + }, + createStatement: + `CREATE TABLE ${namespace}.USERS (` + + `USER_ID BIGINT NOT NULL, ` + + `USERNAME VARCHAR NOT NULL, ` + + `EMAIL VARCHAR NOT NULL, ` + + `CREATED_AT BIGINT NOT NULL` + + `)`, + tableType: "Community", + commitment: { Empty: { hyperKzg: true } }, + source: { UserCreated: "AppData" }, + }, + ]); + + const batchTX = api.tx.utility.batchAll([createNamespaceTX, createTablesTX]); + + const signer = new EthEcdsaSigner(wallet, api); + console.log("Signing and sending transaction..."); + const unsub = await batchTX.signAndSend( + signer.address, + { signer }, + async (status) => { + if (status.isFinalized) { + console.log("Finalized in block", status.blockNumber.toString()); + console.log("Namespace:", namespace); + console.log("Table:", `${namespace}.USERS`); + unsub(); + process.exit(0); + } + }, + ); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/tutorials/community_tutorial/community_insert_data.js b/tutorials/community_tutorial/community_insert_data.js new file mode 100644 index 0000000..2953e06 --- /dev/null +++ b/tutorials/community_tutorial/community_insert_data.js @@ -0,0 +1,66 @@ +import "dotenv/config"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { u8aToHex } from "@polkadot/util"; +import { Wallet } from "ethers"; +import { EthEcdsaSigner } from "../lib/ethecdsa_signer.js"; +import { + Table, + Int64, + Utf8, + vectorFromArray, + tableToIPC, +} from "apache-arrow"; + +async function main() { + console.log("Connecting to RPC..."); + const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); + const api = await ApiPromise.create({ provider, noInitWarn: true }); + console.log("Connected to RPC."); + + // Get wallet address for namespace + const wallet = new Wallet(process.env.PRIVATE_KEY); + const ethAddress = wallet.address.substring(2).toUpperCase(); + const namespace = `MYAPP_${ethAddress}`; + + console.log("Inserting data into:", `${namespace}.USERS`); + + const table = new Table({ + USER_ID: vectorFromArray([1n, 2n, 3n], new Int64()), + USERNAME: vectorFromArray(["alice", "bob", "charlie"], new Utf8()), + EMAIL: vectorFromArray( + ["alice@example.com", "bob@example.com", "charlie@example.com"], + new Utf8() + ), + CREATED_AT: vectorFromArray([1700000000n, 1700000100n, 1700000200n], new Int64()) + }); + + // Submit data (every batch id must be unique) + const batchId = 1; + const insertDataTx = api.tx.indexing.submitData( + { + namespace: namespace, + name: "USERS", + }, + batchId, + u8aToHex(tableToIPC(table)), + ); + + const signer = new EthEcdsaSigner(wallet, api); + console.log("Signing and sending transaction..."); + const unsub = await insertDataTx.signAndSend( + signer.address, + { signer }, + async (status) => { + if (status.isFinalized) { + console.log("Finalized in block", status.blockNumber.toString()); + unsub(); + process.exit(0); + } + }, + ); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/hello_world_tutorial/doc_assets/tutorial_approve.png b/tutorials/doc_assets/tutorial_approve.png similarity index 100% rename from hello_world_tutorial/doc_assets/tutorial_approve.png rename to tutorials/doc_assets/tutorial_approve.png diff --git a/hello_world_tutorial/doc_assets/tutorial_funded_message.png b/tutorials/doc_assets/tutorial_funded_message.png similarity index 100% rename from hello_world_tutorial/doc_assets/tutorial_funded_message.png rename to tutorials/doc_assets/tutorial_funded_message.png diff --git a/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md b/tutorials/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md similarity index 97% rename from hello_world_tutorial/HELLO_WORLD_TUTORIAL.md rename to tutorials/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md index dc03d0b..5fed3ee 100644 --- a/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md +++ b/tutorials/hello_world_tutorial/HELLO_WORLD_TUTORIAL.md @@ -19,7 +19,7 @@ First, we must approve the spend. We call the [`approve`](https://etherscan.io/a We are using Etherscan, so we do the following: -![Approve Funds](doc_assets/tutorial_approve.png) +![Approve Funds](../doc_assets/tutorial_approve.png) Next, we need to actually fund the wallet. We call the [`fundAddress`](https://etherscan.io/address/0xb1bc1d7eb1e6c65d0de909d8b4f27561ef568199#writeContract#F2) function on the SXTChainFunding contract. @@ -28,7 +28,7 @@ Next, we need to actually fund the wallet. We call the [`fundAddress`](https://e As before, on Etherscan, we get: -![Send Funds](doc_assets/tutorial_funded_message.png) +![Send Funds](../doc_assets/tutorial_funded_message.png) ## Step 2: Clone this repo @@ -68,7 +68,7 @@ We will write a node script that will create a table for us. This script is name The first thing we must do in order to create a table is create a connection with a Space and Time RPC node: ```javascript -const provider = new WsProvider("wss://rpc.testnet.sxt.network"); +const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); const api = await ApiPromise.create({ provider, noInitWarn: true }); ``` diff --git a/hello_world_tutorial/hello_world_create_table.js b/tutorials/hello_world_tutorial/hello_world_create_table.js similarity index 100% rename from hello_world_tutorial/hello_world_create_table.js rename to tutorials/hello_world_tutorial/hello_world_create_table.js diff --git a/hello_world_tutorial/hello_world_insert_data.js b/tutorials/hello_world_tutorial/hello_world_insert_data.js similarity index 100% rename from hello_world_tutorial/hello_world_insert_data.js rename to tutorials/hello_world_tutorial/hello_world_insert_data.js diff --git a/tutorials/public_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md b/tutorials/public_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md new file mode 100644 index 0000000..a0d8bce --- /dev/null +++ b/tutorials/public_tutorial/PUBLIC_PERMISSIONLESS_TUTORIAL.md @@ -0,0 +1,254 @@ +# PublicPermissionless Table Tutorial + +In this tutorial, we will create a PublicPermissionless table on the Space and Time chain using an ECDSA wallet. The example demonstrates a movie polling system where users can vote on which movie will lead box office earnings each week. + +PublicPermissionless tables are open-access tables where anyone can insert data. The system automatically tracks who submitted each row using a META_SUBMITTER column. This makes them suitable for crowdsourced datasets, public feedback, polls, or any scenario requiring open participation with accountability. + +The major steps we will walk through are: + +1. Funding a wallet with compute credits +2. Cloning this repository +3. Creating a PublicPermissionless table on the Space and Time chain +4. Inserting data into the table +5. Understanding the META_SUBMITTER column + +## Step 1: Funding a Wallet + +In order to interact with the Space and Time chain, you need compute credits. In this tutorial, we will use the wallet address `0xABC8d709C80262965344f5240Ad123f5cBE51123` and will be funding the wallet with 100 SXT. + +You can fund your wallet using one of two methods: + +### Option A: Manual Funding + +To manually fund your wallet, you can follow the instructions in the [Hello World Tutorial](../../hello_world_tutorial/HELLO_WORLD_TUTORIAL.md) + +### Option B: Dreamspace Pay + +For a simpler funding experience, see the [How to fund a wallet with Dreamspace Pay](../../how_to/HOW_TO_FUND_A_WALLET_WITH_DSPAY.md) guide. + +## Step 2: Clone this Repository + +Clone this repository which contains the scripts you will need: + +```bash +git clone git@github.com:spaceandtimefdn/sxt-chain-examples.git +cd sxt-chain-examples +``` + +Install [Node.js](https://nodejs.org/en/download/current) if you have not already. + +Install the prerequisite npm packages: + +```bash +npm install +``` + +Navigate to the public_example directory: + +```bash +cd tables/public_example +``` + +## Step 3: Creating a PublicPermissionless Table + +We will write a node script that creates a PublicPermissionless table. This script is named `public_create_table.js`. + +For this tutorial, we will create a movie polling table called `VOTES` with the following schema: + +| `MOVIE` (`VARCHAR`) | `WEEK` (`BIGINT`) | `META_SUBMITTER` (`BINARY`) | +|---------------------|-------------------|-----------------------------| +| Movie A | 2 | 0x000...user1address | +| Movie A | 2 | 0x000...user2address | +| Movie B | 2 | 0x000...user3address | + +Note: The META_SUBMITTER column is not included in the CREATE statement. The system automatically adds it to track who submits each row. + +The first step is to create a connection with a Space and Time RPC node: + +```javascript +const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); +const api = await ApiPromise.create({ provider, noInitWarn: true }); +``` + +Next, build a transaction to create a new namespace. The namespace must end with the wallet address (without the `0x` prefix, uppercase). We will use the namespace `MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123`. + +```javascript +const createNamespaceTX = api.tx.tables.createNamespace( + "MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123", + 0, + "CREATE SCHEMA IF NOT EXISTS MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123", + "PublicPermissionless", + { UserCreated: "MoviesPoll" }, +); +``` + +Then, build a transaction for the DDL of the new PublicPermissionless table. Do not include the META_SUBMITTER column in your CREATE statement as it is added automatically. + +```javascript +const createTablesTX = api.tx.tables.createTables([ + { + ident: { + namespace: "MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123", + name: "VOTES", + }, + createStatement: + "CREATE TABLE IF NOT EXISTS MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123.VOTES (" + + "MOVIE VARCHAR NOT NULL, " + + "WEEK BIGINT NOT NULL, " + + "PRIMARY_KEY(MOVIE, WEEK))", + tableType: "PublicPermissionless", + commitment: { Empty: { hyperKzg: true } }, + source: { UserCreated: "MoviesPoll" }, + }, +]); +``` + +Instead of submitting two separate transactions, batch them into a single transaction: + +```javascript +const batchTX = api.tx.utility.batchAll([createNamespaceTX, createTablesTX]); +``` + +To sign the transaction, add the private key of your wallet as an environment variable. Add it to a `.env` file and include `import 'dotenv/config';` in your script. The `.env` file should look like this: + +``` +PRIVATE_KEY=a234███████████████████████ REDACTED ███████████████████████b567 +``` + +Create a wallet in your script using the `ethers` package and the custom `EthEcdsaSigner`: + +```javascript +const wallet = new Wallet(process.env.PRIVATE_KEY); +const signer = new EthEcdsaSigner(wallet, api); +``` + +Finally, submit the transaction to the Space and Time chain to create the namespace and table: + +```javascript +await batchTX.signAndSend(signer.address, { signer }); +``` + +Run the script: + +```bash +node public_create_table.js +``` + +What makes this a PublicPermissionless table: + +- The `tableType` is set to `"PublicPermissionless"` +- Anyone can insert data without permissions +- A `META_SUBMITTER BINARY NOT NULL` column is automatically added to track who submitted each row +- You do not include `META_SUBMITTER` in your CREATE statement + +## Step 4: Inserting Data + +Now that the table exists, anyone can insert data. The corresponding script is named `public_insert_data.js`. + +The majority of the script is identical to the one that created the table. The difference is in how we build the transaction. + +The data is inserted to the chain as an Apache Arrow table. Build the table as follows. Do not include the META_SUBMITTER column in your data as the chain adds it automatically. + +```javascript +const table = new Table({ + MOVIE: vectorFromArray( + ["Movie A", "Movie B", "Movie C"], + new Utf8() + ), + WEEK: vectorFromArray([2n, 2n, 2n], new Int64()) +}); +``` + +Build the transaction that will insert the table. Specify a unique `batchId` for each insertion. + +```javascript +const batchId = "MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123.VOTES." + Date.now(); +const insertDataTx = api.tx.indexing.submitData( + { + namespace: "MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123", + name: "VOTES", + }, + batchId, + u8aToHex(tableToIPC(table)), +); +``` + +Run the script: + +```bash +node public_insert_data.js +``` + +You have now created a PublicPermissionless table and inserted data. Anyone with a funded wallet can now insert data into this table. + +## Step 5: Understanding META_SUBMITTER + +The META_SUBMITTER column is the key feature that distinguishes PublicPermissionless tables. It provides automatic data provenance tracking. + +### What is META_SUBMITTER + +- Type: `BINARY NOT NULL` (32 bytes) +- Content: The AccountId of whoever submitted each row +- Format: + - Ethereum addresses: 12 zero bytes + 20-byte Ethereum address + - Substrate addresses: Full 32-byte public key +- Automatic: You never include it in CREATE statements or INSERT data as the chain adds it + +### Example: Your Address in META_SUBMITTER + +If your Ethereum address is `0xABC8d709C80262965344f5240Ad123f5cBE51123`, your META_SUBMITTER value will be: + +``` +0x000000000000000000000000ABC8D709C80262965344F5240AD123F5CBE51123 +``` + +This is 12 zero bytes padding plus your 20-byte address. + +### Querying by Submitter + +When querying the table with Proof of SQL, you can filter by META_SUBMITTER: + +```sql +-- Find all votes from a specific submitter +SELECT * FROM MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123.VOTES +WHERE META_SUBMITTER = 0x000000000000000000000000ABC8D709C80262965344F5240AD123F5CBE51123; + +-- Count votes by each user +SELECT META_SUBMITTER, COUNT(*) as vote_count +FROM MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123.VOTES +GROUP BY META_SUBMITTER; + +-- Find most popular movie each week +SELECT WEEK, MOVIE, COUNT(*) as votes +FROM MOVIES_ABC8D709C80262965344F5240AD123F5CBE51123.VOTES +GROUP BY WEEK, MOVIE +ORDER BY WEEK, votes DESC; +``` + +### Use Cases for META_SUBMITTER + +- Data provenance: Track who contributed each piece of data +- Reputation systems: Build contributor scores based on submission quality +- Spam filtering: Identify and filter submissions from problematic addresses +- Contribution tracking: Reward active contributors +- Data quality: Implement per-submitter quality metrics + +## PublicPermissionless vs Community Tables + +PublicPermissionless tables are best when you need open access with tracking: + +- Crowdsourced datasets +- Public feedback collection +- Open bug reports +- Community contributions +- Surveys and polls +- Any scenario where tracking submitters matters + +For controlled-access tables where only specific users can insert, see the Community Table documentation. + +## Next Steps + +- Learn about Community tables for controlled-access data +- Query your data using Proof of SQL +- Build reputation systems using META_SUBMITTER tracking +- Explore other table types and features on Space and Time diff --git a/tutorials/public_tutorial/public_create_table.js b/tutorials/public_tutorial/public_create_table.js new file mode 100644 index 0000000..29bb251 --- /dev/null +++ b/tutorials/public_tutorial/public_create_table.js @@ -0,0 +1,66 @@ +import "dotenv/config"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { Wallet } from "ethers"; +import { EthEcdsaSigner } from "../lib/ethecdsa_signer.js"; + +async function main() { + console.log("Connecting to RPC..."); + const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); + const api = await ApiPromise.create({ provider, noInitWarn: true }); + console.log("Connected to RPC."); + + // Get wallet address for namespace + const wallet = new Wallet(process.env.PRIVATE_KEY); + const ethAddress = wallet.address.substring(2).toUpperCase(); // Remove 0x and uppercase + const namespace = `MOVIES_${ethAddress}`; + + console.log("Creating namespace:", namespace); + + const createNamespaceTX = api.tx.tables.createNamespace( + namespace, + 0, + `CREATE SCHEMA IF NOT EXISTS ${namespace}`, + "PublicPermissionless", + { UserCreated: "MoviesPoll" }, + ); + + const createTablesTX = api.tx.tables.createTables([ + { + ident: { + namespace: namespace, + name: "VOTES", + }, + createStatement: + `CREATE TABLE IF NOT EXISTS ${namespace}.VOTES (` + + `MOVIE VARCHAR NOT NULL, ` + + `WEEK BIGINT NOT NULL, ` + + `PRIMARY_KEY(MOVIE, WEEK))`, + tableType: "PublicPermissionless", + commitment: { Empty: { hyperKzg: true } }, + source: { UserCreated: "MoviesPoll" }, + }, + ]); + + const batchTX = api.tx.utility.batchAll([createNamespaceTX, createTablesTX]); + + const signer = new EthEcdsaSigner(wallet, api); + console.log("Signing and sending transaction..."); + const unsub = await batchTX.signAndSend( + signer.address, + { signer }, + async (status) => { + if (status.isFinalized) { + console.log("Finalized in block", status.blockNumber.toString()); + console.log("Namespace:", namespace); + console.log("Table:", `${namespace}.VOTES`); + unsub(); + process.exit(0); + } + }, + ); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/tutorials/public_tutorial/public_insert_data.js b/tutorials/public_tutorial/public_insert_data.js new file mode 100644 index 0000000..bb90cd5 --- /dev/null +++ b/tutorials/public_tutorial/public_insert_data.js @@ -0,0 +1,71 @@ +import "dotenv/config"; +import { ApiPromise, WsProvider } from "@polkadot/api"; +import { u8aToHex } from "@polkadot/util"; +import { Wallet } from "ethers"; +import { EthEcdsaSigner } from "../lib/ethecdsa_signer.js"; +import { + Table, + Int64, + Utf8, + vectorFromArray, + tableToIPC, +} from "apache-arrow"; + +async function main() { + console.log("Connecting to RPC..."); + const provider = new WsProvider("wss://rpc.mainnet.sxt.network"); + const api = await ApiPromise.create({ provider, noInitWarn: true }); + console.log("Connected to RPC."); + + // Get wallet address for namespace + const wallet = new Wallet(process.env.PRIVATE_KEY); + const ethAddress = wallet.address.substring(2).toUpperCase(); + const namespace = `MOVIES_${ethAddress}`; + + console.log("Inserting data into:", `${namespace}.VOTES`); + + // IMPORTANT: Do NOT include META_SUBMITTER - it's added automatically! + const table = new Table({ + MOVIE: vectorFromArray( + ["Movie A", "Movie B", "Movie C"], + new Utf8() + ), + WEEK: vectorFromArray([2n, 2n, 2n], new Int64()) + // META_SUBMITTER will be automatically added by the chain + // containing your 32-byte AccountId + }); + + const batchId = 1; + const insertDataTx = api.tx.indexing.submitData( + { + namespace: namespace, + name: "VOTES", + }, + batchId, + u8aToHex(tableToIPC(table)), + ); + + const signer = new EthEcdsaSigner(wallet, api); + console.log("Signing and sending transaction..."); + + // Calculate META_SUBMITTER value for display + const paddedAddress = "0x" + "00".repeat(12) + ethAddress; + + const unsub = await insertDataTx.signAndSend( + signer.address, + { signer }, + async (status) => { + if (status.isFinalized) { + console.log("Finalized in block", status.blockNumber.toString()); + console.log("Table:", `${namespace}.VOTES`); + unsub(); + process.exit(0); + } + }, + ); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +});