`@ubiquity-os/text-conversation-rewards`

How to Get Started

Install dependencies: Make sure you have Bun installed, then run:
```
bun install
```
Build the UI for production:
```
cd src/web && bun run ui:build
```
Copy and paste the .env.example and populate the environment variables
Start the server:
```
bun run server
```

Technical Architecture

This plugin revolutionizes open source collaboration by implementing an AI-powered reward mechanism for quality contributions.

Core Components

Content Evaluation Engine

At the heart of the system is a content evaluation module that assigns monetary value to contributor comments in the context of work projects with specifications. Here's how it works:

The system processes both issue comments and pull request review comments through different evaluation pipelines, with comprehensive preprocessing that:
- Removes user commands (starting with /) and bot responses
- Filters out quoted text (starting with >)
- Removes HTML comments and footnotes
- For assigned users, considers comment timestamps to optionally exclude those posted during assignment periods, to reduce gaming
- Processes linked pull request comments through GraphQL API
- Handles minimized/hidden comments
- Credits only unique links to prevent duplicates
For issue comments, it generates a context-aware prompt that includes:
- The original issue description and specification
- All comments in the conversation for context
- The specific comments being evaluated
The evaluation process handles GitHub-flavored markdown intelligently:
- It distinguishes between quoted text (starting with '>') and original content
- Only evaluates the commenter's original contributions
- Considers the relationship between comments and their context

The language model assigns relevance scores from 0 to 1:

interface Relevances {
  [commentId: string]: number; // 0 = irrelevant, 1 = highly relevant
}

Review Incentivization System

The review incentivization module implements a sophisticated algorithm for rewarding code reviews:

interface ReviewScore {
  reviewId: number;
  effect: {
    addition: number;
    deletion: number;
  };
  reward: number;
  priority: number;
}

The system calculates rewards based on:

The scope of code reviewed (additions + deletions)
Issue priority labels
The conclusiveness of the review (APPROVED or CHANGES_REQUESTED states receive additional credit)
File-specific exclusions through pattern matching

Permit Generation and Reward Distribution

The permit generation module handles the secure distribution of rewards:

Security Checks:
- Validates that the issue is collaborative
- Verifies private key permissions against organization and repository IDs
- Implements a multi-format encryption system for private keys
Fee Processing:
- Automatically calculates and deducts platform fees
- Supports token-specific fee exemptions through whitelist
- Creates treasury allocations for fee distribution
Reward Distribution:
- Generates ERC20 token permits for each contributor
- Stores permit data securely in a Supabase database
- Creates claimable reward URLs in the format: https://pay.ubq.fi?claim=[encoded_permit]

Technical Implementation Details

Token Management

The system uses decimal.js for precise token calculations:

const feeRateDecimal = new Decimal(100).minus(env.PERMIT_FEE_RATE).div(100);
const totalAfterFee = new Decimal(rewardResult.total).mul(feeRateDecimal).toNumber();

Smart Token Handling

For large conversations, the system implements intelligent token management:

// Dynamically handles token limits and chunking for large conversations
_calculateMaxTokens(prompt: string, totalTokenLimit: number = 16384) {
  // Token limit is configurable and adjusts based on model and rate limits
  const inputTokens = this.tokenizer.encode(prompt).length;
  const limit = Math.min(this._configuration?.tokenCountLimit, this._rateLimit);
  return Math.min(inputTokens, limit);
}

// Splits large conversations into manageable chunks
async _splitPromptForEvaluation(specification: string, comments: Comment[]) {
  let chunks = 2;
  while (this._exceedsTokenLimit(comments, chunks)) {
    chunks++;
  }
  return this._processChunks(specification, comments, chunks);
}

Database Integration

The system maintains a comprehensive record of all permits and rewards:

interface PermitRecord {
  amount: string;
  nonce: string;
  deadline: string;
  signature: string;
  beneficiary_id: number;
  location_id: number;
}

Data structure

{
  "userName": {
    "comments": [
      {
        "content": "comment content",
        "url": "https://url-to-item",
        "type": 18,
        "score": {
          "formatting": {
            "content": {
              "p": {
                "count": 16,
                "score": 1
              }
            },
            "wordValue": 0.1,
            "multiplier": 1
          },
          "reward": 0.8,
          "relevance": 0.5
        }
      }
    ],
    "total": 40.5,
    "task": {
      "reward": 37.5,
      "multiplier": 1
    },
    "feeRate": 0.1,
    "permitUrl": "https://example.com/permit",
    "payoutMode": "permit",
    "userId": 123,
    "evaluationCommentHtml": "<p>Evaluation comment</p>"
  }
}

Reward formula:

$$\sum_{i=0}^{n} \left( \sum_{j=0}^{n} \left(\text{wordCount}^{exponent} \times \text{wordValue} \times \text{relevance}\right) + \left(\text{score} \times \text{elementCount}\right) \right) \times multiplier + \text{task.reward} = \text{total}$$

Plugin configuration

Here is a possible valid configuration to enable this plugin. See these files for more details.

Note: LLM requests use the UOS_AI_URL environment variable for the API base URL. Model IDs and reasoning settings are not configured in the plugin config; only limits and retries are set here.

plugin: ubiquity-os/conversation-rewards
with:
  dataCollection:
    maxAttempts: 10
    delayMs: 10000
  rewards:
    evmNetworkId: 100
    evmPrivateEncrypted: "encrypted-key"
    erc20RewardToken: "0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d"
  incentives:
    closeTaskReward:
      rewardAmount: 5
      stalenessDuration: "30 days"
    specialUsers:
      - ["Copilot", "copilot-swe-agent"]
    requirePriceLabel: true
    limitRewards: true
    collaboratorOnlyPaymentInvocation: true
    externalContent:
      llmImageModel:
        maxRetries: 5
      llmWebsiteModel:
        maxRetries: 5
    contentEvaluator:
      openAi:
        tokenCountLimit: 124000 # Adjustable token limit
        maxRetries: 5 # Number of retries for rate limits/errors
      multipliers:
        - role: [ISSUE_SPECIFICATION]
          relevance: 1
        - role: [PULL_AUTHOR]
          relevance: 1
        - role: [PULL_ASSIGNEE]
          relevance: 1
        - role: [PULL_COLLABORATOR]
          relevance: 1
        - role: [PULL_CONTRIBUTOR]
          relevance: 1
      originalAuthorWeight: 0.5
    userExtractor:
      redeemTask: true
    dataPurge:
      skipCommentsWhileAssigned: all
    reviewIncentivizer:
      baseRate: 100
    formattingEvaluator:
      wordCountExponent: 0.85
      multipliers:
        - role: ["ISSUE_SPECIFICATION"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
        - role: ["ISSUE_AUTHOR"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.2
        - role: ["ISSUE_ASSIGNEE"]
          multiplier: 0
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0
        - role: ["ISSUE_COLLABORATOR"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
        - role: ["ISSUE_CONTRIBUTOR"]
          multiplier: 0.25
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
        - role: ["PULL_SPECIFICATION"]
          multiplier: 0
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0
        - role: ["PULL_AUTHOR"]
          multiplier: 2
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.2
        - role: ["PULL_ASSIGNEE"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
        - role: ["PULL_COLLABORATOR"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
        - role: ["PULL_CONTRIBUTOR"]
          multiplier: 0.25
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              p: { score: 0, countWords: true }
              em: { score: 0, countWords: true }
              img: { score: 5, countWords: true }
              strong: { score: 0, countWords: false }
              blockquote: { score: 0, countWords: false }
              h1: { score: 1, countWords: true }
              h2: { score: 1, countWords: true }
              h3: { score: 1, countWords: true }
              h4: { score: 1, countWords: true }
              h5: { score: 1, countWords: true }
              h6: { score: 1, countWords: true }
              a: { score: 5, countWords: true }
              li: { score: 0.5, countWords: true }
              ul: { score: 1, countWords: true }
              td: { score: 0, countWords: true }
              hr: { score: 0, countWords: true }
              pre: { score: 0, countWords: false }
              ol: { score: 1, countWords: true }
            wordValue: 0.1
      payment:
        autmaticTransferMode: false
      githubComment:
        post: true
        debug: false

Permit2 allowance UI helper

https://permit2-allowance.ubq.fi

How to encrypt the `evmPrivateEncrypted` parameter

Partner private key (evmPrivateEncrypted config param in conversation-rewards plugin) supports 2 formats:

PRIVATE_KEY:GITHUB_OWNER_ID
PRIVATE_KEY:GITHUB_OWNER_ID:GITHUB_REPOSITORY_ID

Here GITHUB_OWNER_ID can be:

GitHub organization id (if ubiquity-os is used within an organization)
GitHub user id (if ubiquity-os is simply installed in a user's repository)

Format PRIVATE_KEY:GITHUB_OWNER_ID restricts in which particular organization (or user related repositories) this private key can be used. It can be set either in the organization wide config either in the repository wide one.

Format PRIVATE_KEY:GITHUB_OWNER_ID:GITHUB_REPOSITORY_ID restricts organization (or user related repositories) and a particular repository where private key is allowed to be used.

How to encrypt for you local organization for testing purposes:

Get your organization (or user) id

curl -H "Accept: application/json" -H "Authorization: token GITHUB_PAT_TOKEN" https://api.github.com/orgs/ubiquity

Open https://keygen.ubq.fi/
Click "Generate" to create a new x25519_PRIVATE_KEY (which will be used in the conversation-rewards plugin to decrypt encrypted wallet private key)
Input a string in the format PRIVATE_KEY:GITHUB_OWNER_ID in the PLAIN_TEXT UI text input where:

PRIVATE_KEY: your ethereum wallet private key without the 0x prefix
GITHUB_OWNER_ID: your github organization id or user id (which you got from step 1)

Click "Encrypt" to get an encrypted value in the CIPHER_TEXT field
Set the encrypted text (from step 5) in the evmPrivateEncrypted config parameter
Set X25519_PRIVATE_KEY environment variable in github secrets of your forked instance of the conversation-rewards plugin

Configuring Reward Fees

To enable and configure the automatic deduction of fees from generated rewards, you need to set specific environment variables:

PERMIT_FEE_RATE:
- Purpose: Defines the percentage of the reward to be deducted as a fee.
- Format: A number representing the percentage (e.g., 5 for 5%, 2.5 for 2.5%).
- Behavior: If this variable is not set, is set to 0, or is not a valid number, the fee deduction process will be skipped entirely.
PERMIT_TREASURY_GITHUB_USERNAME:
- Purpose: Specifies the GitHub username of the account designated to receive the accumulated fees. The system uses this username to fetch the corresponding GitHub user ID for internal accounting.
- Format: A valid GitHub username (e.g., ubiquity-os-treasury).
- Behavior: If this variable is not set or the username does not correspond to a valid GitHub user, the fee deduction process will be skipped.
PERMIT_ERC20_TOKENS_NO_FEE_WHITELIST (Optional):
- Purpose: Allows specific ERC20 tokens to be exempt from fee deductions.
- Format: A comma-separated string of ERC20 token addresses (e.g., 0xTokenAddress1,0xTokenAddress2). Ensure addresses are in the correct checksum format if applicable, although the check is case-insensitive.
- Behavior: If the erc20RewardToken configured for the current run matches any address in this whitelist, fees will not be applied for that specific reward calculation, even if PERMIT_FEE_RATE and PERMIT_TREASURY_GITHUB_USERNAME are set.

Example Setup:

PERMIT_FEE_RATE=5
PERMIT_TREASURY_GITHUB_USERNAME=ubiquity-os-treasury
PERMIT_ERC20_TOKENS_NO_FEE_WHITELIST="0xSomeTokenAddress,0xAnotherTokenAddress"

Name		Name	Last commit message	Last commit date
Latest commit History 4,122 Commits
.github		.github
.husky		.husky
.idea		.idea
scripts		scripts
src		src
supabase/migrations		supabase/migrations
tests		tests
.babelrc		.babelrc
.cspell.json		.cspell.json
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.nvmrc		.nvmrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
.yarnrc.yml		.yarnrc.yml
CHANGELOG.md		CHANGELOG.md
README.md		README.md
action.yml		action.yml
bun.lock		bun.lock
eslint.config.mjs		eslint.config.mjs
graphql.config.yml		graphql.config.yml
jest.config.ts		jest.config.ts
knip.ts		knip.ts
package.json		package.json
tsconfig.json		tsconfig.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

`@ubiquity-os/text-conversation-rewards`

How to Get Started

Technical Architecture

Core Components

Content Evaluation Engine

Review Incentivization System

Permit Generation and Reward Distribution

Technical Implementation Details

Token Management

Smart Token Handling

Database Integration

Data structure

Plugin configuration

Permit2 allowance UI helper

How to encrypt the `evmPrivateEncrypted` parameter

Configuring Reward Fees

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

@ubiquity-os/text-conversation-rewards

How to Get Started

Technical Architecture

Core Components

Content Evaluation Engine

Review Incentivization System

Permit Generation and Reward Distribution

Technical Implementation Details

Token Management

Smart Token Handling

Database Integration

Data structure

Plugin configuration

Permit2 allowance UI helper

How to encrypt the evmPrivateEncrypted parameter

Configuring Reward Fees

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`@ubiquity-os/text-conversation-rewards`

How to encrypt the `evmPrivateEncrypted` parameter

Packages