A unified platform for building, evaluating, and applying reward models.

Documentation | Examples | 中文

News

• 2025-10-20 - Auto-Rubric: Learning to Extract Generalizable Criteria for Reward Modeling - We released a new paper on learning generalizable reward criteria for robust modeling.
• 2025-10-17 - Taming the Judge: Deconflicting AI Feedback for Stable Reinforcement Learning - We introduced techniques to align judge feedback and improve RL stability.
• 2025-07-09 - Released RM-Gallery v0.1.0 on PyPI

Installation

RM-Gallery requires Python 3.10 or higher (< 3.13).

pip install rm-gallery

Or install from source:

git clone https://github.com/modelscope/RM-Gallery.git
cd RM-Gallery
pip install .

Quick Start

from rm_gallery.core.reward.registry import RewardRegistry
from rm_gallery.core.data.schema import DataSample

# Choose from 35+ pre-built reward models
rm = RewardRegistry.get("safety_listwise_reward")

# Evaluate your data
sample = DataSample(...)
result = rm.evaluate(sample)

See the quickstart guide for a complete example, or try our interactive notebooks.

Features

Pre-built Reward Models

Access 35+ reward models for different domains:

rm = RewardRegistry.get("math_correctness_reward")
rm = RewardRegistry.get("code_quality_reward")
rm = RewardRegistry.get("helpfulness_listwise_reward")

View all reward models

Custom Reward Models

Build your own reward models with simple APIs:

from rm_gallery.core.reward import BasePointWiseReward

class CustomReward(BasePointWiseReward):
    def _evaluate(self, sample, **kwargs):
        # Your evaluation logic
        return RewardResult(...)

Learn more about building custom RMs

Benchmarking

Evaluate models on standard benchmarks:

• RewardBench2 - Latest reward model benchmark
• RM-Bench - Comprehensive evaluation suite
• Conflict Detector - Detect evaluation inconsistencies
• JudgeBench - Judge capability assessment

Read the evaluation guide

Applications

• Best-of-N Selection - Choose optimal responses from candidates
• Data Refinement - Improve dataset quality with reward signals
• RLHF Integration - Use rewards in reinforcement learning pipelines
• High-Performance Serving - Deploy models with fault-tolerant infrastructure

Documentation

• Quickstart Guide
• Interactive Examples
• Building Custom RMs
• Training Guide
• API Reference

Contributing

We welcome contributions! Please install pre-commit hooks before submitting pull requests:

pip install -e .
pre-commit install

See our contribution guide for details.

Citation

If you use RM-Gallery in your research, please cite:

@software{
title = {RM-Gallery: A One-Stop Reward Model Platform},
author = {The RM-Gallery Team},
url = {https://github.com/modelscope/RM-Gallery},
month = {07},
year = {2025}
}