Skip to content

Conversation

@xlei77
Copy link

@xlei77 xlei77 commented Jan 28, 2026

This PR integrates Pixio, an image encoder, into MapAnything to enable benchmarking against other SOTA encoders (e.g., DINO series).

Changes

  1. Model Definitions & Wrapper — Added Pixio model definitions and a wrapper to integrate it with the MapAnything architecture.
  2. Configuration — Added YAML configs for the Pixio ViT-H/16 checkpoint.
  3. Benchmark Script — Added a script to run performance benchmarks on Pixio.
  4. Lazy Instantiation Support — Modified model.py to support lazy instantiation of encoder classes from /mapanything/models/external/.

Testing

Manual validation was performed by running benchmark scripts on an H100 node and running local demos.

Benchmark:

bash bash_scripts/benchmark/dense_2_view/pixio.sh <CHECKPOINT_PATH> <OUTPUT_DIR>

Local Demo:

python scripts/demo_local_weight.py \
    --image_folder <INPUT_IMAGES_DIR> \
    --local_config '{
        "path": "configs/train.yaml",
        "model_str": "pixio",
        "checkpoint_path": "<CHECKPOINT_PATH>",
        "config_overrides": [
            "machine=aws",
            "model=pixio",
            "model/task=images_only",
            "model.encoder.uses_torch_hub=false"
        ],
        "trained_with_amp": true,
        "trained_with_amp_dtype": "bf16",
        "data_norm_type": "dinov2",
        "patch_size": 16,
        "resolution": 512,
        "strict": false
    }' \
    --output_path <OUTPUT_FILE>

Note: A README with detailed instructions for running benchmarks and demos will be added in a follow-up.

Performance

See benchmark results here: Pixio Performance

Dependencies

None.

Breaking Changes

None.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jan 28, 2026
@xlei77 xlei77 changed the title Add pixio as external encoder Add Pixio as external encoder Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant