FORLA: Federated Object-centric Representation Learning with Slot Attention
Guiqiu Liao,
Matjaž Jogan,
Eric Eaton,
Daniel Hashimoto
NeurIPS'25 |
GitHub |
arXiv |
Project page
Federated unsupervised feature adaptation
This is the official PyTorch implementation for paper: FORLA: Federated Object-centric Representation Learning with Slot Attention. The code contains:
- Unsupervised object-centric Slot attention (SA) model training with adaptation of a stack of foundation models (DINO,SAM,MAE,CLIP)
- The FORLA framework for Federated Object-centric learning
- 7 Realworld datasets (3 Surgical|4 Natural), establishing a large scale FL SA benchmark
Please refer to install.md for step-by-step guidance on how to install the packages.
Clone the repo
git clone https://github.com/PCASOlab/FORLA.git
cd FORLA
Please refer to data.md for dataset downloading and pre-processing.
Option 1
- use FL_launcher.sh (FL_launcher_w_env.sh if you are using virtual environment) to run server and clients all in once. Config FL_launcher.sh to add more clients (here is a example with 4 clients):
#!/bin/bash
# Federated Learning Tmux Setup Script
# Create server session
tmux new-session -d -s server -n "FL Server"
tmux send-keys -t server "python -m main.FL.FL_server" C-m
# Create client sessions
tmux new-session -d -s client1 -n "FL Client1"
tmux send-keys -t client1 "python -m main.FL.FL_c1" C-m
tmux new-session -d -s client2 -n "FL Client2"
tmux send-keys -t client2 "python -m main.FL.FL_c2" C-m
tmux new-session -d -s client3 -n "FL Client3"
tmux send-keys -t client3 "python -m main.FL.FL_c3" C-m
tmux new-session -d -s client4 -n "FL Client4"
tmux send-keys -t client4 "python -m main.FL.FL_c4" C-m
# Optional: Attach to server session by default
tmux attach-session -t server
- Change this parameter in main.FL.FL_server for aggreggating a specific number of local models (e.g., 4):
FED_MIN_CLIENTS = 4
- For each client code, for instace client1main.FL.FL_c1, set the data keyword to ultilize training config of a given data:
import os
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'train_miccai' # Change this to your target mode
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'eval_miccai' # Change this to your target mode
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'eval_pascal' # Change this to your target mode
os.environ['WORKING_DIR_IMPORT_MODE'] = 'train_pascal' # Change this to your target mode
4. sh FL_launcher.sh
See output like:
Not enough clients (0) for aggregation
Not enough clients (0) for aggregation
Not enough clients (0) for aggregation # no client is ready
Not enough clients (1) for aggregation # 1 clent is ready
Not enough clients (3) for aggregation # 3 clents are ready
Aggregated new global model v1
Aggregated new global model v2
Aggregated new global model v3
Aggregated new global model v4
.
.
Option 2
Start server & each clients within each python commander individually:
-
Follow option 1 to setup key parameters for server and clients.
-
Decompose FL_launcher.sh mannually to different command window: Use one window for server:
conda activate forla # optional if you use conda env
python -m main.FL.FL_server
Then use other multiple windows for clients:
conda activate forla # optional if you use conda env
python -m main.FL.FL_c[client_index]
- For each client code, for instace client1main.FL.FL_c1, set the keyword to ultilize evaluation config of a specific data:
import os
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'train_miccai' # Change this to your target mode
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'eval_miccai' # Change this to your target mode
os.environ['WORKING_DIR_IMPORT_MODE'] = 'eval_pascal' # Change this to your target mode
# os.environ['WORKING_DIR_IMPORT_MODE'] = 'train_pascal' # Change this to your target mode
2 run FL_launcher_eval.sh
tmux kill-server # if had other sessions running
sh FL_launcher_eval.sh
If you want to directly use pretrained models, they can be downloaded here: 7-domain FL model
@article{liao2025forla,
title = {FORLA: Federated Object-centric Representation Learning with Slot Attention},
author = {Liao, Guiqiu and Jogan, Matjaž and Eaton, Eric and Hashimoto, Daniel A.},
journal={NeurIPS},
year={2023}
}
FORLA is released under the PENN ACADEMIC SOFTWARE LICENSE. See the LICENSE file for more details.
If you have any questions about the code, please contact Guiqiu Liao liaoguiqiu@outlook.com