Merge pull request #139 from rommapp/5.3-release-docs

docs: 5.3 release documentation

Author: gantoine

docs/Navigation.md

@@ -50,17 +50,22 @@ search:
     - [Collections](using/collections.md)
     - [Smart Collections](using/smart-collections.md)
     - [Virtual Collections](using/virtual-collections.md)
+    - [Physical Games](using/physical-games.md)
+    - [Recommendations](using/recommendations.md)
     - [Downloads](using/downloads.md)
     - [Uploads](using/uploads.md)
     - In-Browser Play
         - [EmulatorJS](using/in-browser-play/emulatorjs.md)
         - [`js-dos`](using/in-browser-play/js-dos.md)
         - [MS-DOS](using/in-browser-play/ms-dos.md)
+        - [PICO-8](using/in-browser-play/pico-8.md)
         - [Ruffle](using/in-browser-play/ruffle.md)
     - [Emulator Streaming](using/emulator-streaming.md)
         - [Migrating to webstation](using/emulator-streaming-migration.md)
+    - [Jukebox](using/jukebox.md)
     - [Saves & States](using/saves-and-states.md)
     - [RetroAchievements](using/retroachievements.md)
+    - [Walkthroughs](using/walkthroughs.md)
     - [ROM Patcher](using/rom-patcher.md)
     - [Netplay](using/netplay.md)
     - [Account & Profile](using/account-and-profile.md)

docs/administration/firmware-management.md

@@ -19,9 +19,14 @@ RomM does not ship games or firmware, and the team cannot help you obtain BIOS f
 ## Ingesting firmware
 
 1. Put the file in the right `bios/` folder (see [Folder Structure](../getting-started/folder-structure.md))
-2. Run a scan → Firmware is picked up alongside ROMs
-3. Navigate to the platform's gallery and click the `CPU` icon in the top left
-4. Firmware files will display at the bottom of the page
+2. Run a scan, and firmware is picked up alongside ROMs
+3. The platform's firmware is then listed on that platform's page, and offered to the in-browser player
+
+Where `bios/` lives is up to you. The default is `bios/{platform}` at the library root, and you can move it with the [`filesystem.structure.firmware`](../reference/configuration-file.md#filesystemstructure) template.
+
+## Missing firmware
+
+Delete a file from `bios/` and the next scan **flags it missing** instead of dropping it from the database. Put the file back and the next scan clears the flag. If you're never replacing it, the **Cleanup missing firmware** task deletes every flagged row in one go (see [Scheduled Tasks](scheduled-tasks.md#triggering-a-task-manually)).
 
 ## Platform-specific firmware
 

docs/administration/index.md

@@ -5,7 +5,7 @@ description: Running RomM for yourself and others.
 
 # Administration
 
-Administration is everything you do **as the operator** of a RomM instance: managing accounts, controlling access, configuring metadata sources, scheduling scans, watching the library for changes, monitoring the server, and keeping data safe.
+Administration is everything you do **as the server owner** of a RomM instance: managing accounts, controlling access, configuring metadata sources, scheduling scans, watching the library for changes, monitoring the server, and keeping data safe.
 
 The end-user equivalent (how to actually play the games, build collections, upload saves) lives in [Using RomM](../using/index.md).
 

docs/administration/oidc/index.md

@@ -134,6 +134,8 @@ environment:
     - OIDC_USERNAME_ATTRIBUTE=preferred_username
 ```
 
+Whatever that attribute holds gets sanitised before it becomes a username to prevent invalid characters from being used. RomM also sends a PKCE challenge on every authorization request, so it's safe to mark PKCE required for the client if your IdP offers that.
+
 ## Important notes
 
 - **Email must match** between OIDC and any existing local account, otherwise OIDC creates a new account alongside the old one.

docs/administration/scanning-and-watcher.md

@@ -20,7 +20,7 @@ Every scan picks one mode. Modes differ in what they touch, so use the most-targ
 | Mode              | What it does                                                               | When to use                                                                               |
 | ----------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
 | **New Platforms** | Only scans platform folders not already in the DB.                         | After mounting a new ROM set (very fast).                                                 |
-| **Quick**         | Skips files that already exist in the DB, with no metadata refresh.        | Default for scheduled runs and the watcher.                                               |
+| **Quick**         | Adds new games and catches file updates, with no metadata refresh.         | Default for scheduled runs and the watcher.                                               |
 | **Unmatched**     | Re-runs metadata matching against ROMs currently missing external IDs.     | After adding a new metadata provider, or when some titles didn't match on the first scan. |
 | **Update**        | Re-fetches metadata for all already-matched ROMs.                          | When metadata providers have meaningfully changed (e.g. IGDB restructured).               |
 | **Hashes**        | Recalculates CRC/MD5/SHA1 hashes.                                          | After upgrading from a version that didn't hash or when you suspect file corruption.      |
@@ -36,12 +36,14 @@ A manual scan can be scoped to specific platforms and a chosen subset of metadat
 
 Configured via env vars (full table in [Scheduled Tasks](scheduled-tasks.md)):
 
-| Variable                | Default     | Purpose                                                                                                      |
-| ----------------------- | ----------- | ------------------------------------------------------------------------------------------------------------ |
-| `SCHEDULED_RESCAN_CRON` | `0 0 * * *` | Cron expression for the scheduled library scan. Runs a **Quick** scan by default.                            |
-| `SCAN_TIMEOUT`          | `14400`     | Hard cap in seconds: scans that exceed this are killed and logged.                                           |
-| `SCAN_WORKERS`          | `1`         | Concurrent worker processes for scanning; leave as auto unless you're tuning.                                |
-| `SEVEN_ZIP_TIMEOUT`     | `60`        | Per-archive timeout in seconds for `.7z` extraction during scan; raise if scanning huge compressed ROM sets. |
+| Variable                | Default     | Purpose                                                                                                    |
+| ----------------------- | ----------- | ---------------------------------------------------------------------------------------------------------- |
+| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | Cron expression for the scheduled library scan, which runs a **Quick** scan                                |
+| `SCAN_TIMEOUT`          | `14400`     | Hard cap in seconds, after which the scan is killed and the clients watching it are told why               |
+| `SCAN_WORKERS`          | `4`         | How many ROMs a scan processes at once                                                                     |
+| `SEVEN_ZIP_TIMEOUT`     | `60`        | Per-archive timeout in seconds for `.7z` extraction during scan, raise it if scanning huge compressed sets |
+
+Scans get their own queue and worker now, so a long library scan won't hold up the shorter background tasks behind it.
 
 To disable scheduled scans entirely, either unset the cron or set it to something unreachable (`SCHEDULED_RESCAN_CRON=0 0 31 2 *`).
 
@@ -100,7 +102,19 @@ exclude:
                 extensions: [nfo]
 ```
 
-Full schema in [Configuration File](../reference/configuration-file.md).
+Whatever you list here is **added** to the defaults, not swapped in for them. The system folders and the frontend media folders stay excluded either way. Full schema in [Configuration File](../reference/configuration-file.md).
+
+## Platform folder names
+
+Every platform folder has to resolve to a [known slug](../platforms/supported-platforms.md). The [folder name aliases](../platforms/supported-platforms.md#folder-name-aliases) already cover what Batocera, RetroBat and ES-DE call things, so a library from one of those works mostly untouched. Anything else needs a [`system.platforms`](../reference/configuration-file.md#systemplatforms) mapping.
+
+## Title IDs read from the binary
+
+The following platforms stamp a **native Title ID** into the game binary, which scans can read: PSX, PS2, PS3, PSP, PS Vita, Switch, 3DS, Wii, Wii U, GameCube, Dreamcast, Xbox and Xbox 360.
+
+That ID identifies games on the platforms RomM doesn't hash, which is how a renamed or moved file holds on to its saves and collections, and marks the location where the game writes its saves, which is needed for device sync.
+
+To skip the whole thing, use [`filesystem.skip_title_id_extraction`](../reference/configuration-file.md#filesystemskip_title_id_extraction).
 
 ## Region and language preference
 

docs/administration/scheduled-tasks.md

@@ -25,19 +25,21 @@ Every scheduled task takes a standard 5-field cron expression:
 - `*/30 * * * *`: every 30 minutes
 - `0 2 * * 0`: 2 AM every Sunday
 
-Set the env var and restart the container; the scheduler picks up the new schedule the moment RomM comes back up.
+Set the env var and restart the container. The scheduler picks up the new schedule as soon as RomM is back.
 
 ## Enabling a scheduled task
 
-Most tasks have an `ENABLE_*` environment variable, like `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA=true` which enables the LaunchBox sync, and every one of them is off by default. Set both the enable var and its cron var, since a task with an empty cron string has nothing to schedule and stays unscheduled even when enabled.
+Most tasks have an `ENABLE_*` environment variable, like `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA=true` which enables the LaunchBox sync. Set both the enable var and its cron var, since a task with an empty cron string has nothing to schedule and stays unscheduled even when enabled.
+
+Unlike other tasks, **build recommendations index** ships enabled, because the [recommendation](../using/recommendations.md) sections read that index and similar games sits empty without it. Setting `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS=false` stops the nightly rebuild, but doesn't hide either section; users turn those off in their own settings.
 
 The housekeeping tasks (netplay cleanup, upload tmp cleanup, ZIP cache cleanup) are always on and have no env vars. Check the [env var reference](../reference/environment-variables.md) for the full list.
 
 ## Triggering a task manually
 
 ### From the Administration page
 
-**Administration → Tasks** shows every task with a "Run" button. Admins (anyone with `tasks.run` scope) can trigger:
+**Administration → Tasks** lists every task with its status and a way to run it. Anyone with the `tasks.run` scope can fire one off, scheduled tasks included, which saves waiting for the next cron tick after a config change.
 
 ### From the API
 
@@ -59,6 +61,7 @@ A task that's been "running" for hours is usually a scan that hit `SCAN_TIMEOUT`
 On a Raspberry Pi or NAS with 2 GB of RAM and/or a single CPU core:
 
 - Raise the cron intervals (daily → weekly) for the nightlies
-- Set `SCAN_WORKERS=1` to avoid concurrent scan processes
+- Set `SCAN_WORKERS=1` and `WEB_SERVER_CONCURRENCY=1`, both of which default to `4`
 - Enable the watcher but raise `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` to 30+ minutes
-- Disable image conversion if you don't care about WebP (`ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP=false`).
+- Disable image conversion if you don't care about WebP (`ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP=false`)
+- On a big library, set `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS=false` to skip the nightly build

docs/administration/server-stats.md

@@ -11,14 +11,14 @@ description: "The numbers Mason! What do they mean?"
 
 ### Top-line counts
 
-| Metric           | What it counts                                                                    |
-| ---------------- | --------------------------------------------------------------------------------- |
-| **Platforms**    | Every platform RomM has seen at least one ROM for. Deleted platforms don't count. |
-| **Games**        | Total ROM entries. A multi-file game (folder with multiple files) counts as 1.    |
-| **Saves**        | User save files across all users.                                                 |
-| **States**       | Emulator save states across all users.                                            |
-| **Screenshots**  | User-uploaded screenshots. Provider-fetched screenshots aren't counted here.      |
-| **Size on disk** | Total disk usage of all ROMs, saves, states, and screenshots.                     |
+| Metric           | What it counts                                                                            |
+| ---------------- | ----------------------------------------------------------------------------------------- |
+| **Platforms**    | Every platform RomM has seen at least one ROM for. Deleted platforms don't count.         |
+| **Games**        | Total ROM entries. Multi-file (folder with multiple files) and physical games count as 1. |
+| **Saves**        | User save files across all users.                                                         |
+| **States**       | Emulator save states across all users.                                                    |
+| **Screenshots**  | User-uploaded screenshots. Provider-fetched screenshots aren't counted here.              |
+| **Size on disk** | Total disk usage of all ROMs, saves, states, and screenshots.                             |
 
 ### Per-platform breakdown
 
@@ -29,8 +29,6 @@ Under the summary, it's a table sorted by name, size or game count. For each pla
 - Region distribution (how many games tagged USA, Japan, Europe, World, etc.)
 - Metadata coverage (how many games have metadata from each provider)
 
-When you want to know "which platform is eating my disk?" or "which platform has the worst match rate?"
-
 ## API
 
 The same data is available programmatically:

docs/developers/architecture.md

@@ -83,7 +83,7 @@ Environment variables (100+ of them, all listed in `env.template`) cover infrast
 
 ### Background jobs
 
-RQ workers run scheduled jobs (rescans, Switch TitleDB refresh, LaunchBox refresh, image-to-WebP conversion, RA progress sync, netplay cleanup) and manual tasks (`cleanup_missing_roms`, `cleanup_orphaned_resources`, `sync_folder_scan`). Each scheduled task is gated by an `ENABLE_SCHEDULED_*` env var and tunable via the matching `*_CRON`. Operator-side detail in [Scheduled Tasks](../administration/scheduled-tasks.md).
+RQ workers run scheduled jobs (rescans, Switch TitleDB refresh, LaunchBox refresh, image-to-WebP conversion, RA progress sync, netplay cleanup) and manual tasks (`cleanup_missing_roms`, `cleanup_orphaned_resources`, `sync_folder_scan`). Each scheduled task is gated by an `ENABLE_SCHEDULED_*` env var and tunable via the matching `*_CRON`. Server owner detail in [Scheduled Tasks](../administration/scheduled-tasks.md).
 
 ## Frontend
 

docs/developers/development-setup.md

@@ -7,6 +7,15 @@ description: Run RomM locally for development
 
 # Setting up RomM for development
 
+## Prerequisites
+
+| Tool    | Needed for                                                   |
+| ------- | ------------------------------------------------------------ |
+| Python  | 3.14 or newer, pinned in `.python-version`                   |
+| Node.js | 24, with npm 11.10 or newer, per `frontend/package.json`     |
+| uv      | Dependencies, and it fetches the Python in `.python-version` |
+| Docker  | For the database, Valkey, and the optional streaming stack   |
+
 ## Option 1: Using Docker
 
 If you prefer to use Docker for development, you can set up RomM using the provided Docker Compose configuration. This method simplifies the setup process by encapsulating all dependencies within Docker containers.
@@ -111,6 +120,15 @@ uv sync --all-extras --dev
 docker compose up -d
 ```
 
+Two optional stacks have their own compose files:
+
+```sh
+docker compose -f docker-compose.oidc.yml up -d      # Authentik, for testing OIDC
+docker compose -f docker-compose.streaming.yml up -d # webstation, for emulator streaming
+```
+
+The streaming image is amd64 only and runs to several GB, hence opt-in.
+
 #### Run the backend
 
 _Migrations will be run automatically when running the backend._
@@ -126,7 +144,6 @@ uv run python3 main.py
 
 ```sh
 cd frontend
-# npm version >= 9 needed
 npm install
 ```
 

docs/developers/index.md

@@ -5,7 +5,7 @@ description: Build on top of RomM or contribute to it
 
 # API & Development
 
-Everything you need to **build on top of RomM** (third-party apps, scripts, integrations) or **contribute to it** (code, translations, docs). End-user content lives in [Using RomM](../using/index.md). Operator content lives in [Administration](../administration/index.md).
+Everything you need to **build on top of RomM** (third-party apps, scripts, integrations) or **contribute to it** (code, translations, docs). End-user content lives in [Using RomM](../using/index.md). Server owner content lives in [Administration](../administration/index.md).
 
 ## Calling the API