feat: allow unknown models to be installed #8582
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…, simplify model manager models list
I can't remember what this was for and don't see any reference to it. Maybe it's just remnants from a previous implementation?
psychedelicious
requested review from
blessedcoolant,
maryhipp,
hipsterusername and
lstein
as code owners
github-actions
bot
added
python
psychedelicious
force-pushed
the
psyche/feat/mm/unknown-model
branch
from
6bf6966 to
6c2da8c
Compare
psychedelicious
force-pushed
the
psyche/feat/mm/unknown-model
branch
from
6c2da8c to
4070f26
Compare
|
This PR also does a bit of cleanup in the frontend for models:
|
hipsterusername
approved these changes
Sep 19, 2025
psychedelicious
added a commit
that referenced
this pull request
Sep 24, 2025
Previously, we had a multi-phase strategy to identify models from their files on disk: 1. Run each model config classes' `matches()` method on the files. It checks if the model could possibly be an identified as the candidate model type. This was intended to be a quick check. Break on the first match. 2. If we have a match, run the config class's `parse()` method. It derive some additional model config attrs from the model files. This was intended to encapsulate heavier operations that may require loading the model into memory. 3. Derive the common model config attrs, like name, description, calculate the hash, etc. Some of these are also heavier operations. This strategy has some issues: - It is not clear how the pieces fit together. There is some back-and-forth between different methods and the config base class. It is hard to trace the flow of logic until you fully wrap your head around the system and therefore difficult to add a model architecture to the probe. - The assumption that we could do quick, lightweight checks before heavier checks is incorrect. We often _must_ load the model state dict in the `matches()` method. So there is no practical perf benefit to splitting up the responsibility of `matches()` and `parse()`. - Sometimes we need to do the same checks in `matches()` and `parse()`. In these cases, splitting the logic is has a negative perf impact because we are doing the same work twice. - As we introduce the concept of an "unknown" model config (i.e. a model that we cannot identify, but still record in the db; see #8582), we will _always_ run _all_ the checks for every model. Therefore we need not try to defer heavier checks or resource-intensive ops like hashing. We are going to do them anyways. - There are situations where a model may match multiple configs. One known case are SD pipeline models with merged LoRAs. In the old probe API, we relied on the implicit order of checks to know that if a model matched for pipeline _and_ LoRA, we prefer the pipeline match. But, in the new API, we do not have this implicit ordering of checks. To resolve this in a resilient way, we need to get all matches up front, then use tie-breaker logic to figure out which should win (or add "differential diagnosis" logic to the matchers). - Field overrides weren't handled well by this strategy. They were only applied at the very end, if a model matched successfully. This means we cannot tell the system "Hey, this model is type X with base Y. Trust me bro.". We cannot override the match logic. As we move towards letting users correct mis-identified models (see #8582), this is a requirement. We can simplify the process significantly and better support "unknown" models. Firstly, model config classes now have a single `from_model_on_disk()` method that attempts to construct an instance of the class from the model files. This replaces the `matches()` and `parse()` methods. If we fail to create the config instance, a special exception is raised that indicates why we think the files cannot be identified as the given model config class. Next, the flow for model identification is a bit simpler: - Derive all the common fields up-front (name, desc, hash, etc). - Merge in overrides. - Call `from_model_on_disk()` for every config class, passing in the fields. Overrides are handled in this method. - Record the results for each config class and choose the best one. The identification logic is a bit more verbose, with the special exceptions and handling of overrides, but it is very clear what is happening. The one downside I can think of for this strategy is we do need to check every model type, instead of stopping at the first match. It's a bit less efficient. In practice, however, this isn't a hot code path, and the improved clarity is worth far more than perf optimizations that the end user will likely never notice.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Our model installation process is often a pain point for users. We require models to be identified during installation, else we error and do not install them. There are a few core problems with this approach:
This PR loosens the model install process a bit and cleans up some of the related model manager UI:
Unknownto the model base, type and format enums.UnknownModelConfigas a fallback class. When we fail to identify a model, instead of raising and error and deleting the candidate model files, we go ahead and install it, but set its base, type and format to unknown.allow_unknown_modelssetting to feature-flag the new fallback-to-unknown behaviour. It is enabled by default. To opt out of the new behaviour, set it toFalse.QA Instructions
Try installing a model that isn't supported by Invoke's model manager. You could use this one: https://huggingface.co/facebook/sam2-hiera-tiny/resolve/main/model.safetensors
It's the "tiny" variant of SAM 2, weighs ~150MB. You should get the toast and be able to edit the model.
Merge Plan
n/a
Checklist
What's Newcopy (if doing a release after this PR)