Assistant: add command to validate configuredModels setting #10702
Description
Proposal
Add a command that validates all configured models by attempting to connect to each one. This would help administrators and support teams diagnose model configuration issues.
- Maybe we want to include this as part of the PA diagnostics recently added (command:
Positron Assistant: Collect Diagnostics)
Background
Discussion originated from PR #10611 where we identified the need for a way to validate model configurations without relying on the existing connection attempt mechanism during extension activation.
Proposed Solution
- Command: Check configured models (or similar)
- Functionality: Iterate through all models in the configured models setting and attempt a connection check for each
- Prerequisites: Users must already be authenticated to the model providers
Benefits
- Helps administrators validate model configurations
- Provides support teams with a diagnostic tool for model connection issues
- Avoids delays during extension activation (compared to checking all models automatically as part of activation)
Notes
- Currently
positron.assistant.maxConnectionAttempts(default: 3) provides a workaround by trying multiple models and logging failures in the output log - This command would provide a more deliberate and comprehensive validation approach