diff --git a/docs/source/architecture.md b/docs/source/architecture.md deleted file mode 100644 index 1fe508bc..00000000 --- a/docs/source/architecture.md +++ /dev/null @@ -1,146 +0,0 @@ -# Architecture Diagram - -## New Data Flow - -![New Data Flow](_static/mcp_data_flow.png) - -## Component Interaction - -![Component Interaction Sequence](_static/component_interaction.png) - -## Data Adapter Pattern - -![Data Adapter Pattern](_static/data_adapter_pattern.png) - -## Data Validation Flow - -``` -Data Source - │ - ▼ -adapter.load() - │ - ├─> Set time index - ├─> Sort by time - ├─> Infer/set frequency - │ - ▼ -adapter.validate() - │ - ├─> Check DatetimeIndex ✓ - ├─> Check for duplicates ✓ - ├─> Check missing values ⚠ - ├─> Check monotonic ⚠ - ├─> Check frequency ⚠ - ├─> Check data size ⚠ - │ - ▼ -Validation Report - { - "valid": true/false, - "errors": [...], # Critical issues - "warnings": [...] # Non-critical issues - } - │ - ▼ -adapter.to_sktime_format() - │ - ├─> Extract target (y) - ├─> Extract exogenous (X) - │ - ▼ -(y, X) ready for sktime -``` - -## Handle Management - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Executor._data_handles │ -│ { │ -│ "data_abc123": { │ -│ "y": Series(...), │ -│ "X": DataFrame(...), │ -│ "metadata": {...}, │ -│ "validation": {...}, │ -│ "config": {...} │ -│ }, │ -│ "data_xyz789": {...} │ -│ } │ -└─────────────────────────────────────────────────────────────────┘ - │ │ - │ fit_predict(data_handle=…) │ release_data_handle() - │ │ - ▼ ▼ - Retrieve data Delete handle - Fit estimator Free memory - Generate predictions -``` - -## Complete Workflow Example - -``` -1. Load Data - ┌──────────────────────────────────────────────────────────┐ - │ load_data_source({ │ - │ "type": "sql", │ - │ "connection_string": "postgresql://...", │ - │ "query": "SELECT * FROM sales", │ - │ "time_column": "date", │ - │ "target_column": "revenue" │ - │ }) │ - └────────────────────┬─────────────────────────────────────┘ - │ - ▼ - ┌──────────────────────────────────────────────────────────┐ - │ Returns: │ - │ { │ - │ "success": true, │ - │ "data_handle": "data_abc123", │ - │ "metadata": { │ - │ "rows": 1000, │ - │ "columns": ["revenue", "temperature"], │ - │ "frequency": "D", │ - │ "start_date": "2020-01-01", │ - │ "end_date": "2022-09-27" │ - │ }, │ - │ "validation": { │ - │ "valid": true, │ - │ "warnings": [] │ - │ } │ - │ } │ - └────────────────────┬─────────────────────────────────────┘ - │ -2. Instantiate Model │ - ┌──────────────────────────────────────────────────────────┐ - │ instantiate_estimator("ARIMA", {"order": [1,1,1]}) │ - └────────────────────┬─────────────────────────────────────┘ - │ - ▼ - ┌──────────────────────────────────────────────────────────┐ - │ Returns: {"handle": "est_xyz789"} │ - └────────────────────┬─────────────────────────────────────┘ - │ -3. Fit & Predict │ - ┌──────────────────────────────────────────────────────────┐ - │ fit_predict( … data_handle= … ) │ - │ estimator_handle="est_xyz789", │ - │ data_handle="data_abc123", │ - │ horizon=7 │ - │ ) │ - └────────────────────┬─────────────────────────────────────┘ - │ - ▼ - ┌──────────────────────────────────────────────────────────┐ - │ Returns: │ - │ { │ - │ "success": true, │ - │ "predictions": { │ - │ "2022-09-28": 1250.5, │ - │ "2022-09-29": 1255.2, │ - │ ... │ - │ }, │ - │ "horizon": 7 │ - │ } │ - └──────────────────────────────────────────────────────────┘ -``` diff --git a/docs/source/background-jobs.md b/docs/source/background-jobs.md deleted file mode 100644 index cbbbb17c..00000000 --- a/docs/source/background-jobs.md +++ /dev/null @@ -1,407 +0,0 @@ -# Background Job Management - -## Overview - -The sktime-mcp server now supports **non-blocking background jobs** for long-running operations like model training. This architecture prevents the MCP server from becoming unresponsive when training large models. - -## Architecture - -The system uses a multi-threaded, asynchronous architecture inspired by CodeGraphContext: - -### Key Components - -1. **JobManager** (`runtime/jobs.py`) - Thread-safe job tracking -2. **Executor** (`runtime/executor.py`) - Async execution with job tracking -3. **MCP Server** (`server.py`) - Event loop coordination - -## How It Works - -### 1. Non-Blocking Execution - -When you call `fit_predict_async`, the server: - -1. Creates a job with a unique ID -2. Schedules the training as an async coroutine on the event loop -3. **Returns immediately** with the job ID -4. Continues handling other requests while training runs in the background - -```python -# Traditional (blocking) - Server freezes during training -result = fit_predict(handle, "airline", horizon=12) - -# Async (non-blocking) - Server stays responsive -job_info = fit_predict_async(handle, "airline", horizon=12) -# Returns: {"success": True, "job_id": "abc-123-def"} - -# Check progress -status = check_job_status("abc-123-def") -# Returns: {"status": "running", "progress_percentage": 66.7, ...} -``` - -### 2. Progress Tracking - -Jobs track their progress through multiple steps: - -```python -{ - "job_id": "abc-123-def", - "status": "running", - "total_steps": 3, - "completed_steps": 2, - "current_step": "Generating predictions (horizon=12)...", - "progress_percentage": 66.7, - "estimated_time_remaining": 5.2, - "estimated_time_remaining_human": "5s" -} -``` - -### 3. Job Lifecycle - -``` -PENDING → RUNNING → COMPLETED - ↘ FAILED - ↘ CANCELLED -``` - -## Available Tools - -### Training Tools - -#### `fit_predict_async` - -Train a model in the background (non-blocking). - -**Arguments:** -- `estimator_handle`: Handle from `instantiate_estimator` -- `dataset`: Dataset name (e.g., "airline", "sunspots") -- `horizon`: Forecast horizon (default: 12) - -**Returns:** -```json -{ - "success": true, - "job_id": "abc-123-def-456", - "message": "Training job started for ARIMA on airline...", - "estimator": "ARIMA", - "dataset": "airline", - "horizon": 12 -} -``` - -**Example:** -```python -# Instantiate model -model = instantiate_estimator("ARIMA", {"order": [1, 1, 1]}) -# Returns: {"handle": "est_abc123"} - -# Start training (non-blocking!) -job = fit_predict_async("est_abc123", "airline", horizon=24) -# Returns immediately with job_id - -# Server is still responsive - you can do other things! -``` - -### Job Management Tools - -#### `check_job_status` - -Check the status and progress of a background job. - -**Arguments:** -- `job_id`: Job ID to check - -**Returns:** -```json -{ - "success": true, - "job_id": "abc-123-def", - "status": "running", - "job_type": "fit_predict", - "estimator_name": "ARIMA", - "dataset_name": "airline", - "total_steps": 3, - "completed_steps": 2, - "current_step": "Generating predictions...", - "progress_percentage": 66.7, - "elapsed_time": 10.5, - "estimated_time_remaining": 5.2, - "estimated_time_remaining_human": "5s", - "result": null -} -``` - -When completed: -```json -{ - "status": "completed", - "result": { - "success": true, - "predictions": {...}, - "horizon": 12 - } -} -``` - -#### `list_jobs` - -List all background jobs with optional filtering. - -**Arguments:** -- `status` (optional): Filter by status ("pending", "running", "completed", "failed", "cancelled") -- `limit` (optional): Maximum number of jobs to return (default: 20) - -**Returns:** -```json -{ - "success": true, - "count": 3, - "jobs": [ - { - "job_id": "abc-123", - "status": "completed", - "estimator_name": "ARIMA", - ... - }, - { - "job_id": "def-456", - "status": "running", - "progress_percentage": 50.0, - ... - } - ] -} -``` - -**Example:** -```python -# List all running jobs -running_jobs = list_jobs(status="running") - -# List all jobs (any status) -all_jobs = list_jobs(limit=50) -``` - -#### `cancel_job` - -Cancel a running or pending job. - -**Arguments:** -- `job_id`: Job ID to cancel - -**Returns:** -```json -{ - "success": true, - "message": "Job 'abc-123-def' cancelled" -} -``` - -**Note:** You can only cancel jobs in "pending" or "running" status. - -#### `delete_job` - -Delete a job from the job manager. - -**Arguments:** -- `job_id`: Job ID to delete - -**Returns:** -```json -{ - "success": true, - "message": "Job 'abc-123-def' deleted" -} -``` - -#### `cleanup_old_jobs` - -Remove jobs older than specified hours. - -**Arguments:** -- `max_age_hours` (optional): Maximum age in hours (default: 24) - -**Returns:** -```json -{ - "success": true, - "message": "Removed 5 old job(s)", - "count": 5 -} -``` - -## Complete Workflow Example - -```python -# 1. Instantiate a model -model_result = instantiate_estimator("AutoARIMA") -handle = model_result["handle"] - -# 2. Start training in background -job_result = fit_predict_async(handle, "airline", horizon=24) -job_id = job_result["job_id"] - -# 3. Server is responsive - you can do other things! -# Check available demo datasets -datasets = list_available_data(is_demo=True) - -# Search for other estimators -estimators = list_estimators(query="prophet") - -# 4. Check training progress -status = check_job_status(job_id) -print(f"Progress: {status['progress_percentage']}%") -print(f"Current step: {status['current_step']}") -print(f"Time remaining: {status['estimated_time_remaining_human']}") - -# 5. Wait for completion (or poll periodically) -while status["status"] == "running": - time.sleep(1) - status = check_job_status(job_id) - -# 6. Get results -if status["status"] == "completed": - predictions = status["result"]["predictions"] - print(f"Predictions: {predictions}") -else: - print(f"Job failed: {status['errors']}") - -# 7. Clean up -delete_job(job_id) -``` - -## Technical Details - -### Async Execution - -The `fit_predict_async` method uses: - -1. **`asyncio.run_in_executor()`** - Runs synchronous sktime operations in a thread pool -2. **`await asyncio.sleep(0.01)`** - Yields control to the event loop every 10ms -3. **`asyncio.run_coroutine_threadsafe()`** - Schedules coroutines on the main event loop - -This prevents blocking the MCP server's event loop. - -### Thread Safety - -The `JobManager` uses `threading.Lock()` to ensure thread-safe access to job data: - -```python -with self.lock: - self.jobs[job_id] = JobInfo(...) -``` - -### Progress Updates - -Jobs update their status in real-time: - -```python -# Step 1: Load data -job_manager.update_job(job_id, - completed_steps=0, - current_step="Loading dataset 'airline'..." -) - -# Step 2: Fit model -job_manager.update_job(job_id, - completed_steps=1, - current_step="Fitting ARIMA on airline..." -) - -# Step 3: Predict -job_manager.update_job(job_id, - completed_steps=2, - current_step="Generating predictions..." -) -``` - -### Memory Management - -Jobs are automatically cleaned up after 24 hours (configurable): - -```python -# Manual cleanup -cleanup_old_jobs(max_age_hours=12) - -# Or delete specific job -delete_job(job_id) -``` - -## Benefits - -✅ **Non-blocking** - Server stays responsive during long operations -✅ **Progress tracking** - Real-time updates on training progress -✅ **Time estimation** - Estimated time remaining -✅ **Error handling** - Graceful failure with error messages -✅ **Cancellation** - Ability to cancel long-running jobs -✅ **Memory efficient** - Automatic cleanup of old jobs - -## Comparison: Blocking vs Non-Blocking - -### Traditional (Blocking) - -```python -# ❌ Server freezes for 30 seconds -result = fit_predict(handle, "large_dataset", horizon=100) -# Server cannot handle any other requests during this time -``` - -### Async (Non-Blocking) - -```python -# ✅ Server returns immediately -job = fit_predict_async(handle, "large_dataset", horizon=100) -# Server continues handling other requests - -# Check progress whenever you want -status = check_job_status(job["job_id"]) -``` - -## When to Use Each - -### Use `fit_predict` (blocking) when: -- Training on small datasets (< 1 second) -- You need results immediately -- Running in a script/notebook (not via MCP) - -### Use `fit_predict_async` (non-blocking) when: -- Training large models -- Working with big datasets -- Server responsiveness is critical -- You want progress updates -- Training might take > 5 seconds - -## Future Enhancements - -Potential improvements: - -- [ ] Support for custom progress callbacks -- [ ] Job persistence across server restarts -- [ ] Job priority queue -- [ ] Parallel job execution limits -- [ ] Streaming progress updates via WebSocket -- [ ] Job result caching - -## Troubleshooting - -### Job stuck in "running" status - -```python -# Cancel and restart -cancel_job(job_id) -new_job = fit_predict_async(handle, dataset, horizon) -``` - -### Too many old jobs - -```python -# Clean up jobs older than 1 hour -cleanup_old_jobs(max_age_hours=1) -``` - -### Job failed - -```python -status = check_job_status(job_id) -if status["status"] == "failed": - print(f"Errors: {status['errors']}") -``` diff --git a/docs/source/concepts.md b/docs/source/concepts.md new file mode 100644 index 00000000..58e02b84 --- /dev/null +++ b/docs/source/concepts.md @@ -0,0 +1,41 @@ +# Core Concepts + +`sktime-mcp` is a specialized interface that allows you to perform advanced time-series analysis by collaborating with an AI assistant. It bridges the gap between natural language requests and the rigorous execution environment of the `sktime` library. + +## Collaborative Model Discovery + +Instead of requiring you to know the exact names of hundreds of forecasting models, `sktime-mcp` enables a discovery-based workflow. You can ask your assistant to find models based on your data's characteristics: +- **"Find models for multivariate data"** +- **"Which estimators handle missing values?"** +- **"Show me probabilistic forecasters."** + +The system queries the `sktime` registry in real-time, ensuring your assistant always has access to the most up-to-date models and their metadata. + +## Stateful Interaction (Handles) + +To allow for complex, multi-turn conversations, the server maintains a stateful runtime. This is managed through **Handles**. + +### How it works for you +When you ask the assistant to load a dataset or create a model, the server creates that object in memory and assigns it a "handle" (a unique ID). +- You don't need to track these IDs yourself. +- You can refer to objects naturally: *"Use the model we just created"* or *"Run that forecast on the sales data I loaded earlier."* +- The assistant manages the handles behind the scenes to ensure your requests are executed on the correct objects. + +### Memory Management +Because these objects stay in memory to support follow-up questions, you can tell the assistant to "clear the session" or "release the data" when you are finished to free up system resources. + +## Asynchronous Background Jobs + +Time-series forecasting can sometimes be computationally intensive. If a task (like training a deep learning model) will take a long time, you can ask the assistant to **run it in the background**. + +- This allows you to continue the conversation or perform other tasks while the model trains. +- You can ask for a status update at any time: *"Is the model training finished yet?"* +- The assistant will notify you once the results are ready. + +## Safety and Reproducibility + +### Secure Execution +Unlike systems that generate and run arbitrary code (which can be prone to errors or security risks), `sktime-mcp` uses a strictly defined set of tools. Your assistant interacts with `sktime` through a validated API, ensuring that operations are safe and predictable. + +### From Conversation to Code +Once you've found a workflow that works, you can turn your conversation into a permanent asset. Ask the assistant to **"export the Python code,"** and it will generate a standalone script that reproduces your entire analysis exactly. diff --git a/docs/source/data-sources.md b/docs/source/data-sources.md deleted file mode 100644 index c3b3fd07..00000000 --- a/docs/source/data-sources.md +++ /dev/null @@ -1,354 +0,0 @@ -# Data Source Support - -sktime-mcp now supports loading data from multiple sources beyond the built-in demo datasets! - -## Supported Data Sources - -1. **SQL Databases** - PostgreSQL, MySQL, SQLite, MSSQL -2. **Files** - CSV, TSV, Excel, Parquet - -## Quick Start - -### 1. CSV File - -```python -result = executor.load_data_source({ - "type": "file", - "path": "/path/to/data.csv", - "time_column": "date", - "target_column": "sales", - "exog_columns": ["temperature", "promotion"], # Optional -}) -``` - -### 2. SQL Database - -```python -# SQLite -result = executor.load_data_source({ - "type": "sql", - "connection_string": "sqlite:///path/to/database.db", - "query": "SELECT date, sales FROM sales_table", - "time_column": "date", - "target_column": "sales", -}) - -# PostgreSQL -result = executor.load_data_source({ - "type": "sql", - "connection_string": "postgresql://user:pass@host:5432/db", - "query": "SELECT * FROM sales WHERE date >= '2020-01-01'", - "time_column": "date", - "target_column": "sales", -}) -``` - -## MCP Tools - -### `load_data_source` - -Load data from any supported source. - -**Arguments:** -- `config` (dict): Data source configuration - -**Returns:** -- `success` (bool): Whether loading succeeded -- `data_handle` (str): Handle to reference the loaded data -- `metadata` (dict): Information about the data (rows, columns, frequency, etc.) -- `validation` (dict): Data quality validation results - -**Example:** -```json -{ - "config": { - "type": "pandas", - "data": {"date": [...], "value": [...]}, - "time_column": "date", - "target_column": "value" - } -} -``` - -### `fit_predict` (custom data) - -After `load_data_source`, call `fit_predict` with `data_handle` set (and omit `dataset`, or pass an empty string). This is the same MCP tool used for demo datasets. - -**Arguments:** -- `estimator_handle` (str): Handle from `instantiate_estimator` -- `data_handle` (str): Handle from `load_data_source` -- `horizon` (int): Forecast horizon (default: 12) - -**Example:** -```json -{ - "estimator_handle": "est_abc123", - "data_handle": "data_xyz789", - "horizon": 7 -} -``` - -### `list_data_sources` - -List all available data source types. - -**Returns:** -- `sources` (list): Available source types -- `descriptions` (dict): Description for each source - -### `list_available_data` - -List available data in a single response, including demo datasets and active handles. - -**Arguments (optional):** -- `is_demo` (bool): - - `True` → only demo datasets - - `False` → only active data handles - - omitted → both - -**Returns:** -- `system_demos` (list): Built-in demo dataset names -- `active_handles` (list): Active data handle information -- `total` (int): Combined count of returned entries - -### `release_data_handle` - -Release a data handle and free memory. - -**Arguments:** -- `data_handle` (str): Handle to release - -## Configuration Options - -### Pandas Adapter - -```python -{ - "type": "pandas", - "data": df_or_dict, # DataFrame or dict - "time_column": "date", # Optional, will try to detect - "target_column": "sales", # Optional, defaults to first column - "exog_columns": ["temp", "promo"], # Optional - "frequency": "D" # Optional, will try to infer -} -``` - -### SQL Adapter - -```python -{ - "type": "sql", - - # Option 1: Connection string - "connection_string": "postgresql://user:pass@host:5432/db", - - # Option 2: Individual components - "dialect": "postgresql", # postgresql, mysql, sqlite, mssql - "host": "localhost", - "port": 5432, - "database": "mydb", - "username": "user", - "password": "pass", - - # Query - "query": "SELECT * FROM sales", # Direct SQL query - # OR - "table": "sales", # Table name - "filters": {"region": "North"}, # Optional filters - - # Column mapping - "time_column": "date", - "target_column": "sales", - "exog_columns": ["temperature"], - - # Optional - "parse_dates": ["date"], - "frequency": "D" -} -``` - -### File Adapter - -```python -{ - "type": "file", - "path": "/path/to/data.csv", - "format": "csv", # csv, excel, parquet (auto-detected if not specified) - - # Column mapping - "time_column": "date", - "target_column": "sales", - "exog_columns": ["temperature"], - - # CSV-specific options - "csv_options": { - "sep": ",", - "header": 0, - "encoding": "utf-8" - }, - - # Excel-specific options - "excel_options": { - "sheet_name": 0, - "header": 0 - }, - - # Common options - "parse_dates": True, - "frequency": "D" -} -``` - -## Data Validation - -All data sources are automatically validated for: - -- ✅ DatetimeIndex presence -- ✅ No duplicate time indices -- ✅ Sufficient data points -- ✅ Missing value detection -- ✅ Frequency inference - -Validation results are included in the response: - -```python -{ - "valid": True, - "errors": [], # Critical issues that prevent usage - "warnings": ["Missing values detected: {'sales': 5.0}"] # Non-critical issues -} -``` - -## Installation - -### Core (Pandas support included) -```bash -pip install -e . -``` - -### With SQL support -```bash -pip install -e ".[sql]" -``` - -### With file format support -```bash -pip install -e ".[files]" -``` - -### With all optional dependencies -```bash -pip install -e ".[all]" -``` - -## Examples - -See the `examples/` directory for complete working examples: - -- `pandas_example.py` - Loading from pandas DataFrames -- `csv_example.py` - Loading from CSV/TSV files -- `sql_example.py` - Loading from SQL databases (SQLite, PostgreSQL, MySQL) - -## Architecture - -``` -Data Source Layer -├── base.py # Abstract adapter interface -├── registry.py # Adapter registry -└── adapters/ - ├── pandas_adapter.py # In-memory DataFrames - ├── sql_adapter.py # SQL databases - └── file_adapter.py # CSV, Excel, Parquet -``` - -Each adapter implements: -- `load()` - Load data from source -- `validate()` - Check data quality -- `to_sktime_format()` - Convert to (y, X) format - -## Metadata - -Each loaded data source provides rich metadata: - -```python -{ - "source": "sql", - "rows": 100, - "columns": ["sales", "temperature"], - "frequency": "D", - "start_date": "2020-01-01", - "end_date": "2020-04-09", - "missing_values": {"sales": 0, "temperature": 2} -} -``` - -## Error Handling - -All operations return structured error information: - -```python -{ - "success": False, - "error": "Could not convert index to datetime", - "error_type": "ValueError", - "validation": { - "valid": False, - "errors": ["Index must be DatetimeIndex"] - } -} -``` - -## 💡 LLM Guidelines & Best Practices - -When using these tools to handle user data, follow these best practices to avoid common execution errors: - -### 1. Column Identification -* **Target vs. Features**: Identify which column is the target (`y`) and which are exogenous features (`X`). -* **Numeric Only**: Ensure the target column contains only numeric data. Forecasting models will fail if string/categorical columns are accidentally included in `y`. -* **User Confirmation**: If the dataset has multiple columns and the user hasn't specified which one to forecast, **ask for clarification** instead of guessing. - -### 2. Strategic Loading -* **Use `usecols`**: When loading from CSV or Excel, use `csv_options: {"usecols": ["Column1", "Column2"]}` to load only the necessary columns. This prevents string-based columns from being accidentally passed to the model. -* **Time Column Mapping**: Always map the `time_column` if one exists. If it doesn't, sktime will default to a `RangeIndex`, which is often safer for simple sequences. - -### 3. Frequency Hurdles -* **Month-Start vs Month-End**: If you encounter errors like ` is not supported as period frequency`, it is likely due to `statsmodels` limitations. In such cases, consider dropping the timestamp index and using a simple integer index (RangeIndex) by not specifying a `time_column`. - -### 4. Validation First -* **Status Check**: Always inspect the `validation` object returned by `load_data_source`. If `valid` is `false`, read the `errors` list to understand why the data might break the model. - -## Troubleshooting - -### "No module named 'sqlalchemy'" -```bash -pip install "sktime-mcp[sql]" -``` - -### "No module named 'openpyxl'" (for Excel files) -```bash -pip install "sktime-mcp[files]" -``` - -### "No module named 'pyarrow'" (for Parquet files) -```bash -pip install "sktime-mcp[files]" -``` - -### "Could not infer frequency" -Specify frequency explicitly in config: -```python -config = { - ... - "frequency": "D" # Daily, "W" for weekly, "M" for monthly, etc. -} -``` - -### "Index must be DatetimeIndex" -Ensure your time column is specified correctly: -```python -config = { - ... - "time_column": "date", # Name of your date/time column - "parse_dates": True -} -``` diff --git a/docs/source/dev-guide.md b/docs/source/dev-guide.md deleted file mode 100644 index a492aad0..00000000 --- a/docs/source/dev-guide.md +++ /dev/null @@ -1,121 +0,0 @@ -# Developer Guide - -This guide explains how the project is structured, how to develop new features, and how to test changes. - -## Development Prerequisites - -- Python 3.10+ -- pip -- Optional: `sphinx` if you want to build the documentation site - -## Setup - -From the repo root: - -```bash -pip install -e ".[dev]" -``` - -Validate the environment before running tests: - -```bash -python -c "import sktime; print(sktime.__version__)" -``` - -If this command fails with `ModuleNotFoundError: No module named 'sktime'`, reinstall project dependencies in the active virtual environment: - -```bash -pip install -e ".[dev]" -``` - -If you work with SQL or file formats: - -```bash -pip install -e ".[sql]" -pip install -e ".[files]" -``` - -## Running The Server Locally - -```bash -python -m sktime_mcp.server -``` - -## Running Tests - -```bash -pytest -``` - -## Standard Local Checks - -From the repo root: - -```bash -make check -``` - -Auto-fix formatting and fixable lint issues: - -```bash -make format-fix -``` - -If `make` is unavailable (common on Windows), run: - -```bash -python -m ruff format --check . -python -m ruff check . -python -m pytest -``` - -## Project Layout - -- `src/sktime_mcp/server.py` MCP entry point and tool router -- `src/sktime_mcp/tools/` MCP tool implementations -- `src/sktime_mcp/registry/` sktime estimator discovery and tag logic -- `src/sktime_mcp/composition/` pipeline validation rules -- `src/sktime_mcp/runtime/` executor and handle manager -- `src/sktime_mcp/data/` data source adapters and registry -- `examples/` runnable examples - -## Architecture Overview - -High-level flow: - -1. MCP server receives a tool call -2. The server routes to a tool implementation in `tools/` -3. Tools use the registry/runtime layers to execute operations -4. Results are sanitized and returned to the client - -This separation keeps the MCP surface small while allowing deeper functionality in the runtime and data layers. - -## Adding A New MCP Tool - -1. Create a new module under `src/sktime_mcp/tools/` with a `*_tool` function. -2. Add a tool schema in `src/sktime_mcp/server.py` within `list_tools()`. -3. Add a handler in `call_tool()` to dispatch the tool name. -4. Add tests in `tests/`. -5. Document the tool in `docs/source/user-guide.md`. - -## Adding A New Data Source Adapter - -1. Implement a new adapter in `src/sktime_mcp/data/adapters/` that inherits `DataSourceAdapter`. -2. Register it in `src/sktime_mcp/data/registry.py` using `DataSourceRegistry.register()`. -3. Add tests covering `load()`, `validate()`, and `to_sktime_format()`. -4. Update `docs/source/data-sources.md`. - -## Adding A Demo Dataset - -Update `DEMO_DATASETS` in `src/sktime_mcp/runtime/executor.py` with a new loader string. Ensure the loader function returns a sktime-compatible dataset. - -## Documentation Build - -If you want to build the docs site locally: - -```bash -pip install -r docs/requirements.txt -sphinx-build -b html docs/source docs/build/html -``` - -The config lives in `docs/source/conf.py` and the docs content is under `docs/source/`. diff --git a/docs/source/developer/architecture.md b/docs/source/developer/architecture.md new file mode 100644 index 00000000..efca919b --- /dev/null +++ b/docs/source/developer/architecture.md @@ -0,0 +1,54 @@ +# Architecture + +`sktime-mcp` is a specialized implementation of the Model Context Protocol (MCP) server designed to expose the `sktime` library's capabilities to Large Language Models. + +## High-Level Overview + +The server operates as a bridge between the stateless nature of LLM interactions and the stateful requirements of machine learning workflows. It manages object lifecycles, provides semantic discovery of estimators, and handles complex data ingestion. + +```text +graph TD + Client[MCP Client: Claude/Cursor] <--> Server[sktime-mcp Server] + Server <--> Registry[sktime Estimator Registry] + Server <--> Runtime[Stateful Runtime: Handles & Jobs] + Runtime <--> Objects[Estimators & Datasets in RAM] + Runtime <--> Storage[Local Filesystem / MLflow] +``` + +## Core Components + +### 1. Server Layer (`server.py`) +This layer implements the MCP specification. It defines the tool schemas and dispatches incoming requests to the appropriate internal modules. It also handles JSON serialization/deserialization, ensuring that complex Python types are safely converted for the LLM. + +### 2. Registry Module (`registry/`) +The registry module dynamically inspects `sktime`. It uses `sktime.registry.all_estimators` to discover available models and their capabilities (tags). This ensures the server is always in sync with the installed version of `sktime`. + +### 3. Runtime and Handle Registry (`runtime/`) +The runtime is the heart of the server's statefulness. It maintains two primary registries: +- **Handle Registry**: A mapping of string identifiers to active Python objects (estimators and datasets). +- **Job Registry**: Tracks the status and results of background operations. + +### 4. Tool Implementation (`tools/`) +Tools are modularized into functional groups: +- **Discovery**: Querying the registry. +- **Data**: Ingesting and formatting time series. +- **Execution**: Fitting, predicting, and evaluating. +- **Composition**: Building pipelines. +- **Persistence**: Saving models and exporting code. + +## Data Flow: The "Handle" Pattern + +1. **Instantiation**: The client requests a model (e.g., `ARIMA`). The server instantiates the class, generates a UUID-based handle, and stores the object in the `Handle Registry`. +2. **Action**: The client requests an action (e.g., `fit_predict`) providing the handle. The server retrieves the object from the registry and executes the requested method. +3. **Response**: The server serializes the result (e.g., forecast values) into JSON and returns it to the client. + +## Concurrency Model + +- **Sync Tools**: Run on the main event loop. These should be fast, metadata-only operations. +- **Async Tools**: Heavy operations (like `fit_predict_async`) are dispatched to a `ThreadPoolExecutor`. This prevents a single long-running training job from blocking the entire server, allowing the client to continue querying status or other metadata. + +## Security Considerations + +- **Strict Typing**: All tool inputs are validated against JSON schemas. +- **Restricted Execution**: The server only allows instantiation of classes found in the `sktime` registry. It does not permit arbitrary code execution. +- **Local Access**: The server has the same permissions as the user running it. It can read/write files based on the user's filesystem access. diff --git a/docs/source/developer/contributing.md b/docs/source/developer/contributing.md new file mode 100644 index 00000000..d3a56d6d --- /dev/null +++ b/docs/source/developer/contributing.md @@ -0,0 +1,66 @@ +# Contributing Guide + +Thank you for your interest in contributing to `sktime-mcp`. This project aims to provide a robust MCP layer for the `sktime` ecosystem. + +## Development Setup + +1. **Fork and Clone**: + ```bash + git clone https://github.com/sktime/sktime-mcp.git + cd sktime-mcp + ``` + +2. **Environment**: + We recommend using a virtual environment. + ```bash + python3 -m venv .venv + source .venv/bin/activate + ``` + +3. **Install Dependencies**: + Install the package in editable mode with all development and optional dependencies. + ```bash + pip install -e ".[dev,all]" + ``` + +4. **Pre-commit Hooks**: + Enable pre-commit hooks to ensure code quality and formatting. + ```bash + pre-commit install + ``` + +## Code Quality Standards + +We use `ruff` for linting and formatting. + +- **Check Linting**: `ruff check .` +- **Format Code**: `ruff format .` + +Ensure all new code is properly type-hinted and includes docstrings following the NumPy format. + +## Testing + +The project uses `pytest`. All new features should be accompanied by unit or integration tests. + +### Running Tests +```bash +pytest tests/ +``` + +### Async Tests +Since the MCP server is heavily asynchronous, many tests use `pytest-asyncio`. Ensure your test environment is correctly configured. + +## Project Structure + +- `src/sktime_mcp/server.py`: The main entry point and tool dispatcher. +- `src/sktime_mcp/tools/`: Implementation of individual MCP tools, grouped by category. +- `src/sktime_mcp/runtime/`: State management, handle registry, and job execution. +- `src/sktime_mcp/data/`: Data loading and formatting logic. +- `src/sktime_mcp/composition/`: Pipeline validation and building. + +## Pull Request Process + +1. Create a new branch for your feature or bug fix. +2. Ensure all tests pass and linting is clean. +3. Update the documentation if you are adding or changing tools. +4. Submit a PR with a clear description of the changes and references to any related issues. diff --git a/docs/source/implementation.md b/docs/source/implementation.md deleted file mode 100644 index af433493..00000000 --- a/docs/source/implementation.md +++ /dev/null @@ -1,613 +0,0 @@ -# Complete Code Explanation: sktime-mcp - -## 📋 Table of Contents -1. [Project Overview](#project-overview) -2. [Architecture](#architecture) -3. [File-by-File Breakdown](#file-by-file-breakdown) -4. [How It All Works Together](#how-it-all-works-together) -5. [Key Concepts](#key-concepts) - ---- - -## Project Overview - -**sktime-mcp** is a Model Context Protocol (MCP) server that exposes the `sktime` time series library to Large Language Models (LLMs). It allows LLMs to: - -- **Discover** time series estimators from sktime's registry -- **Reason** about their capabilities using tags -- **Compose** estimators into pipelines -- **Execute** real forecasting workflows on datasets - -### What Problem Does It Solve? - -LLMs can't directly interact with Python libraries. This MCP server acts as a **semantic bridge**, translating between: -- **LLM world**: JSON-RPC requests with simple arguments -- **Python world**: Complex object instantiation, method calls, and data manipulation - ---- - -## Architecture - -The codebase is organized into **5 main layers**: - -``` -┌─────────────────────────────────────────┐ -│ MCP Server (server.py) │ ← Entry point, handles JSON-RPC -├─────────────────────────────────────────┤ -│ Tools Layer (tools/) │ ← MCP tool implementations -├─────────────────────────────────────────┤ -│ Registry (registry/) │ Composition │ ← Discovery & Validation -│ │ (composition/)│ -├─────────────────────────────────────────┤ -│ Runtime (runtime/) │ ← Execution & Handle Management -├─────────────────────────────────────────┤ -│ sktime Library │ ← Actual ML library -└─────────────────────────────────────────┘ -``` - ---- - -## File-by-File Breakdown - -### 📁 Root Level Files - -#### `README.md` -- **Purpose**: Project documentation and quick start guide -- **Key Sections**: - - Installation instructions - - Available MCP tools overview - - Example LLM workflow - - Project structure - -#### `pyproject.toml` -- **Purpose**: Python project configuration (PEP 518) -- **Key Contents**: - - Package metadata (name, version, description) - - Dependencies: `mcp`, `sktime`, `pandas`, `numpy`, `scikit-learn` - - Optional dependencies for dev and extended features - - Entry point: `sktime-mcp` command → `sktime_mcp.server:main` - - Tool configurations (ruff, pytest) - ---- - -### 📁 `src/sktime_mcp/` - Core Source Code - -#### `server.py` - MCP Server Entry Point -**Purpose**: Main MCP server that handles all tool calls - -**Key Components**: -1. **`sanitize_for_json(obj)`**: Converts Python objects to JSON-serializable format - - Handles numpy arrays, pandas objects, special types - -2. **`@server.list_tools()`**: Registers all available MCP tools - - Returns tool schemas (name, description, input schema) - - Tools span Discovery, Instantiation, Execution, Data, Export, Persistence, Validation, and Job Management. (e.g., `list_estimators`, `instantiate_pipeline`, `fit_predict_async`, `load_data_source`, `save_model`, `check_job_status`). - -3. **`@server.call_tool(name, arguments)`**: Routes tool calls to implementations - - Validates arguments - - Calls appropriate tool function - - Sanitizes and returns results - -4. **`main()`**: Entry point that starts the MCP server - - Uses stdio transport (reads from stdin, writes to stdout) - - Compatible with Claude Desktop and other MCP clients - -**Flow**: -``` -LLM → JSON-RPC request → server.call_tool() → tool function → sanitize → JSON response → LLM -``` - ---- - -### 📁 `src/sktime_mcp/registry/` - Estimator Discovery - -#### `interface.py` - Registry Interface -**Purpose**: Wraps sktime's `all_estimators()` function and provides structured access - -**Key Classes**: - -1. **`EstimatorNode`** (dataclass) - - Represents a single estimator with all its metadata - - **Fields**: - - `name`: Class name (e.g., "ARIMA") - - `task`: Task type (e.g., "forecasting") - - `module`: Python module path - - `class_ref`: Actual Python class - - `tags`: Capability tags (e.g., `{"capability:pred_int": True}`) - - `hyperparameters`: Constructor parameters with defaults - - `docstring`: Class documentation - - **Methods**: - - `to_dict()`: JSON serialization - - `to_summary()`: Minimal info for list operations - -2. **`RegistryInterface`** (singleton) - - **Purpose**: Lazy-loads and caches all sktime estimators - - **Key Methods**: - - `get_all_estimators(task, tags)`: Filter estimators by task and tags - - `get_estimator_by_name(name)`: Lookup specific estimator - - `list_estimators(query=...)`: Text search in names/docstrings - - `get_available_tasks()`: List all task types - - `get_available_tags()`: List all capability tags - - **Internal Methods**: - - `_load_registry()`: Calls sktime's `all_estimators()` for each task - - `_create_node()`: Extracts metadata from estimator class - - `_get_tags()`: Calls `cls.get_class_tags()` - - `_get_hyperparameters()`: Inspects `__init__` signature - -**How It Works**: -```python -# First call triggers lazy loading -registry = get_registry() -registry._load_registry() # Calls sktime.all_estimators("forecasting"), etc. - -# Creates EstimatorNode for each estimator -for name, cls in estimators: - node = EstimatorNode( - name=name, - task="forecasting", - class_ref=cls, - tags=cls.get_class_tags(), - hyperparameters=inspect.signature(cls.__init__).parameters - ) -``` - -#### `tag_resolver.py` - Tag Resolution -**Purpose**: Handles tag-based filtering and compatibility checking - -**Key Functions**: -- Resolves tag queries (e.g., `{"capability:pred_int": True}`) -- Checks if estimator tags match requirements -- Used by registry filtering and composition validation - ---- - -### 📁 `src/sktime_mcp/composition/` - Pipeline Validation - -#### `validator.py` - Composition Validator -**Purpose**: Validates that estimator compositions are valid before instantiation - -**Key Classes**: - -1. **`CompositionType`** (Enum) - - Types of compositions: PIPELINE, TRANSFORMER_PIPELINE, FORECASTING_PIPELINE, MULTIPLEXER, ENSEMBLE, REDUCTION - -2. **`CompositionRule`** (dataclass) - - Defines valid composition patterns - - Example: Transformers can precede forecasters - -3. **`ValidationResult`** (dataclass) - - **Fields**: `valid`, `errors`, `warnings`, `suggestions` - - **Method**: `to_dict()` for JSON serialization - -4. **`CompositionValidator`** (singleton) - - **Key Methods**: - - `validate_pipeline(components)`: Check if pipeline is valid - - `_check_pair_compatibility(first, second)`: Validate two estimators can be composed - - `_check_tag_compatibility(first, second)`: Check tag requirements - - `get_valid_compositions(estimator_name)`: What can precede/follow this estimator - - `suggest_pipeline(task, requirements)`: Suggest a valid pipeline - -**Validation Rules**: -```python -# Valid: Transformer → Forecaster -["Detrender", "ARIMA"] ✅ - -# Invalid: Forecaster → Forecaster -["ARIMA", "NaiveForecaster"] ❌ - -# Valid: Multiple Transformers → Forecaster -["ConditionalDeseasonalizer", "Detrender", "ARIMA"] ✅ -``` - ---- - -### 📁 `src/sktime_mcp/runtime/` - Execution Engine - -#### `handles.py` - Handle Manager -**Purpose**: Manages references to instantiated estimator objects - -**Why Needed?**: -- LLMs can't hold Python object references -- Solution: Create string handles (e.g., `"est_abc123"`) that map to objects - -**Key Classes**: - -1. **`HandleInfo`** (dataclass) - - Stores metadata about a handle - - **Fields**: `handle_id`, `estimator_name`, `instance`, `params`, `created_at`, `fitted`, `metadata` - -2. **`HandleManager`** (singleton) - - **Key Methods**: - - `create_handle(estimator_name, instance, params)`: Create new handle → returns `"est_xyz"` - - `get_instance(handle_id)`: Retrieve actual Python object - - `get_info(handle_id)`: Get handle metadata - - `mark_fitted(handle_id)`: Mark estimator as fitted - - `is_fitted(handle_id)`: Check if fitted - - `release_handle(handle_id)`: Free memory - - `list_handles()`: List all active handles - - `_cleanup_oldest()`: Auto-cleanup when max_handles reached - -**Flow**: -```python -# Instantiation -instance = ARIMA(order=[1,1,1]) -handle = manager.create_handle("ARIMA", instance, {"order": [1,1,1]}) -# Returns: "est_a1b2c3d4e5f6" - -# Later retrieval -instance = manager.get_instance("est_a1b2c3d4e5f6") -instance.fit(y) -``` - -#### `executor.py` - Execution Runtime -**Purpose**: Orchestrates estimator instantiation, data loading, fitting, and prediction - -**Key Class**: `Executor` (singleton) - -**Key Methods**: - -1. **`instantiate(estimator_name, params)`** - - Looks up estimator in registry - - Instantiates with parameters - - Creates handle - - Returns: `{"success": True, "handle": "est_xyz", ...}` - -2. **`load_dataset(name)`** - - Loads demo datasets (airline, sunspots, etc.) - - Uses sktime's dataset loaders - - Returns: pandas Series/DataFrame - -3. **`fit(handle_id, y, X, fh)`** - - Retrieves instance from handle - - Calls `instance.fit(y, X=X, fh=fh)` - - Marks handle as fitted - -4. **`predict(handle_id, fh, X)`** - - Retrieves fitted instance - - Calls `instance.predict(fh=fh, X=X)` - - Returns predictions - -5. **`fit_predict(handle_id, dataset, horizon)`** - - Convenience method: load → fit → predict - - Returns: `{"success": True, "predictions": {...}, "horizon": 12}` - -6. **`instantiate_pipeline(components, params_list)`** ⭐ **Most Complex** - - **Purpose**: Create complete pipelines from component names - - **Steps**: - 1. Validate pipeline composition - 2. Instantiate each component - 3. Build `steps` argument: `[("name1", instance1), ("name2", instance2)]` - 4. Determine pipeline type (TransformedTargetForecaster, Pipeline, etc.) - 5. Instantiate pipeline with steps - 6. Create handle - - **Why Complex**: Handles the "steps problem" - LLMs can't pass Python objects, so we build them server-side - -**Example Flow**: -```python -# LLM sends: -{"components": ["Detrender", "ARIMA"], "params_list": [{}, {"order": [1,1,1]}]} - -# Executor does: -detrender = Detrender() -arima = ARIMA(order=[1,1,1]) -steps = [("transformer", detrender), ("forecaster", arima)] -pipeline = TransformedTargetForecaster(steps=steps) -handle = handle_manager.create_handle("Pipeline", pipeline) - -# Returns to LLM: -{"success": True, "handle": "est_xyz", "pipeline": "Detrender → ARIMA"} -``` - ---- - -### 📁 `src/sktime_mcp/tools/` - MCP Tool Implementations - -Each file implements one or more MCP tools that LLMs can call. - -#### `list_estimators.py` -**Tools**: -1. **`list_estimators_tool(task, tags, query, limit)`** - - Calls `registry.get_all_estimators(task, tags)` - - Returns: `{"success": True, "estimators": [...], "total": 50}` - -2. **`get_available_tags()`** - - Returns all capability tags - - Example: `["capability:pred_int", "handles-missing-data", ...]` - -#### `describe_estimator.py` -**Tool**: `describe_estimator_tool(estimator)` -- Looks up estimator in registry -- Returns full EstimatorNode details -- Includes: name, task, module, tags, hyperparameters, docstring - -#### `instantiate.py` -**Tools**: -1. **`instantiate_estimator_tool(estimator, params)`** - - Calls `executor.instantiate(estimator, params)` - - Returns handle - -2. **`instantiate_pipeline_tool(components, params_list)`** ⭐ - - Calls `executor.instantiate_pipeline(components, params_list)` - - Solves the "steps problem" - - Returns single handle for entire pipeline - -3. **`release_handle_tool(handle)`** - - Frees memory for a handle - -4. **`list_handles_tool()`** - - Lists all active handles - -5. **`load_model_tool(path)`** - - Loads a previously saved model via MLflow - -#### `fit_predict.py` -**Tools**: -1. **`fit_predict_tool(estimator_handle, dataset, horizon)`** - - Calls `executor.fit_predict(handle, dataset, horizon)` - - Complete workflow in one call - -2. **`fit_predict_async_tool(estimator_handle, dataset, horizon)`** - - Dispatches a background job for fit and predict. - -#### `evaluate.py` -**Tool**: `evaluate_estimator_tool(estimator_handle, dataset, cv_folds)` -- Runs cross-validation using an expanding window splitter -- Returns comparison metrics like MAE and RMSE - -#### `format_tools.py` -**Tools**: -1. **`format_time_series_tool(...)`** - - Auto-formats, infers frequency, drops duplicates, and fills missing values. -2. **`auto_format_on_load_tool(enabled)`** - - Toggles whether new data sources get auto-formatted on load. - -#### `job_tools.py` -**Tools**: `check_job_status_tool`, `list_jobs_tool`, `cancel_job_tool`, `delete_job_tool`, `cleanup_old_jobs_tool` -- Interfaces with `JobManager` to control background training jobs. - -#### `save_model.py` -**Tool**: `save_model_tool(estimator_handle, path, mlflow_params)` -- Persists fitted estimators using MLflow. - -#### `list_available_data.py` -**Tool**: `list_available_data_tool(is_demo)` - - Returns available demo datasets and/or active user data handles - ---- - -### 📁 `examples/` - Usage Examples - -#### `01_forecasting_workflow.py` -**Purpose**: Demonstrates all MCP capabilities end-to-end - -**Steps**: -1. List datasets -2. Discover forecasting estimators -3. Filter by tags (probabilistic forecasters) -4. Describe an estimator -5. Validate pipeline compositions -6. Instantiate estimator -7. Fit and predict -8. List active handles -9. Show available tags - -**Run**: `python examples/01_forecasting_workflow.py` - -#### `02_llm_query_simulation.py` -**Purpose**: Simulates how an LLM would interact with the MCP - -**Scenario**: User asks "Forecast airline passengers with a probabilistic model" - -**LLM Steps**: -1. `list_estimators(task="forecasting", tags={"capability:pred_int": True})` -2. `describe_estimator("ARIMA")` -3. `instantiate_estimator("ARIMA", {"order": [1,1,1]})` -4. `fit_predict(handle, "airline", 12)` - -#### `03_pipeline_instantiation.py` -**Purpose**: Demonstrates pipeline creation - -**Examples**: -1. Simple 2-component pipeline -2. Complex 3-component pipeline (deseasonalize → detrend → forecast) -3. Pipeline with custom parameters -4. Invalid pipeline (shows validation errors) - -#### `04_mcp_pipeline_demo.py` -**Purpose**: End-to-end pipeline workflow - -**Steps**: -1. Validate pipeline -2. Instantiate pipeline → get handle -3. Fit and predict → get forecasts - -#### Additional Examples -- `05_simple_deseasonalize_detrend_forecaster.py`: Deseasonalize + detrend workflow -- `06_simple_naive_forecaster.py`: Basic NaiveForecaster example -- `background_training_example.py`: Demonstrates async background jobs -- `job_management_demo.py`: Demonstrates checking and listing job status -- `pandas_example.py`: Demonstrates loading from in-memory pandas objects -- `csv_example.py`: Demonstrates loading from CSV/TSV files -- `sql_example.py`: Demonstrates loading from SQL databases - ---- - -### 📁 `docs/` - Documentation - -#### `architecture.md` -- **Purpose**: High-level block diagrams explaining the data flow and adapter registry. - -#### `data-sources.md` -- **Purpose**: Detailed guide on loading data from Pandas, SQL, and various file formats. - -#### `user-guide.md` -- **Purpose**: Information for end-users on how to use the MCP tools. - -#### `dev-guide.md` -- **Purpose**: Guidelines for contributors on extending the server or adding new adapters. - ---- - -### 📁 `tests/` - Test Suite - -#### `test_core.py` -**Purpose**: Unit tests for core functionality - -**Test Classes**: -1. **`TestRegistryInterface`** - - Tests registry loading, filtering, lookup - -2. **`TestHandleManager`** - - Tests handle creation, retrieval, fitting, release - -3. **`TestCompositionValidator`** - - Tests pipeline validation logic - -4. **`TestTools`** - - Tests MCP tool functions - -**Run**: `pytest tests/` - ---- - -## How It All Works Together - -### Example: LLM Forecasting Workflow - -**User Prompt**: "Forecast airline passengers using ARIMA" - -**Step 1: Discovery** -``` -LLM → list_estimators(task="forecasting") - → server.call_tool("list_estimators", {"task": "forecasting"}) - → list_estimators_tool(task="forecasting") - → registry.get_all_estimators(task="forecasting") - → Returns: [{"name": "ARIMA", ...}, {"name": "NaiveForecaster", ...}, ...] -``` - -**Step 2: Description** -``` -LLM → describe_estimator("ARIMA") - → describe_estimator_tool("ARIMA") - → registry.get_estimator_by_name("ARIMA") - → Returns: {"name": "ARIMA", "hyperparameters": {"order": ...}, ...} -``` - -**Step 3: Instantiation** -``` -LLM → instantiate_estimator("ARIMA", {"order": [1,1,1]}) - → instantiate_estimator_tool("ARIMA", {"order": [1,1,1]}) - → executor.instantiate("ARIMA", {"order": [1,1,1]}) - → ARIMA_class = registry.get_estimator_by_name("ARIMA").class_ref - → instance = ARIMA_class(order=[1,1,1]) - → handle = handle_manager.create_handle("ARIMA", instance) - → Returns: {"success": True, "handle": "est_abc123"} -``` - -**Step 4: Execution** -``` -LLM → fit_predict("est_abc123", "airline", 12) - → fit_predict_tool("est_abc123", "airline", 12) - → executor.fit_predict("est_abc123", "airline", 12) - → y = executor.load_dataset("airline") - → instance = handle_manager.get_instance("est_abc123") - → instance.fit(y) - → predictions = instance.predict(fh=[1,2,...,12]) - → Returns: {"success": True, "predictions": {...}, "horizon": 12} -``` - -### Data Flow Diagram - -``` -┌─────────┐ -│ LLM │ -└────┬────┘ - │ JSON-RPC request - ▼ -┌─────────────────┐ -│ MCP Server │ ← server.py -│ (stdio) │ -└────┬────────────┘ - │ Route to tool - ▼ -┌─────────────────┐ -│ Tool Function │ ← tools/*.py -└────┬────────────┘ - │ Call business logic - ▼ -┌──────────────────────────────────┐ -│ Registry / Executor / Validator │ ← registry/, runtime/, composition/ -└────┬─────────────────────────────┘ - │ Interact with sktime - ▼ -┌─────────────────┐ -│ sktime Library │ -└─────────────────┘ -``` - ---- - -## Key Concepts - -### 1. **Registry-First Design** -- Don't parse code or docs -- Use sktime's `all_estimators()` as source of truth -- Extract metadata from classes directly - -### 2. **Handle-Based References** -- LLMs can't hold Python objects -- Solution: String handles (`"est_abc123"`) map to objects -- Handle manager maintains the mapping - -### 3. **Lazy Loading** -- Registry loads on first access -- Singleton pattern ensures one instance -- Caches all estimators for fast lookups - -### 4. **Tag-Based Discovery** -- Estimators have capability tags -- LLMs can filter by requirements -- Example: `{"capability:pred_int": True}` finds probabilistic forecasters - -### 5. **Composition Validation** -- Check pipeline validity before instantiation -- Prevents runtime errors -- Provides helpful error messages - -### 6. **The Steps Problem** -- **Problem**: Pipelines need `steps=[("name", instance), ...]` -- **Solution**: LLM sends component names, server builds instances -- **Benefit**: LLM uses simple JSON, server handles complexity - -### 7. **JSON Sanitization** -- Convert numpy/pandas to JSON-serializable types -- Handle special values (NaN, Infinity) -- Ensure all responses are valid JSON - -### 8. **Singleton Pattern** -- Registry, Executor, HandleManager, Validator are singletons -- Ensures shared state across tool calls -- Efficient memory usage - ---- - -## Summary - -**sktime-mcp** is a well-architected MCP server that: - -1. **Exposes** sktime's 200+ estimators to LLMs -2. **Validates** compositions before execution -3. **Manages** object lifecycles via handles -4. **Executes** real ML workflows on real data -5. **Translates** between JSON (LLM) and Python (sktime) - -**Key Innovation**: The `instantiate_pipeline` tool solves the "steps problem", enabling LLMs to create complex pipelines with a single JSON-RPC call. - -**Architecture Highlights**: -- Clean separation of concerns (registry, composition, runtime, tools) -- Singleton pattern for shared state -- Handle-based object management -- Comprehensive validation before execution -- JSON-first API design - -This enables LLMs to perform sophisticated time series forecasting workflows without writing any Python code! 🚀 diff --git a/docs/source/index.rst b/docs/source/index.rst index 62579048..47687727 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -9,40 +9,43 @@ sktime-mcp :target: https://github.com/sktime/sktime-mcp/blob/main/LICENSE :alt: License -The Semantic Engine for Time-Series Forecasting with LLMs. +The Semantic Engine for Time-Series Forecasting with Large Language Models. + +`sktime-mcp` is a Model Context Protocol (MCP) server that exposes the full power of the `sktime` ecosystem to AI assistants. It provides a stateful runtime for discovering, instantiating, and executing complex time-series forecasting workflows. .. toctree:: :maxdepth: 2 :caption: Getting Started - intro - use-cases - user-guide + installation + quickstart + concepts .. toctree:: :maxdepth: 2 - :caption: Features + :caption: User Guide - usage-examples - data-sources - background-jobs + user-guide/data-management + user-guide/forecasting-workflows + user-guide/pipelines + user-guide/async-operations + user-guide/persistence .. toctree:: :maxdepth: 2 - :caption: Development + :caption: Developer Guide - architecture - ideal-mcp-tools - implementation - dev-guide + developer/architecture + developer/contributing api .. toctree:: :maxdepth: 2 - :caption: Links + :caption: Project Links :hidden: GitHub Repository + sktime Project Indices and tables ================== diff --git a/docs/source/installation.md b/docs/source/installation.md new file mode 100644 index 00000000..08223e2d --- /dev/null +++ b/docs/source/installation.md @@ -0,0 +1,78 @@ +# Installation + +This guide covers the steps required to install `sktime-mcp` for both end-users and developers. + +## Prerequisites + +- **Python**: Version 3.10 or higher. +- **pip**: The Python package installer. +- **MCP Client**: A compatible Model Context Protocol client such as Claude Desktop, Cursor, or a VS Code extension (e.g., Cline). + +## Standard Installation + +The recommended way to install `sktime-mcp` is via pip or uv. To enable full functionality, including support for various data formats and forecasting models, use the `[all]` extra. + +### Using pip +```bash +pip install "sktime-mcp[all]" +``` + +### Using uv +If you prefer [uv](https://github.com/astral-sh/uv), you can run the server directly without manual installation using `uvx`: + +```bash +uvx sktime-mcp +``` + +Or install it globally: +```bash +uv tool install "sktime-mcp[all]" +``` + +### Optional Extras + +Depending on your use case, you may choose to install specific dependency groups: + +| Extra | Description | +| :--- | :--- | +| `forecasting` | Extended forecasting models (Prophet, TBATS, StatsForecast). | +| `sql` | Support for loading data from SQL databases (PostgreSQL, MySQL). | +| `files` | Support for Excel and Parquet file formats. | +| `mlflow` | Integration with MLflow for model persistence. | +| `dl` | Deep learning models (TensorFlow, PyTorch). | + +To install a specific extra, use: +```bash +pip install "sktime-mcp[extra_name]" +``` + +## Developer Installation + +If you intend to contribute to the project or run the test suite, install `sktime-mcp` from source in editable mode. + +1. **Clone the repository**: + ```bash + git clone https://github.com/sktime/sktime-mcp.git + cd sktime-mcp + ``` + +2. **Create and activate a virtual environment**: + ```bash + python3 -m venv .venv + source .venv/bin/activate # On Windows use `.venv\Scripts\activate` + ``` + +3. **Install in editable mode with development dependencies**: + ```bash + pip install -e ".[dev,all]" + ``` + +## Verification + +After installation, verify that the `sktime-mcp` CLI tool is available by checking its version: + +```bash +sktime-mcp --version +``` + +If the command is not found, ensure that your Python scripts directory is in your system's PATH. diff --git a/docs/source/intro.md b/docs/source/intro.md deleted file mode 100644 index 64b4408c..00000000 --- a/docs/source/intro.md +++ /dev/null @@ -1,105 +0,0 @@ - -# sktime-mcp - -**[Read the Official Documentation](http://sktime.github.io/sktime-mcp/)** | **[PyPI Package](https://pypi.org/project/sktime-mcp/)** - -
-

The Semantic Engine for Time-Series

-

- Enables Large Language Models to discover, reason about, and execute - sktime's advanced forecasting algorithms on real-world data. -

-
- -> **Why sktime-mcp?** -> Combines **LLM reasoning** with **time-series precision**. -> Instead of hallucinating Python code, your agent interacts with a strictly typed, -> safe, and stateful API to perform complex forecasting tasks. - ---- - -## 👋 Who is this for? - -sktime‑mcp is designed for: - -- **Developers** building LLM agents that need reliable, production‑grade time‑series forecasting. -- **Data scientists** who want to expose sktime workflows to language models without unsafe code generation. -- **Platform teams** integrating forecasting capabilities into tools like Claude Desktop, Cursor, or custom MCP clients. - -If you are new to MCP‑based workflows, start with the **Quick Start** below, then explore the **Use Cases** and **User Guide** for deeper examples. - ---- - -## 🔥 Key Features - -- **Semantic Discovery:** Find the perfect estimator using semantic similarity and capability tags (e.g., "probabilistic forecaster that handles missing data"). -- **Safe Composition:** Build complex pipelines (Transformer → Forecaster) with built-in validation to ensure components are compatible before execution. -- **Universal Data Loading:** Seamlessly ingest data from SQL, Pandas, Parquet, Excel, and CSV files. -- **Execution Runtime:** Stateful engine that manages object lifecycles, fitting, and predicting, all via simple JSON-RPC tools. - ---- - - -## ⚡ Quick Start - -Get up and running in seconds. Use with **Claude Desktop**, **Cursor**, or any MCP-compatible client. - -### 1. Install - -```bash -# Install directly from PyPI (with all optional dependencies recommended) -pip install "sktime-mcp[all]" -``` - -Alternatively, when contributing, use GitHub to install from source: - -```bash -git clone https://github.com/sktime/sktime-mcp.git -cd sktime-mcp -python3 -m venv .venv -source .venv/bin/activate -pip install -e ".[dev]" -``` - -### 2. Run -```bash -sktime-mcp -``` - -### 3. Connect (Claude Desktop Config) -Add this to your `claude_desktop_config.json`: -```json -{ - "mcpServers": { - "sktime": { - "command": "sktime-mcp" - } - } -} -``` - ---- - - -## 📚 Documentation Map - -| Section | Description | -| :--- | :--- | -| [**Use Cases**](use-cases.md) | Step-by-step workflows for coders and business users. | -| [**User Guide**](user-guide.md) | Comprehensive manual on using tools, workflows, and best practices. | -| [**Usage Examples**](usage-examples.md) | Example scripts and advanced usage patterns. | -| [**Background Jobs**](background-jobs.md) | Running long operations asynchronously. | -| [**Data Sources**](data-sources.md) | Comprehensive guide to loading data from SQL, Files, and Pandas. | -| [**Architecture**](architecture.md) | High-level system design, data flow, and limitations. | -| [**Implementation**](implementation.md) | Detailed code walkthrough and file breakdown. | -| [**Developer Guide**](dev-guide.md) | Contributing guidelines, testing, and extending the server. | - ---- - - -## 🚀 Get Started - -- See [Use Cases](use-cases.md) for step-by-step workflows. -- See [User Guide](user-guide.md) for detailed instructions and advanced features. - -[Get Started Now](use-cases.md){ .md-button .md-button--primary } diff --git a/docs/source/quickstart.md b/docs/source/quickstart.md new file mode 100644 index 00000000..a8e6500a --- /dev/null +++ b/docs/source/quickstart.md @@ -0,0 +1,60 @@ +# Quickstart + +This guide provides the fastest route to running the `sktime-mcp` server and connecting it to a client. + +## 1. Start the Server + +Once installed, you can start the MCP server directly from your terminal: + +```bash +sktime-mcp +``` + +By default, the server communicates over standard input/output (stdio). + +## 2. Connect to a Client + +To use `sktime-mcp` with an AI assistant, you must configure the client to launch the server. + +### Claude Desktop + +Add the following configuration to your `claude_desktop_config.json`: + +**Linux**: `~/.config/Claude/claude_desktop_config.json` +**macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json` +**Windows**: `%APPDATA%\Claude\claude_desktop_config.json` + +```json +{ + "mcpServers": { + "sktime": { + "command": "sktime-mcp" + } + } +} +``` + +### Cursor or VS Code (MCP Extensions) + +Configure the command `sktime-mcp` in the MCP settings section of your IDE or extension. + +## 3. Verify Functionality + +Once connected, you can verify the server is working by asking your assistant: + +> "What forecasting models are available in sktime?" + +The assistant will use the `list_estimators` tool and return a list of available models for your review. + +## 4. Run Your First Forecast + +Try an end-to-end workflow by directing your assistant to use a built-in dataset: + +> "Load the 'airline' demo dataset and forecast the next 12 months using a NaiveForecaster." + +The assistant will handle the technical steps — locating the data, instantiating the model, and fitting it — before presenting the final predictions to you in the chat. + +## Next Steps + +- Explore **Core Concepts** to understand how the system manages state. +- Refer to the **User Guide** for instructions on loading your own data and building pipelines. diff --git a/docs/source/usage-examples.md b/docs/source/usage-examples.md deleted file mode 100644 index cbb0679d..00000000 --- a/docs/source/usage-examples.md +++ /dev/null @@ -1,316 +0,0 @@ -# Usage Examples - -Examples of tasks performed via sktime-mcp and their corresponding MCP tool calls. - ---- - -## 🔄 End-to-End Examples - -### Example 1: Simple Forecasting - -**What you type:** - -> *"Forecast monthly airline passengers for the next 12 months using a probabilistic model."* - -**What happens:** - -1. The assistant searches sktime's registry for forecasting models that support prediction intervals. -2. It picks a suitable model (e.g., ARIMA) and shows you the options, or asks you to choose. -3. Once confirmed, it instantiates the model, fits it on the `airline` demo dataset, and generates a 12-month forecast. -4. You receive the predicted values (and optionally prediction intervals) in the conversation. - -**What you get back (example):** - -> Here are the forecasted airline passenger counts for the next 12 months: -> -> | Month | Forecast | Lower 95% | Upper 95% | -> |-------|----------|-----------|-----------| -> | 1 | 450.2 | 420.1 | 480.3 | -> | 2 | 455.1 | 418.7 | 491.5 | -> | ... | ... | ... | ... | -> -> The model used was ARIMA(1,1,1) fitted on the `airline` dataset (144 observations). - -**Follow-up prompts you could try:** - -- *"Try the same forecast with ExponentialSmoothing instead."* -- *"Show me the Python code to reproduce this."* -- *"Plot the forecast against the historical data."* - -
-🔧 Detailed MCP Tool Calls - -**1. Discover models with prediction interval support:** -```json -{ - "name": "list_estimators", - "arguments": { - "task": "forecasting", - "tags": {"capability:pred_int": true} - } -} -``` - -**2. Inspect the chosen estimator:** -```json -{ - "name": "describe_estimator", - "arguments": {"estimator": "ARIMA"} -} -``` - -**3. Instantiate with specific parameters:** -```json -{ - "name": "instantiate_estimator", - "arguments": {"estimator": "ARIMA", "params": {"order": [1, 1, 1]}} -} -``` -*Returns:* `{"handle": "est_abc123"}` - -**4. Fit and predict:** -```json -{ - "name": "fit_predict", - "arguments": {"estimator_handle": "est_abc123", "dataset": "airline", "horizon": 12} -} -``` - -
- ---- - -### Example 2: Pipeline Forecasting - -**What you type:** - -> *"I want to forecast with deseasonalization and detrending as preprocessing. Use ARIMA as the final model."* - -**What happens:** - -1. The assistant validates that `ConditionalDeseasonalizer → Detrender → ARIMA` is a valid pipeline in sktime. -2. It creates the pipeline and fits it on the data. -3. Predictions reflect the full preprocessing chain — deseasonalized, detrended, then forecast, then inverse-transformed back to the original scale. - -**What you get back:** - -> Pipeline created: **ConditionalDeseasonalizer → Detrender → ARIMA(1,1,1)** -> -> Forecast for the next 12 months on the `airline` dataset: -> -> | Month | Forecast | -> |-------|----------| -> | 1 | 448.7 | -> | 2 | 461.3 | -> | ... | ... | -> -> The preprocessing steps (deseasonalization and detrending) were applied automatically. - -**Follow-up prompts you could try:** - -- *"Replace ARIMA with ExponentialSmoothing in this pipeline."* -- *"Evaluate this pipeline with cross-validation."* -- *"What other preprocessing transformers are available?"* - -
-🔧 Detailed MCP Tool Calls - -**1. Validate composition:** -```json -{ - "name": "validate_pipeline", - "arguments": {"components": ["ConditionalDeseasonalizer", "Detrender", "ARIMA"]} -} -``` -*Returns:* `{"valid": true}` - -**2. Instantiate pipeline:** -```json -{ - "name": "instantiate_pipeline", - "arguments": { - "components": ["ConditionalDeseasonalizer", "Detrender", "ARIMA"], - "params_list": [{}, {}, {"order": [1, 1, 1]}] - } -} -``` -*Returns:* `{"handle": "est_xyz789", "pipeline": "ConditionalDeseasonalizer → Detrender → ARIMA"}` - -**3. Execute:** -```json -{ - "name": "fit_predict", - "arguments": {"estimator_handle": "est_xyz789", "dataset": "airline", "horizon": 12} -} -``` - -
- ---- - -### Example 3: Forecasting on Your Own CSV - -**What you type:** - -> *"Load my sales data from /home/user/sales.csv — the date column is 'month' and the target is 'revenue'. Then forecast 6 months ahead with AutoARIMA."* - -**What happens:** - -1. The assistant loads your CSV, using the columns you specified. -2. It auto-formats the data (infers frequency, handles missing values if needed) and reports a summary. -3. It instantiates `AutoARIMA`, fits it on your data, and returns a 6-month forecast. - -**What you get back:** - -> Loaded **sales.csv**: 48 rows, monthly frequency (2020-01 to 2023-12), target column "revenue". -> -> AutoARIMA selected order (2,1,1) with seasonal order (1,1,0,12). -> -> Forecast: -> -> | Month | Revenue Forecast | -> |---------|-----------------| -> | 2024-01 | 12,450 | -> | 2024-02 | 13,100 | -> | ... | ... | - -
-🔧 Detailed MCP Tool Calls - -**1. Load data:** -```json -{ - "name": "load_data_source", - "arguments": { - "config": { - "type": "file", - "path": "/home/user/sales.csv", - "time_column": "month", - "target_column": "revenue" - } - } -} -``` -*Returns:* `{"handle": "data_abc"}` - -**2. Instantiate AutoARIMA:** -```json -{ - "name": "instantiate_estimator", - "arguments": {"estimator": "AutoARIMA"} -} -``` -*Returns:* `{"handle": "est_def456"}` - -**3. Fit and predict:** -```json -{ - "name": "fit_predict", - "arguments": {"estimator_handle": "est_def456", "data_handle": "data_abc", "horizon": 6} -} -``` - -
- ---- - -### Example 4: Model Comparison - -**What you type:** - -> *"Compare NaiveForecaster, ExponentialSmoothing, and ARIMA on the airline dataset using cross-validation."* - -**What happens:** - -1. The assistant instantiates each model. -2. For each model, it runs cross-validated evaluation (expanding window backtest). -3. It presents a comparison table of error metrics so you can pick the best model. - -**What you get back:** - -> Cross-validation results on `airline` (3 folds, expanding window): -> -> | Model | Mean Absolute Error | RMSE | -> |------------------------|--------------------:|-------:| -> | NaiveForecaster | 38.2 | 45.1 | -> | ExponentialSmoothing | 24.7 | 31.3 | -> | ARIMA(1,1,1) | 22.1 | 28.9 | -> -> **ARIMA(1,1,1)** performed best on this dataset. - -**Follow-up prompts:** - -- *"Use ARIMA then — forecast 24 months ahead."* -- *"Save the ARIMA model for later."* -- *"Export the comparison code as a Python script."* - -### Example 5: Background Training - -**What you type:** - -> *"Train ARIMA(1,1,1) on my airline dataset in the background so I can do other things."* - -**What happens:** - -1. The assistant creates an estimator. -2. It uses `fit_predict_async` to launch the training job without blocking the connection. -3. You get a job ID immediately and can check the status later. - -**What you get back:** - -> Background training started! Job ID: `abc-123-def`. -> You can ask me to check its status or list all running jobs. - ---- - -### Example 6: Auto-Formatting Data - -**What you type:** - -> *"Load /home/user/messy_data.csv and fix any missing values or timestamp issues."* - -**What happens:** - -1. The assistant loads the data and notices warnings. -2. It uses `format_time_series` to infer frequency, fill missing values, and remove duplicates. -3. It reports the cleaned data summary. - ---- - -## 🛠️ Individual Tasks - -Below are standalone tasks you can ask about at any time — they don't need to be part of a larger workflow. - -### Discovering Models - -> *"What classification models does sktime have?"* -> -> *"Show me forecasters that handle missing data."* -> -> *"Tell me about the ThetaForecaster — what parameters does it take?"* - -### Loading and Inspecting Data - -> *"What demo datasets are available?"* -> -> *"Load my data from /home/user/experiment.parquet with time column 'ts' and target 'measurement'."* -> -> *"Describe the data I just loaded — how many rows, what's the frequency?"* - -### Exporting and Saving - -> *"Give me the Python code for the pipeline I just built."* -> -> *"Save this fitted model to /home/user/models/best_forecaster."* - -### Cleanup - -> *"Release all active handles to free up memory."* -> -> *"What handles are currently active?"* - ---- - -## 📋 MCP Tool Reference - -For developers and advanced users who want to understand the full MCP tool signatures, see the [Ideal MCP Tools](ideal-mcp-tools.md) design document. That page documents every tool's input schema, parameters, and intended use cases. diff --git a/docs/source/use-cases.md b/docs/source/use-cases.md deleted file mode 100644 index 20e3e629..00000000 --- a/docs/source/use-cases.md +++ /dev/null @@ -1,131 +0,0 @@ -# 🎯 Use Cases & Personas - -Describes typical user personas and their corresponding workflows. For step-by-step instructions, see the [User Guide](user-guide.md). For prompt examples, see [Usage Examples](usage-examples.md). - ---- - -## 👤 User Personas - -### 1. Coder - -**Goal:** Build, benchmark, and iterate on time-series models — and get reproducible Python code as output. - -#### Use-Case 1: Benchmarking Time Series Models - -**Scenario:** You want to compare several forecasting models (e.g., ARIMA, ExponentialSmoothing, Theta) on a dataset and pick the best one. - -**What you do:** - -1. Tell your AI assistant which dataset to use (a built-in demo or your own CSV/SQL data). -2. Ask it to compare several models using cross-validation. -3. Review the comparison table of metrics (MAE, RMSE, etc.). -4. Ask for the Python code that reproduces the best result. - -**Example prompt:** - -> *"Compare ARIMA, ExponentialSmoothing, and ThetaForecaster on the airline dataset with 5-fold cross-validation. Then export the code for the best model."* - -**What you get:** - -- A table of cross-validation metrics for each model. -- A recommendation for the best-performing model. -- A Python script that reproduces the experiment end-to-end. - ---- - -#### Use-Case 2: Composing Pipelines - -**Scenario:** You want to build a multi-step pipeline (e.g., deseasonalize → detrend → forecast) and evaluate it as a single unit. - -**What you do:** - -1. Describe the pipeline components and their order. -2. The assistant validates compatibility and creates the pipeline. -3. Ask the assistant to run it on your data. -4. Request the code to share with your team. - -**Example prompt:** - -> *"Build a pipeline: ConditionalDeseasonalizer, then Detrender, then ARIMA(1,1,1). Run it on the airline dataset for 12 months. Export the code."* - -**What you get:** - -- Confirmation that the pipeline is valid. -- Forecast results from the composed pipeline. -- A standalone Python script that constructs and runs the pipeline. - -**Tips for designing pipelines:** - -- Transformers (e.g., `Deseasonalizer`, `Detrender`, `BoxCoxTransformer`) go first. -- The final component should be a forecaster. -- If unsure which transformers to use, ask: *"What preprocessing transformers work well before ARIMA?"* - ---- - -### 2. Business User - -**Goal:** Get forecasts, analysis, and summary reports — without writing code. - -#### Use-Case: Get Forecasts & Reports - -**Scenario:** You have a CSV of monthly sales data and want a forecast for the next quarter, presented as a table you can share with stakeholders. - -**What you do:** - -1. Tell the assistant where your data file is and which columns to use. -2. Ask for a forecast (you can specify the model or let the assistant choose). -3. Ask the assistant to summarize the results. - -**Example prompt:** - -> *"Load my sales data from /home/user/quarterly_sales.csv (date column: 'quarter', target: 'revenue'). Forecast revenue for the next 4 quarters. Summarize the results."* - -**What you get:** - -- A table of forecasted values for each future period. -- A plain-language summary of the trend (e.g., *"Revenue is projected to grow ~5% per quarter"*). -- A downloadable Python script reproducing the process (if requested). -- Optionally, you can ask for a chart or a CSV export of the results. - ---- - -## 📂 Loading Data - -### Demo Datasets - -Built-in datasets are available for quick experimentation — no file paths needed: - -> *"What demo datasets are available?"* -> -> *"Use the airline dataset."* - -Demo datasets are useful for learning the workflows before bringing in your own data. - -### Your Own Data - -| Source | What to tell the assistant | Example | -|--------|---------------------------|---------| -| **Local CSV / Parquet / Excel** | Absolute file path, time column name, target column name. Tell it the `type` is `file`. | *"Load /home/user/data.csv, type 'file', time column 'date', target 'value'"* | -| **SQL database** | Connection string, query, time and target columns. Tell it the `type` is `sql`. | *"Load from my PostgreSQL database at localhost:5432/mydb with type 'sql', run 'SELECT date, sales FROM monthly', time column 'date', target 'sales'"* | - -For detailed configuration options, see [Data Sources](data-sources.md). - ---- - -## 📝 What You Can Get Out - -| Output | How to ask for it | Best for | -|--------|-------------------|----------| -| **Forecast table** | *"Forecast X for N periods"* | Quick answers, sharing with stakeholders | -| **Comparison metrics** | *"Compare these models with cross-validation"* | Model selection | -| **Python code** | *"Export the code for this"* | Reproducibility, integration into production | -| **Saved model artifact** | *"Save this model to /path"* | Persistence across server restarts | - ---- - -## 🔗 See Also - -- [User Guide](user-guide.md) — Step-by-step instructions for every workflow. -- [Usage Examples](usage-examples.md) — Concrete prompt examples with expected outputs. -- [Data Sources](data-sources.md) — Detailed data loading configuration. -- [Background Jobs](background-jobs.md) — Running long operations asynchronously. diff --git a/docs/source/user-guide.md b/docs/source/user-guide.md deleted file mode 100644 index 36a0eb04..00000000 --- a/docs/source/user-guide.md +++ /dev/null @@ -1,361 +0,0 @@ -# 📘 User Guide - -Welcome to the **sktime-mcp** User Guide. This guide walks you through installing, configuring, and using the MCP server for time-series forecasting — all through natural-language conversations with your AI assistant. - ---- - -## 🚀 Getting Started - -### Prerequisites - -Before you begin, ensure you have: - -- **Python 3.10+** installed. -- **pip** package manager. -- A compatible MCP client (like **Claude Desktop**, **Cursor**, or compatible **VS Code extensions** like Cline). - -### Installation - -Install the package directly from the source. We recommend installing with all dependencies to unlock full functionality. - -```bash -# Standard installation -pip install -e . - -# Recommended: Install with all optional extras (SQL, Forecasting, Files) -pip install -e ".[all]" -``` - -### Running the Server - -Start the MCP server to begin listening for connections: - -```bash -sktime-mcp -``` - -*Or manually via Python:* -```bash -python -m sktime_mcp.server -``` - -!!! tip "Client Configuration" - Ensure your MCP client (e.g., Claude Desktop) is configured to run this command. See the [official VSCode guidelines](https://code.visualstudio.com/docs/copilot/customization/mcp-servers#_configure-the-mcpjson-file) for configuration examples. - ---- - -## How It Works - -You interact with `sktime-mcp` through your AI assistant — you type requests in plain English, and the assistant translates them into the right sequence of operations behind the scenes. You never need to write JSON payloads or call tools directly. - -The typical flow looks like this: - -1. **You ask** your AI assistant a question or give it a task (e.g., *"Forecast airline passengers for the next 12 months"*). -2. **The assistant** selects the appropriate sktime-mcp tools, loads data, picks models, and runs them. -3. **You receive** results — forecasts, evaluation metrics, code, or summaries — in the conversation. - -The sections below describe what you can ask for and what to expect. - ---- - -## 🛠️ Core Capabilities - -The `sktime-mcp` server exposes a suite of tools that your AI assistant uses on your behalf: - -| What you can ask for | What happens behind the scenes | Example prompt | -|----------------------|-------------------------------|----------------| -| **Find models** | The assistant searches the sktime registry by task, tags, or keywords. | *"What forecasting models are available?"* | -| **Create a model or pipeline** | An estimator or multi-step pipeline is instantiated with your chosen parameters. | *"Set up an ARIMA(1,1,1) model"* | -| **Run a forecast** | The model is fitted on your data and predictions are generated. | *"Forecast the airline dataset 12 months ahead"* | -| **Cross-validate a model** | The model is evaluated across multiple folds using an expanding window to get metrics like MAE and RMSE. | *"Evaluate ARIMA using 3-fold cross-validation"* | -| **Run async background jobs** | Heavy training operations run in the background without blocking the client. | *"Fit this model in the background"* | -| **Load your own data** | CSV, Parquet, Excel, or SQL data is loaded and prepared for modelling. | *"Load my sales data from /home/user/sales.csv"* | -| **Export code** | A standalone Python script is generated so you can reproduce results outside the MCP server. | *"Give me the Python code for this model"* | -| **Save a trained model** | The fitted estimator is persisted to disk for later reuse. | *"Save this model to /home/user/models/arima"* | - ---- - -## ⚡ Workflows - -### 1. The "Hello World" of Forecasting - -**What you want:** Run a forecast on a built-in demo dataset. - -**Step 1: Find out what demo data is available** -Ask your assistant something like: - -> *"What demo datasets can I use?"* - -The assistant will return a list of built-in datasets (e.g., `airline`, `shampoo_sales`, `longley`). Each entry includes a short description and the shape of the data. - -**Step 2: Pick a forecasting model** -Ask your assistant to suggest models: - -> *"What forecasting models are available? Show me 5 options."* - -The assistant will return a list of registered forecasters from sktime with their names and brief descriptions. You can ask for more detail on any of them (e.g., *"Tell me more about ARIMA"*). - -**Step 3: Run the forecast** -Once you have picked a dataset and model, ask the assistant to run it: - -> *"Forecast the airline dataset 12 months ahead using NaiveForecaster."* - -The assistant will: - -1. Instantiate the model you chose. -2. Fit it on the dataset. -3. Generate predictions for the requested horizon. -4. Return the forecast values to you. - -You can then ask follow-up questions like *"Plot these results"*, *"Try a different model"*, or *"Give me the Python code for this"*. - -
-🔧 Detailed MCP Tool Calls - -The assistant translates this workflow into the following tool calls: - -```json -{"tool": "list_available_data", "arguments": {"is_demo": true}} -``` -```json -{"tool": "list_estimators", "arguments": {"task": "forecasting", "limit": 5}} -``` -```json -{"tool": "instantiate_estimator", "arguments": {"estimator": "NaiveForecaster"}} -``` -Returns a handle, e.g., `{"success": true, "handle": "est_abc123"}`. -```json -{"tool": "fit_predict", "arguments": {"estimator_handle": "est_abc123", "dataset": "airline", "horizon": 12}} -``` - -
- ---- - -### 2. Forecasting on Your Own Data - -**What you want:** Load a CSV file and forecast it. - -**Step 1: Load your data** -Tell the assistant where your file is and which columns matter: - -> *"Load my time series from /home/user/data/sales.csv. The time column is 'date' and the target column is 'revenue'."* - -The assistant loads the file and reports back with a summary (number of rows, date range, detected frequency). If the data has formatting issues, the assistant will auto-format it or ask you how to handle problems. - -**Step 2: Run a forecast** -Now ask for a forecast just like with demo data: - -> *"Forecast revenue 6 months ahead using ARIMA."* - -The assistant fits the model on your loaded data and returns predictions. - -**Step 3 (optional): Export the code** -If you want a standalone Python script: - -> *"Export the Python code for this model and data loading."* - -You receive a script you can run independently of the MCP server. - -
-🔧 Detailed MCP Tool Calls - -```json -{ - "tool": "load_data_source", - "arguments": { - "config": { - "type": "file", - "path": "/home/user/data/sales.csv", - "time_column": "date", - "target_column": "revenue" - } - } -} -``` -Returns a data handle, e.g., `{"handle": "data_xyz"}`. -```json -{"tool": "instantiate_estimator", "arguments": {"estimator": "ARIMA", "params": {"order": [1, 1, 1]}}} -``` -```json -{"tool": "fit_predict", "arguments": {"estimator_handle": "est_abc123", "data_handle": "data_xyz", "horizon": 6}} -``` -```json -{"tool": "export_code", "arguments": {"handle": "est_abc123", "include_fit_example": true}} -``` - -
- -!!! warning "Absolute Paths Required" - The server requires **absolute file paths** (e.g., `/home/user/data.csv`). Relative paths may fail depending on where the server was started. - ---- - -### 3. Advanced Pipeline Composition - -**What you want:** Build a multi-step preprocessing + forecasting pipeline. - -**Step 1: Describe the pipeline you want** -Tell the assistant the components and their order: - -> *"Build a pipeline that deseasonalizes, detrends, then applies ARIMA(1,1,1) to forecast."* - -The assistant will first validate that these components are compatible (e.g., that transformers feed into a forecaster correctly). If something is incompatible, it will explain why and suggest alternatives. - -**Step 2: Run it on data** -Once the pipeline is created, use it like any other model: - -> *"Run this pipeline on the airline dataset for 12 months ahead."* - -The entire pipeline (preprocessing + forecasting) runs end-to-end, and you get predictions back. - -
-🔧 Detailed MCP Tool Calls - -```json -{ - "tool": "validate_pipeline", - "arguments": {"components": ["ConditionalDeseasonalizer", "Detrender", "ARIMA"]} -} -``` -```json -{ - "tool": "instantiate_pipeline", - "arguments": { - "components": ["ConditionalDeseasonalizer", "Detrender", "ARIMA"], - "params_list": [{}, {}, {"order": [1, 1, 1]}] - } -} -``` -```json -{ - "tool": "fit_predict", - "arguments": {"estimator_handle": "est_xyz789", "dataset": "airline", "horizon": 12} -} -``` - -
- ---- - -### 4. Saving a Trained Model - -**What you want:** Persist a fitted model so it survives server restarts. - -After fitting a model (in any of the workflows above), ask: - -> *"Save this model to /home/user/models/my_forecaster."* - -The assistant persists the fitted estimator to the specified path using sktime's MLflow integration. You can later reload it: - -> *"Load the model from /home/user/models/my_forecaster and predict 6 months ahead on the airline dataset."* - -
-🔧 Detailed MCP Tool Calls - -**Saving:** -```json -{ - "tool": "save_model", - "arguments": { - "estimator_handle": "est_abc123", - "path": "/home/user/models/my_forecaster" - } -} -``` - -**Loading and predicting:** -```json -{ - "tool": "load_model", - "arguments": {"path": "/home/user/models/my_forecaster"} -} -``` -```json -{ - "tool": "fit_predict", - "arguments": {"estimator_handle": "est_restored", "dataset": "airline", "horizon": 6} -} -``` - -
- ---- - -## 💾 Data Management - -### Supported Data Sources - -| Source | How to use it | Example prompt | -|--------|--------------|----------------| -| **Demo datasets** | Reference by name — no file paths needed. | *"Use the airline dataset"* | -| **Local CSV / Parquet / Excel** | Provide the absolute path, time column, and target column. | *"Load /home/user/data.csv with time column 'date' and target 'value'"* | -| **SQL databases** | Provide connection details, query, and column names. | *"Load data from my PostgreSQL database using this query: SELECT ..."* | - -For full details on data source configuration, see the [Data Sources](data-sources.md) page. - -### Tips for Loading Data - -- **Always use absolute paths** for local files. -- If your data has formatting issues (missing values, duplicate timestamps, irregular frequency), mention it and the assistant can auto-format it for you. -- After loading, you can ask *"Describe my loaded data"* to see a summary before modelling. - ---- - -## 💡 Best Practices - -- **Start with demo data** to familiarize yourself with the workflow before loading your own files. -- **Ask for code export** after a successful experiment — this gives you a reproducible Python script. -- **Save fitted models** if you need them to survive server restarts. -- **Clean up when done** — ask the assistant to release data and model handles to free memory (e.g., *"Release all handles"*). -- **Use background execution for heavy jobs** — if a model takes a long time to fit, mention that you want it to run in the background (e.g., *"Fit this in the background and let me know when it's done"*). See [Background Jobs](background-jobs.md) for details. - ---- - -## ⚠️ Known Limitations - -While `sktime-mcp` is a powerful tool for prototyping, please be aware of the current limitations. - -#### 1. In-Memory Handles (Explicit Persistence Required) -The server stores active handles in standard Python dictionaries. -> **Impact**: If the server restarts or connection drops, in-memory handles are lost. Use `save_model` to persist fitted estimators when needed. - -#### 2. Mixed Sync/Async Execution -Heavy operations can block when using synchronous tools. -> **Impact**: For long-running jobs, ask the assistant to run them in the background so the server remains responsive. - -#### 3. Memory Limits -Data is read entirely into RAM. -> **Impact**: Loading multi-gigabyte files may crash the server. Consider pre-filtering large datasets before loading. - -#### 4. Security -Instantiation allows arbitrary parameters within the registry. -> **Impact**: While constrained to valid estimators, there is limited validation on parameter values. - -#### 5. Rigid Data Formatting -The auto-format logic is heuristic-based. -> **Impact**: Complex time series with irregular gaps or mixed frequencies might fail to auto-format correctly, requiring manual pre-processing outside the tool. - -#### 6. Local-Only Filesystem -> **Impact**: The server cannot access files inside isolated Docker containers unless volumes are mounted. Direct HTTP upload is not yet supported. - -#### 7. JSON Serialization Loss -Complex sktime types (Periods, Intervals) are converted to strings. -> **Impact**: Some rich metadata is simplified when returned in conversation. - -#### 8. Code Export Limitations -`export_code` uses template-based generation. -> **Impact**: Highly complex pipelines with lambda functions or edge cases might generate code that needs minor manual fixes. - ---- - -## ❓ Troubleshooting - -| Problem | What to do | -|---------|------------| -| **"Unknown estimator"** | Ask the assistant to search for the model by name — the exact casing matters. | -| **`No module named 'sktime'`** | Activate your project virtual environment and reinstall: `pip install -e ".[dev]"`. | -| **"Missing dependencies"** | Run `pip install -e ".[all]"` to ensure all optional extras are present. | -| **Model saving fails** | Ensure MLflow is installed in the server's environment. | -| **Data validation failures** | Ask the assistant to auto-format the data, or pre-process your file to fix timestamp issues. | -| **Server seems unresponsive** | A heavy model may be fitting synchronously. Wait for it to complete, or use background execution next time. | diff --git a/docs/source/user-guide/async-operations.md b/docs/source/user-guide/async-operations.md new file mode 100644 index 00000000..94aeb200 --- /dev/null +++ b/docs/source/user-guide/async-operations.md @@ -0,0 +1,37 @@ +# Asynchronous Operations + +For tasks that require significant time or computing power, you can ask your AI assistant to handle them in the background. This keeps your conversation responsive while the heavy work is performed. + +## When to Use Background Tasks + +Consider asking for background execution when: +- **Loading large datasets**: If you are ingesting millions of rows from a database or a file. +- **Training complex models**: If you are using deep learning or ensemble methods that take minutes to fit. +- **Extensive evaluation**: If you are running cross-validation with many folds. + +## How to Start a Background Task + +Simply mention to your assistant that you want the operation to run in the background. + +> *Example: "Fit this Prophet model in the background on my sales data."* + +The assistant will initiate the job and give you a confirmation. It will also mention a `job_id` (e.g., `job_8e1f2a`), which the system uses to track progress. + +## Monitoring Progress + +You don't need to wait silently. You can ask your assistant for updates at any time. + +> *Example: "What is the status of my background training job?"* + +The assistant will report the current state: +- **Pending/Running**: The job is still in progress. +- **Completed**: The job is finished, and the results (like a model handle or a forecast) are ready for use. +- **Failed**: Something went wrong, and the assistant will provide the error details. + +## Managing Jobs + +If you change your mind, you can tell the assistant to cancel a task. + +> *Example: "Cancel that background training job."* + +The assistant will terminate the operation and free up the associated system resources. diff --git a/docs/source/user-guide/data-management.md b/docs/source/user-guide/data-management.md new file mode 100644 index 00000000..bddba702 --- /dev/null +++ b/docs/source/user-guide/data-management.md @@ -0,0 +1,57 @@ +# Data Management + +Loading and preparing data is the first step in any forecasting workflow. With `sktime-mcp`, you don't need to write data-loading code; you simply tell your AI assistant where your data is and how it is structured. + +## Loading Your Data + +You can ask your assistant to load data from several different sources. When you do, be sure to provide the necessary context like file paths or database connection details. + +### Local Files +To load a local file, provide the absolute path. Supported formats include CSV, Excel (`.xlsx`), and Parquet. + +**What to tell your assistant:** +- The **absolute path** to the file. +- Which column contains the **timestamps** (if any). +- Which column is the **target** you want to forecast. + +> *Example: "Load my data from /home/user/sales.csv. Use 'Date' as the time column and 'Total' as the target."* + +### SQL Databases +You can also pull data directly from a database. + +**What to tell your assistant:** +- The **connection string** (URI). +- The **SQL query** to fetch the data. + +> *Example: "Load data from my PostgreSQL database at 'postgresql://user:pass@localhost/db' using 'SELECT * FROM monthly_revenue'."* + +### Remote URLs +If your data is hosted online, just provide the URL. + +> *Example: "Load the CSV data from https://example.com/timeseries.csv"* + +## Working with Handles + +When the assistant successfully loads your data, it will mention a `data_handle` (e.g., `data_5f2b3c`). This is an internal identifier that the system uses to track your dataset. You don't need to memorize these; you can simply refer to it as "my loaded data" or "the sales dataset" in your follow-up requests. + +## Formatting and Cleaning + +If your data isn't perfectly formatted, you can ask the assistant to clean it up for you. + +### Automatic Cleanup +You can tell the assistant to: +- **Infer the frequency**: "Fix the frequency of my data." +- **Fill missing values**: "Forward fill any gaps in the time series." +- **Remove duplicates**: "Remove any duplicate timestamps in my dataset." + +The assistant will use the `format_time_series` tool behind the scenes to ensure the data meets the requirements for forecasting. + +## Managing Resources + +The system keeps your loaded data in memory for the duration of your session. To free up resources, you can tell the assistant to "release my data" or "clear the memory" when you are finished with a specific dataset. + +## Built-in Demo Datasets + +For testing, you can ask to use standard time-series datasets by name (e.g., `airline`, `sunspots`, `lynx`) without needing to provide any files. + +> *Example: "What demo datasets are available?" or "Use the airline dataset for this forecast."* diff --git a/docs/source/user-guide/forecasting-workflows.md b/docs/source/user-guide/forecasting-workflows.md new file mode 100644 index 00000000..46b6448e --- /dev/null +++ b/docs/source/user-guide/forecasting-workflows.md @@ -0,0 +1,52 @@ +# Forecasting Workflows + +This guide covers how to discover models, train them on your data, and generate forecasts by collaborating with your AI assistant. + +## Model Discovery + +With hundreds of estimators available in `sktime`, you can ask your assistant to find the perfect model for your task. + +### Finding the Right Model +Ask your assistant to search for models based on their capabilities or specific tasks. + +> *Example: "Find models that can handle multivariate data" or "Show me probabilistic forecasters for forecasting."* + +The assistant will use the `list_estimators` tool to find matches. Common capabilities you might ask for include: +- **Prediction Intervals**: For probabilistic forecasting. +- **Multivariate Support**: For data with multiple related time series. +- **Missing Data Handling**: For datasets with gaps. + +### Inspecting a Model +If you want to know more about a specific model, ask for its details. + +> *Example: "Tell me more about the ARIMA model and its parameters."* + +The assistant will provide a description and a list of available hyperparameters. + +## The Fit-Predict Workflow + +Once you have a model and data, you can ask the assistant to perform the forecast. + +### Running a Forecast +You can request a forecast using either a demo dataset or your own loaded data. + +> *Example: "Forecast the next 12 months using ARIMA on the airline dataset."* + +The assistant will handle the instantiation, fitting, and prediction steps in one go. If you want to customize the model, you can specify parameters: + +> *Example: "Set up an ARIMA model with order (1, 1, 1) and forecast 6 steps ahead on my loaded data."* + +## Evaluation + +To see how well a model performs, ask your assistant to evaluate it. + +> *Example: "Evaluate this model using 3-fold cross-validation on the airline dataset."* + +The assistant will run the evaluation and report back with performance metrics like Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). + +## Best Practices for Users + +1. **Use Natural Language**: You don't need to know the technical tool names. Just tell the assistant what you want to achieve (e.g., "I want to forecast sales for next year"). +2. **Request Reproducibility**: After a successful experiment, ask for the code: "Give me the Python code for this workflow." This allows you to run the same analysis independently later. +3. **Manage Your Session**: If you have been working with many different models, you can tell the assistant to "release all handles" or "clear the session" to free up server memory. +4. **Leverage Background Jobs**: For very large datasets, tell the assistant to "run this in the background" so you can continue the conversation while the model trains. diff --git a/docs/source/user-guide/persistence.md b/docs/source/user-guide/persistence.md new file mode 100644 index 00000000..4f5dba7d --- /dev/null +++ b/docs/source/user-guide/persistence.md @@ -0,0 +1,43 @@ +# Persistence and Export + +`sktime-mcp` allows you to save your work for future sessions and export your analysis as production-ready code by working with your AI assistant. + +## Saving and Loading Your Models + +If you've trained a model that you want to use later, you can ask your assistant to save it to your local filesystem. + +### Saving a Model +Tell your assistant where you want to store the model. + +> *Example: "Save this fitted ARIMA model to /home/user/models/my_model."* + +The assistant will use the `save_model` tool to persist the estimator using MLflow. This ensures that the model's state (including its trained weights) is preserved. + +### Loading a Model +In a new session, you can bring back a previously saved model by telling the assistant where it is. + +> *Example: "Load my saved model from /home/user/models/my_model."* + +Once loaded, you can immediately start forecasting or evaluating the model as if you had just trained it. + +## Exporting Your Workflow as Code + +One of the most powerful features of `sktime-mcp` is the ability to turn a conversation into a standalone Python script. This is perfect for moving from exploration to production. + +### Generating Python Code +Simply ask your assistant to provide the code for your current workflow. + +> *Example: "Give me the Python code for the model we just built."* + +The assistant will generate a script that includes: +- All necessary **library imports** (`sktime`, `pandas`, etc.). +- The exact **model configuration** and hyperparameter settings. +- The **pipeline structure** (if you used multiple steps). +- An **example workflow** showing how to fit and predict with the model. + +This allows you to reproduce your results exactly, even without the MCP server running. + +## Important Considerations + +- **Absolute Paths**: When saving or loading, always provide absolute paths to ensure the assistant can find the correct location on your system. +- **Environment Matching**: When running exported code, make sure your Python environment has the same libraries installed as the server (e.g., `sktime`). diff --git a/docs/source/user-guide/pipelines.md b/docs/source/user-guide/pipelines.md new file mode 100644 index 00000000..cd9a175c --- /dev/null +++ b/docs/source/user-guide/pipelines.md @@ -0,0 +1,37 @@ +# Pipelines and Composition + +`sktime-mcp` allows you to build complex, multi-step workflows that combine data transformation with forecasting by simply describing them to your AI assistant. + +## Describing a Pipeline + +Instead of writing complex code to chain models together, you can describe the sequence of steps you want to perform. + +> *Example: "I want to deseasonalize and detrend my data before applying an ARIMA model."* + +Your assistant will understand these steps and use the `sktime` pipeline architecture to connect them correctly. + +## Validation + +Before a pipeline is created, your assistant can verify that the steps you've described are scientifically compatible. + +> *Example: "Check if combining a Detrender with an ARIMA model is valid."* + +The assistant will use the `validate_pipeline` tool to ensure that: +- Transformers are placed before the final forecaster. +- Data types (like univariate or multivariate) match across all steps. + +## Creating and Using Pipelines + +Once you are satisfied with the design, tell the assistant to build the pipeline. + +> *Example: "Create a pipeline with those steps and use it to forecast the airline dataset."* + +A pipeline is treated just like a single model. You can ask to fit it, evaluate it, or export its code. The system automatically handles the flow of data through every stage of the pipeline on your behalf. + +## Common Use Cases + +You can ask your assistant to help with: +- **Detrending**: Removing long-term trends from your data. +- **Deseasonalization**: Removing periodic patterns (e.g., monthly seasonality). +- **Imputation**: Filling missing values automatically as part of the model. +- **Scaling**: Standardizing data ranges for better model performance.