Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/Advanced/automated.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Automated Detection

## Important Notice ⚠️
This section has been withdrawn from version 2.0.3 for further tuning. Please refer to the updated documentation for the latest information and guidelines regarding automated detection.


Instead of running manually you can configure Tempo to run on a schedule using the following commands.

## Start Automated Inference
Expand Down
31 changes: 20 additions & 11 deletions docs/Advanced/finetune.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ Our baseline model provides robust performance across many use cases. However, f
- Start by testing the baseline model on a representative subset of your data using the [Evaluation function](#performance-evaluation)
- Collect performance metrics:
- Accuracy rate
- False Negative and Positive rate (F1 Score)
- Recall
- Cohen’s Kappa

2. Decision Criteria for Fine-Tuning
- Consider fine-tuning if:
Expand Down Expand Up @@ -41,32 +40,41 @@ Our baseline model provides robust performance across many use cases. However, f

### Performance Evaluation

You can evaluate model performance using the `CALL STATIC_DETECTION.evaluation();` function after assigning a table with labeled data to the evaluation reference in the reference table.
You can evaluate model performance using the `CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE();` function after assigning a table with labeled data to the evaluation reference in the reference table.

1. Assign a labeled data table to the evaluation reference within the reference table.

2. Run the `STATIC_DETECTION.evaluation()` procedure to generate performance metrics based on the assigned data.
2. Run the `MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE()` procedure to generate performance metrics based on the assigned data.

```sql
CALL STATIC_DETECTION.evaluation();
CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE();
```

### **Interpretation of Kappa Score:**
| **Kappa Score (κ)** | **Level of Agreement** |
|----------------------|------------------------|
| < 0.0 | Poor (Worse than chance) |
| 0.0 – 0.20 | Slight agreement |
| 0.21 – 0.40 | Fair agreement |
| 0.41 – 0.60 | Moderate agreement |
| 0.61 – 0.80 | Substantial agreement |
| 0.81 – 1.00 | Almost perfect agreement |

# Fine-tuning the Model

If you want to increase the accuracy by tuning the model to your own network you can use the following commands.

```sql
CALL model_optimization.tune_model('service_name');
CALL model_optimization.tune_model();
```
**Parameters:**
- `service_name`: Name of the service for model tuning (string)

Purpose: Updates model based on tuning log data from the reference page as shown in the screenshot
![reference page](../assets/reference_page.png)


### Model Rollback
```sql
CALL management.model_rollback(version);
CALL INFRA_CONTROLS.MODEL_ROLLBACK(version:int);
```
Removes the specified version of the model and its metadata from the app.

Expand All @@ -75,7 +83,8 @@ Removes the specified version of the model and its metadata from the app.

**Usage Example:**
```sql
CALL management.model_rollback(3);
CALL INFRA_CONTROLS.MODEL_ROLLBACK(3);
```

Warning: THIS ACTION CAN NOT BE UNDONE!!! Rolling back will remove all models after the version you roll back to.
⚠️ **WARNING: THIS ACTION IS PERMANENT AND IRREVERSIBLE!** ⚠️
Rolling back will **permanently delete** all model versions after the specified version. Proceed with extreme caution.
2 changes: 1 addition & 1 deletion docs/mitreclass.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Note: If no reference is specified, the application will default to its demo dat
To classify detected anomalies using MITRE ATT&CK mappings, execute the following SQL command in your worksheet:

```sql
CALL INSPECT.mitre_classification();
CALL THREAT_INTELLIGENCE.MITRE_TACTIC_CLASSIFICATION();
```

### Purpose
Expand Down
60 changes: 8 additions & 52 deletions docs/snow_commandRef.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

### Initialize Application Resources
```sql
CALL management.create_resources();
CALL INFRA_CONTROLS.CREATE_RESOURCES();
```
Purpose: Initializes the application by loading required model weights and configurations
Required Permissions: Warehouse, compute pool, and task management access
Expand All @@ -13,90 +13,46 @@ Required Permissions: Warehouse, compute pool, and task management access

### Run Inference
```sql
CALL static_detection.inference(True);
CALL STATIC_DETECTION.ANOMALY_DETECTION(with_mitre:boolean);
```
Parameters:
- `True/False`: Specifies whether to include optional MITRE tactic mappings in the classified anomalies. Set to TRUE to include the mappings, or False to exclude them.
Purpose: Executes inference on specified service data

### Deep Dive Analysis
```sql
CALL inspect.deepdive(sequence_id);
CALL INSPECT.INVESTIGATE_SEQUENCE(sequence_id:int);
```
Parameters:
- `sequence_id`: Identifier of the sequence to analyze (string/integer)
- `sequence_id`: Identifier of the sequence to analyze (integer)

Purpose: Investigates specific sequences flagged as anomalies

### MITRE ATT&CK Technique Classification
```sql
CALL inspect.mitre_classification();
CALL THREAT_INTELLIGENCE.MITRE_TACTIC_CLASSIFICATION();
```
Purpose: Classifies known anomlies to the corresponding MITRE ATT&CK technique

## Automated Detection

### Start Automated Inference
```sql
CALL automated_detection.start_automated_inference(
'source_table_name',
slot_number
);
```
Parameters:
- `source_table_name`: Fully qualified name of the source table (string)
- `slot_number`: Reference slot number (integer)
Notes:
- Scheduled for 8:00am UTC daily
- Stream names are automatically generated based on slot numbers:
- Slot 1: `stream_one_interactions`
- Slot 2: `stream_two_interactions`
- Slot 3: `stream_three_interactions`

### Manage Automated Inference
```sql
CALL automated_detection.alter_automated_inference('stream_name', 'action');
```
Parameters:
- `stream_name`: Name of the stream to manage (string)
- `action`: One of the following (string):
- `'suspend'`: Pause inference while maintaining stream updates
- `'resume'`: Restart paused inference
- `'stop'`: Terminate inference and clear source data

Example Usage:
```sql
-- Suspend stream
CALL automated_detection.alter_automated_inference('stream_one_interactions', 'suspend');

-- Resume stream
CALL automated_detection.alter_automated_inference('stream_one_interactions', 'resume');

-- Stop stream
CALL automated_detection.alter_automated_inference('stream_one_interactions', 'stop');
```

## Model Optimization

### Model Evaluation

```sql
CALL model_optimization.evaluation();
CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE();
```
Purpose: Evaluates the model's effectiveness using labeled user data to determine if the base model is suitable for the given use case.

### Fine-tune Model
```sql
CALL model_optimization.tune_model('service_name');
CALL MODEL_OPTIMIZATION.TUNE_MODEL();
```
Parameters:
- `service_name`: Name of the service for model tuning (string)

Purpose: Updates model based on tuning log data from the reference page

### Model Rollback
```sql
CALL management.model_rollback(version);
CALL INFRA_CONTROLS.MODEL_ROLLBACK(version:int);
```
Parameters:
- `version`: The integer version number of the model to be rolled back.
Expand Down
6 changes: 3 additions & 3 deletions docs/snowflake.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ In the new worksheet we now need to setup our procedures. We will start with ini

### Initialize Application Resources
```sql
CALL management.create_resources();
CALL INFRA_CONTROLS.CREATE_RESOURCES();
```
#### Purpose:
Initializes the application by loading required model weights and configuration using the granted permissions for warehouse and compute pool creation and task management
Expand All @@ -60,7 +60,7 @@ It is recommended that you run this command before running the sheet as a whole.

### Run Static Inference
```sql
CALL static_detection.inference(True);
CALL STATIC_DETECTION.ANOMALY_DETECTION(with_mitre:boolean);
```

**Parameters:**
Expand All @@ -73,7 +73,7 @@ This parameter toggles whether MITRE strategy mappings are added to the identifi
One unique capability of Tempo is that potential incidents are tied back to underlying data sequences. Every sequence used by Tempo is assigned a unique ID and is available for additional analysis. You can see this Sequence ID with every anomaly identified.

```sql
CALL inspect.deepdive(sequence_id);
CALL INSPECT.INVESTIGATE_SEQUENCE(sequence_id:int);
```
**Parameters:**
- `sequence_id`: Identifier of the sequence to analyze (integer). This ID can be used down the road if any anomalies are detected to run deeper investigation on suspicious interactions.
Expand Down