diff --git a/docs/Advanced/automated.md b/docs/Advanced/automated.md index 8b0ad89..f4a1375 100644 --- a/docs/Advanced/automated.md +++ b/docs/Advanced/automated.md @@ -1,5 +1,9 @@ # Automated Detection +## Important Notice ⚠️ +This section has been withdrawn from version 2.0.3 for further tuning. Please refer to the updated documentation for the latest information and guidelines regarding automated detection. + + Instead of running manually you can configure Tempo to run on a schedule using the following commands. ## Start Automated Inference diff --git a/docs/Advanced/finetune.md b/docs/Advanced/finetune.md index 39e123b..162fd01 100644 --- a/docs/Advanced/finetune.md +++ b/docs/Advanced/finetune.md @@ -12,8 +12,7 @@ Our baseline model provides robust performance across many use cases. However, f - Start by testing the baseline model on a representative subset of your data using the [Evaluation function](#performance-evaluation) - Collect performance metrics: - Accuracy rate - - False Negative and Positive rate (F1 Score) - - Recall + - Cohen’s Kappa 2. Decision Criteria for Fine-Tuning - Consider fine-tuning if: @@ -41,32 +40,41 @@ Our baseline model provides robust performance across many use cases. However, f ### Performance Evaluation -You can evaluate model performance using the `CALL STATIC_DETECTION.evaluation();` function after assigning a table with labeled data to the evaluation reference in the reference table. +You can evaluate model performance using the `CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE();` function after assigning a table with labeled data to the evaluation reference in the reference table. 1. Assign a labeled data table to the evaluation reference within the reference table. -2. Run the `STATIC_DETECTION.evaluation()` procedure to generate performance metrics based on the assigned data. +2. Run the `MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE()` procedure to generate performance metrics based on the assigned data. ```sql -CALL STATIC_DETECTION.evaluation(); +CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE(); ``` +### **Interpretation of Kappa Score:** +| **Kappa Score (κ)** | **Level of Agreement** | +|----------------------|------------------------| +| < 0.0 | Poor (Worse than chance) | +| 0.0 – 0.20 | Slight agreement | +| 0.21 – 0.40 | Fair agreement | +| 0.41 – 0.60 | Moderate agreement | +| 0.61 – 0.80 | Substantial agreement | +| 0.81 – 1.00 | Almost perfect agreement | + # Fine-tuning the Model If you want to increase the accuracy by tuning the model to your own network you can use the following commands. ```sql -CALL model_optimization.tune_model('service_name'); +CALL model_optimization.tune_model(); ``` -**Parameters:** -- `service_name`: Name of the service for model tuning (string) Purpose: Updates model based on tuning log data from the reference page as shown in the screenshot ![reference page](../assets/reference_page.png) + ### Model Rollback ```sql -CALL management.model_rollback(version); +CALL INFRA_CONTROLS.MODEL_ROLLBACK(version:int); ``` Removes the specified version of the model and its metadata from the app. @@ -75,7 +83,8 @@ Removes the specified version of the model and its metadata from the app. **Usage Example:** ```sql -CALL management.model_rollback(3); +CALL INFRA_CONTROLS.MODEL_ROLLBACK(3); ``` -Warning: THIS ACTION CAN NOT BE UNDONE!!! Rolling back will remove all models after the version you roll back to. +⚠️ **WARNING: THIS ACTION IS PERMANENT AND IRREVERSIBLE!** ⚠️ +Rolling back will **permanently delete** all model versions after the specified version. Proceed with extreme caution. diff --git a/docs/mitreclass.md b/docs/mitreclass.md index 407062d..a0d1190 100644 --- a/docs/mitreclass.md +++ b/docs/mitreclass.md @@ -28,7 +28,7 @@ Note: If no reference is specified, the application will default to its demo dat To classify detected anomalies using MITRE ATT&CK mappings, execute the following SQL command in your worksheet: ```sql -CALL INSPECT.mitre_classification(); +CALL THREAT_INTELLIGENCE.MITRE_TACTIC_CLASSIFICATION(); ``` ### Purpose diff --git a/docs/snow_commandRef.md b/docs/snow_commandRef.md index 0a850d0..b9cd7e2 100644 --- a/docs/snow_commandRef.md +++ b/docs/snow_commandRef.md @@ -4,7 +4,7 @@ ### Initialize Application Resources ```sql -CALL management.create_resources(); +CALL INFRA_CONTROLS.CREATE_RESOURCES(); ``` Purpose: Initializes the application by loading required model weights and configurations Required Permissions: Warehouse, compute pool, and task management access @@ -13,7 +13,7 @@ Required Permissions: Warehouse, compute pool, and task management access ### Run Inference ```sql -CALL static_detection.inference(True); +CALL STATIC_DETECTION.ANOMALY_DETECTION(with_mitre:boolean); ``` Parameters: - `True/False`: Specifies whether to include optional MITRE tactic mappings in the classified anomalies. Set to TRUE to include the mappings, or False to exclude them. @@ -21,82 +21,38 @@ Purpose: Executes inference on specified service data ### Deep Dive Analysis ```sql -CALL inspect.deepdive(sequence_id); +CALL INSPECT.INVESTIGATE_SEQUENCE(sequence_id:int); ``` Parameters: -- `sequence_id`: Identifier of the sequence to analyze (string/integer) +- `sequence_id`: Identifier of the sequence to analyze (integer) Purpose: Investigates specific sequences flagged as anomalies ### MITRE ATT&CK Technique Classification ```sql -CALL inspect.mitre_classification(); +CALL THREAT_INTELLIGENCE.MITRE_TACTIC_CLASSIFICATION(); ``` Purpose: Classifies known anomlies to the corresponding MITRE ATT&CK technique -## Automated Detection - -### Start Automated Inference -```sql -CALL automated_detection.start_automated_inference( - 'source_table_name', - slot_number -); -``` -Parameters: -- `source_table_name`: Fully qualified name of the source table (string) -- `slot_number`: Reference slot number (integer) -Notes: -- Scheduled for 8:00am UTC daily -- Stream names are automatically generated based on slot numbers: - - Slot 1: `stream_one_interactions` - - Slot 2: `stream_two_interactions` - - Slot 3: `stream_three_interactions` - -### Manage Automated Inference -```sql -CALL automated_detection.alter_automated_inference('stream_name', 'action'); -``` -Parameters: -- `stream_name`: Name of the stream to manage (string) -- `action`: One of the following (string): - - `'suspend'`: Pause inference while maintaining stream updates - - `'resume'`: Restart paused inference - - `'stop'`: Terminate inference and clear source data - -Example Usage: -```sql --- Suspend stream -CALL automated_detection.alter_automated_inference('stream_one_interactions', 'suspend'); - --- Resume stream -CALL automated_detection.alter_automated_inference('stream_one_interactions', 'resume'); - --- Stop stream -CALL automated_detection.alter_automated_inference('stream_one_interactions', 'stop'); -``` - ## Model Optimization ### Model Evaluation ```sql -CALL model_optimization.evaluation(); +CALL MODEL_OPTIMIZATION.EVALUATE_PERFORMANCE(); ``` Purpose: Evaluates the model's effectiveness using labeled user data to determine if the base model is suitable for the given use case. ### Fine-tune Model ```sql -CALL model_optimization.tune_model('service_name'); +CALL MODEL_OPTIMIZATION.TUNE_MODEL(); ``` -Parameters: -- `service_name`: Name of the service for model tuning (string) Purpose: Updates model based on tuning log data from the reference page ### Model Rollback ```sql -CALL management.model_rollback(version); +CALL INFRA_CONTROLS.MODEL_ROLLBACK(version:int); ``` Parameters: - `version`: The integer version number of the model to be rolled back. diff --git a/docs/snowflake.md b/docs/snowflake.md index 878a1d9..69c2e06 100644 --- a/docs/snowflake.md +++ b/docs/snowflake.md @@ -48,7 +48,7 @@ In the new worksheet we now need to setup our procedures. We will start with ini ### Initialize Application Resources ```sql -CALL management.create_resources(); +CALL INFRA_CONTROLS.CREATE_RESOURCES(); ``` #### Purpose: Initializes the application by loading required model weights and configuration using the granted permissions for warehouse and compute pool creation and task management @@ -60,7 +60,7 @@ It is recommended that you run this command before running the sheet as a whole. ### Run Static Inference ```sql -CALL static_detection.inference(True); +CALL STATIC_DETECTION.ANOMALY_DETECTION(with_mitre:boolean); ``` **Parameters:** @@ -73,7 +73,7 @@ This parameter toggles whether MITRE strategy mappings are added to the identifi One unique capability of Tempo is that potential incidents are tied back to underlying data sequences. Every sequence used by Tempo is assigned a unique ID and is available for additional analysis. You can see this Sequence ID with every anomaly identified. ```sql -CALL inspect.deepdive(sequence_id); +CALL INSPECT.INVESTIGATE_SEQUENCE(sequence_id:int); ``` **Parameters:** - `sequence_id`: Identifier of the sequence to analyze (integer). This ID can be used down the road if any anomalies are detected to run deeper investigation on suspicious interactions.