diff --git a/README.md b/README.md index 80aaeda..5aaa050 100755 --- a/README.md +++ b/README.md @@ -15,24 +15,51 @@ This model is then compared to an Azure AutoML run. ## Summary **In 1-2 sentences, explain the problem statement: e.g "This dataset contains data about... we seek to predict..."** +The dataset contains information from a bank marketing campaign. The problem is a binary classification problem where we need to predict whether the client subscribed for a term deposit(y) or not(n). The target column is represented by 'y' in the given dataset. Source of information: UCI ML Repository + **In 1-2 sentences, explain the solution: e.g. "The best performing model was a ..."** +The best performing model was a VotingEnsemble model trained by the AutoML feature of AzureML. It had accuracy of 91.68% + ## Scikit-learn Pipeline **Explain the pipeline architecture, including data, hyperparameter tuning, and classification algorithm.** +The scikit-learn pipeline consists of the following stages: +1. Fetching the data from the remote URL +2. Cleaning the data +3. Splitting the data into train and test sets +4. Hyperparameter Tuning on a Logistic Regression classifier using Hyperdrive python package of AzureML + **What are the benefits of the parameter sampler you chose?** +I chose RandomParameterSampling. Here, hyperparameter values are randomly selected from the defined search space. Random sampling allows the search space to include both discrete and continuous hyperparameters. This is usually faster than Grid Search, because parameters are picked up randomly. + **What are the benefits of the early stopping policy you chose?** +I chose BanditPolicy. The policy early terminates any runs where the primary metric is not within the specified slack factor/slack amount with respect to the best performing training run. This prevents certain unnecesary Runs from consuming compute resources. + ## AutoML **In 1-2 sentences, describe the model and hyperparameters generated by AutoML.** ## Pipeline comparison **Compare the two models and their performance. What are the differences in accuracy? In architecture? If there was a difference, why do you think there was one?** +The model generated in Hyperdrive optimization is a Logistic Regression model, with accuracy 91%. +The model generated by AutoML is a VotingEnsemble model, with accuracy 91.68%. + ## Future work **What are some areas of improvement for future experiments? Why might these improvements help the model?** +1. Add additional parameters to AutoMLConfig +2. Give more choices for the hyper-parameters inside RandomParameterSampling, i.e. for C and max_iter +3. Try other Parameter Sampling techniques in Hyperdrive +4. Try other Early Stopping policies in Hyperdrive +5. Train Deep Learning models instead of Logistic Regression, they are capable of improving accuracy further. Also include deep learning models in the AutoML feature +6. We noticed that there is class imbalance in the dataset. By using over-sampling techniques on the minority class, we can bring about class balance, and this could potentially improve model performance. +7. It might help to store the data permanently in our Datastore. Currently, we are directly bringing the data from the remote URL to our notebook. In future: + => if the data changes in the remote URL, the results would vary. We need to be able to associate model performance with the corresponding data that was used. + => Also, if the URL does not work in future, it becomes problematic. + ## Proof of cluster clean up **If you did not delete your compute cluster in the code, please complete this section. Otherwise, delete this section.** **Image of cluster marked for deletion** diff --git a/udacity-project.ipynb b/udacity-project.ipynb index e1531ad..825021f 100755 --- a/udacity-project.ipynb +++ b/udacity-project.ipynb @@ -15,17 +15,11 @@ "\n", "run = exp.start_logging()" ], - "outputs": [ - { - "output_type": "stream", - "name": "stdout", - "text": "Workspace name: quick-starts-ws-207125\nAzure region: southcentralus\nSubscription id: 976ee174-3882-4721-b90a-b5fef6b72f24\nResource group: aml-quickstarts-207125\n" - } - ], + "outputs": [], "execution_count": 1, "metadata": { "gather": { - "logged": 1662967210545 + "logged": 1662977319339 } } }, @@ -53,13 +47,13 @@ { "output_type": "stream", "name": "stdout", - "text": "Found existing cluster, use it.\nSucceeded\nAmlCompute wait for completion finished\n\nMinimum number of nodes requested have been provisioned\n" + "text": "InProgress.\nSucceededProvisioning operation finished, operation \"Succeeded\"\nSucceeded\nAmlCompute wait for completion finished\n\nMinimum number of nodes requested have been provisioned\n" } ], "execution_count": 2, "metadata": { "gather": { - "logged": 1662967210650 + "logged": 1662977326060 }, "jupyter": { "outputs_hidden": false, @@ -133,7 +127,7 @@ "execution_count": 3, "metadata": { "gather": { - "logged": 1662967214537 + "logged": 1662977330136 }, "jupyter": { "outputs_hidden": false, @@ -156,7 +150,7 @@ "execution_count": 4, "metadata": { "gather": { - "logged": 1662967215375 + "logged": 1662977331329 } } }, @@ -175,7 +169,7 @@ "application/vnd.jupyter.widget-view+json": { "version_major": 2, "version_minor": 0, - "model_id": "47c0845436ef4da394a3191f33a65d96" + "model_id": "5f8205ef20e6406ab44a4a4cbaef08ea" } }, "metadata": {} @@ -183,55 +177,37 @@ { "output_type": "display_data", "data": { - "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/HD_595634fe-649f-4c02-877d-b358149585fb?wsid=/subscriptions/976ee174-3882-4721-b90a-b5fef6b72f24/resourcegroups/aml-quickstarts-207125/workspaces/quick-starts-ws-207125&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\", \"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb\", \"run_properties\": {\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb\", \"created_utc\": \"2022-09-12T07:20:15.110531Z\", \"properties\": {\"primary_metric_config\": \"{\\\"name\\\":\\\"Accuracy\\\",\\\"goal\\\":\\\"maximize\\\"}\", \"resume_from\": \"null\", \"runTemplate\": \"HyperDrive\", \"azureml.runsource\": \"hyperdrive\", \"platform\": \"AML\", \"ContentSnapshotId\": \"56c44205-9676-4c8a-9e38-129639e93ca7\", \"user_agent\": \"python/3.8.5 (Linux-5.15.0-1017-azure-x86_64-with-glibc2.10) msrest/0.7.1 Hyperdrive.Service/1.0.0 Hyperdrive.SDK/core.1.44.0\", \"space_size\": \"8\", \"score\": \"0.9100509832483612\", \"best_child_run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_4\", \"best_metric_status\": \"Succeeded\", \"best_data_container_id\": \"dcid.HD_595634fe-649f-4c02-877d-b358149585fb_4\"}, \"tags\": {\"_aml_system_max_concurrent_jobs\": \"4\", \"_aml_system_max_total_jobs\": \"10\", \"_aml_system_max_duration_minutes\": \"10080\", \"_aml_system_policy_config\": \"{\\\"name\\\":\\\"Bandit\\\",\\\"properties\\\":{\\\"evaluation_interval\\\":1,\\\"delay_evaluation\\\":5,\\\"slack_factor\\\":0.1}}\", \"_aml_system_generator_config\": \"{\\\"name\\\":\\\"RANDOM\\\",\\\"parameter_space\\\":{\\\"C\\\":[\\\"choice\\\",[[0.01,0.001,0.1,1]]],\\\"max_iter\\\":[\\\"choice\\\",[[50,100]]]},\\\"properties\\\":null}\", \"_aml_system_primary_metric_config\": \"{\\\"name\\\":\\\"Accuracy\\\",\\\"goal\\\":\\\"maximize\\\"}\", \"_aml_system_platform_config\": \"{\\\"ServiceAddress\\\": \\\"https://southcentralus.experiments.azureml.net\\\", \\\"SubscriptionId\\\": \\\"976ee174-3882-4721-b90a-b5fef6b72f24\\\", \\\"ResourceGroupName\\\": \\\"aml-quickstarts-207125\\\", \\\"WorkspaceName\\\": \\\"quick-starts-ws-207125\\\", \\\"ExperimentName\\\": \\\"udacity-project\\\", \\\"Definition\\\": {\\\"Configuration\\\": null, \\\"Attribution\\\": null, \\\"TelemetryValues\\\": {\\\"amlClientType\\\": \\\"azureml-sdk-train\\\", \\\"amlClientModule\\\": \\\"[Scrubbed]\\\", \\\"amlClientFunction\\\": \\\"[Scrubbed]\\\", \\\"tenantId\\\": \\\"660b3398-b80e-49d2-bc5b-ac1dc93b5254\\\", \\\"amlClientRequestId\\\": \\\"a95be214-a556-458d-a450-ec0bd6ef72f7\\\", \\\"amlClientSessionId\\\": \\\"6eae51e4-f234-4e3f-aa70-2434c0c7cdc2\\\", \\\"subscriptionId\\\": \\\"976ee174-3882-4721-b90a-b5fef6b72f24\\\", \\\"estimator\\\": \\\"NoneType\\\", \\\"samplingMethod\\\": \\\"RANDOM\\\", \\\"terminationPolicy\\\": \\\"Bandit\\\", \\\"primaryMetricGoal\\\": \\\"maximize\\\", \\\"maxTotalRuns\\\": 10, \\\"maxConcurrentRuns\\\": 4, \\\"maxDurationMinutes\\\": 10080, \\\"vmSize\\\": null}, \\\"Overrides\\\": {\\\"Script\\\": \\\"train.py\\\", \\\"Command\\\": \\\"\\\", \\\"UseAbsolutePath\\\": false, \\\"Arguments\\\": [], \\\"SourceDirectoryDataStore\\\": null, \\\"Framework\\\": 0, \\\"Communicator\\\": 0, \\\"Target\\\": \\\"optimize-pipeline-cluster\\\", \\\"DataReferences\\\": {}, \\\"Data\\\": {}, \\\"OutputData\\\": {}, \\\"Datacaches\\\": [], \\\"JobName\\\": null, \\\"MaxRunDurationSeconds\\\": 2592000, \\\"NodeCount\\\": 1, \\\"InstanceTypes\\\": [], \\\"Priority\\\": null, \\\"CredentialPassthrough\\\": false, \\\"Identity\\\": null, \\\"Environment\\\": {\\\"Name\\\": \\\"sklearn-env\\\", \\\"AutoRebuild\\\": true, \\\"Python\\\": {\\\"InterpreterPath\\\": \\\"python\\\", \\\"UserManagedDependencies\\\": false, \\\"CondaDependencies\\\": {\\\"dependencies\\\": [\\\"python=3.6.2\\\", \\\"scikit-learn\\\", \\\"numpy\\\", \\\"pandas\\\", {\\\"pip\\\": [\\\"azureml-defaults\\\"]}]}, \\\"BaseCondaEnvironment\\\": null}, \\\"EnvironmentVariables\\\": {\\\"EXAMPLE_ENV_VAR\\\": \\\"EXAMPLE_VALUE\\\"}, \\\"Docker\\\": {\\\"BaseImage\\\": \\\"mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220708.v1\\\", \\\"Platform\\\": {\\\"Os\\\": \\\"Linux\\\", \\\"Architecture\\\": \\\"amd64\\\"}, \\\"BaseDockerfile\\\": null, \\\"BaseImageRegistry\\\": {\\\"Address\\\": null, \\\"Username\\\": null, \\\"Password\\\": null}, \\\"Enabled\\\": false, \\\"Arguments\\\": []}, \\\"Spark\\\": {\\\"Repositories\\\": [], \\\"Packages\\\": [], \\\"PrecachePackages\\\": true}, \\\"InferencingStackVersion\\\": null}, \\\"History\\\": {\\\"OutputCollection\\\": true, \\\"DirectoriesToWatch\\\": [\\\"logs\\\"], \\\"EnableMLflowTracking\\\": true, \\\"snapshotProject\\\": true}, \\\"Spark\\\": {\\\"Configuration\\\": {\\\"spark.app.name\\\": \\\"Azure ML Experiment\\\", \\\"spark.yarn.maxAppAttempts\\\": \\\"1\\\"}}, \\\"ParallelTask\\\": {\\\"MaxRetriesPerWorker\\\": 0, \\\"WorkerCountPerNode\\\": 1, \\\"TerminalExitCodes\\\": null, \\\"Configuration\\\": {}}, \\\"BatchAi\\\": {\\\"NodeCount\\\": 0}, \\\"AmlCompute\\\": {\\\"Name\\\": null, \\\"VmSize\\\": null, \\\"RetainCluster\\\": false, \\\"ClusterMaxNodeCount\\\": null}, \\\"AISuperComputer\\\": {\\\"InstanceType\\\": \\\"D2\\\", \\\"FrameworkImage\\\": null, \\\"ImageVersion\\\": \\\"pytorch-1.7.0\\\", \\\"Location\\\": null, \\\"AISuperComputerStorageData\\\": null, \\\"Interactive\\\": false, \\\"ScalePolicy\\\": null, \\\"VirtualClusterArmId\\\": null, \\\"TensorboardLogDirectory\\\": null, \\\"SSHPublicKey\\\": null, \\\"SSHPublicKeys\\\": null, \\\"EnableAzmlInt\\\": true, \\\"Priority\\\": \\\"Medium\\\", \\\"SLATier\\\": \\\"Standard\\\", \\\"UserAlias\\\": null}, \\\"KubernetesCompute\\\": {\\\"InstanceType\\\": null}, \\\"Tensorflow\\\": {\\\"WorkerCount\\\": 1, \\\"ParameterServerCount\\\": 1}, \\\"Mpi\\\": {\\\"ProcessCountPerNode\\\": 1}, \\\"PyTorch\\\": {\\\"CommunicationBackend\\\": \\\"nccl\\\", \\\"ProcessCount\\\": null}, \\\"Hdi\\\": {\\\"YarnDeployMode\\\": 2}, \\\"ContainerInstance\\\": {\\\"Region\\\": null, \\\"CpuCores\\\": 2.0, \\\"MemoryGb\\\": 3.5}, \\\"ExposedPorts\\\": null, \\\"Docker\\\": {\\\"UseDocker\\\": false, \\\"SharedVolumes\\\": true, \\\"ShmSize\\\": \\\"2g\\\", \\\"Arguments\\\": []}, \\\"Cmk8sCompute\\\": {\\\"Configuration\\\": {}}, \\\"CommandReturnCodeConfig\\\": {\\\"ReturnCode\\\": 0, \\\"SuccessfulReturnCodes\\\": []}, \\\"EnvironmentVariables\\\": {}, \\\"ApplicationEndpoints\\\": {}, \\\"Parameters\\\": []}, \\\"SnapshotId\\\": \\\"56c44205-9676-4c8a-9e38-129639e93ca7\\\", \\\"Snapshots\\\": [], \\\"SourceCodeDataReference\\\": null, \\\"ParentRunId\\\": null, \\\"DataContainerId\\\": null, \\\"RunType\\\": null, \\\"DisplayName\\\": null, \\\"EnvironmentAssetId\\\": null, \\\"Properties\\\": {}, \\\"Tags\\\": {}, \\\"AggregatedArtifactPath\\\": null}, \\\"ParentRunId\\\": \\\"HD_595634fe-649f-4c02-877d-b358149585fb\\\"}\", \"_aml_system_resume_child_runs\": \"null\", \"_aml_system_all_jobs_generated\": \"true\", \"_aml_system_cancellation_requested\": \"false\", \"_aml_system_progress_metadata_evaluation_timestamp\": \"\\\"2022-09-12T07:20:16.160271\\\"\", \"_aml_system_progress_metadata_digest\": \"\\\"b461d0bac4c68498bfb5c2f706e4f25edeedba43c865c7cd96be96f8a40925cc\\\"\", \"_aml_system_progress_metadata_active_timestamp\": \"\\\"2022-09-12T07:20:16.160271\\\"\", \"_aml_system_optimizer_state_artifact\": \"null\", \"_aml_system_outdated_optimizer_state_artifacts\": \"\\\"[]\\\"\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_0\": \"{\\\"C\\\": 0.01, \\\"max_iter\\\": 100}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_1\": \"{\\\"C\\\": 0.001, \\\"max_iter\\\": 50}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_2\": \"{\\\"C\\\": 0.1, \\\"max_iter\\\": 50}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_3\": \"{\\\"C\\\": 1, \\\"max_iter\\\": 100}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_4\": \"{\\\"C\\\": 0.1, \\\"max_iter\\\": 100}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_5\": \"{\\\"C\\\": 0.001, \\\"max_iter\\\": 100}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_6\": \"{\\\"C\\\": 1, \\\"max_iter\\\": 50}\", \"_aml_system_HD_595634fe-649f-4c02-877d-b358149585fb_7\": \"{\\\"C\\\": 0.01, \\\"max_iter\\\": 50}\", \"_aml_system_final_best_metric_update_retry_count\": \"1\"}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-09-12T07:26:48.694318Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/hyperdrive.txt\": \"https://mlstrg207125.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_595634fe-649f-4c02-877d-b358149585fb/azureml-logs/hyperdrive.txt?sv=2019-07-07&sr=b&sig=USBXJd7bAFwB%2F95mQ9K7brZM0p0vHDsnwZDvmUnXpb8%3D&skoid=19345451-8081-46cd-8160-586311b11bff&sktid=660b3398-b80e-49d2-bc5b-ac1dc93b5254&skt=2022-09-12T07%3A04%3A09Z&ske=2022-09-13T15%3A14%3A09Z&sks=b&skv=2019-07-07&st=2022-09-12T07%3A43%3A05Z&se=2022-09-12T15%3A53%3A05Z&sp=r\"}, \"log_groups\": [[\"azureml-logs/hyperdrive.txt\"]], \"run_duration\": \"0:06:33\", \"run_number\": \"1662967215\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}, \"hyper_parameters\": {\"C\": [\"choice\", [[0.01, 0.001, 0.1, 1]]], \"max_iter\": [\"choice\", [[50, 100]]]}}, \"child_runs\": [{\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_1\", \"run_number\": 1662967216, \"metric\": 0.90798738, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T07:23:29.506586Z\", \"end_time\": \"2022-09-12T07:24:44.903847Z\", \"created_time\": \"2022-09-12T07:20:16.899076Z\", \"created_time_dt\": \"2022-09-12T07:20:16.899076Z\", \"duration\": \"0:04:28\", \"hyperdrive_id\": \"595634fe-649f-4c02-877d-b358149585fb\", \"arguments\": null, \"param_C\": 0.001, \"param_max_iter\": 50, \"best_metric\": 0.90798738}, {\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_3\", \"run_number\": 1662967217, \"metric\": 0.9088371, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T07:23:29.121358Z\", \"end_time\": \"2022-09-12T07:24:44.37845Z\", \"created_time\": \"2022-09-12T07:20:17.079216Z\", \"created_time_dt\": \"2022-09-12T07:20:17.079216Z\", \"duration\": \"0:04:27\", \"hyperdrive_id\": \"595634fe-649f-4c02-877d-b358149585fb\", \"arguments\": null, \"param_C\": 1, \"param_max_iter\": 100, \"best_metric\": 0.9088371}, {\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_0\", \"run_number\": 1662967218, \"metric\": 0.90907987, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T07:23:29.506588Z\", \"end_time\": \"2022-09-12T07:24:46.148667Z\", \"created_time\": \"2022-09-12T07:20:18.423534Z\", \"created_time_dt\": \"2022-09-12T07:20:18.423534Z\", \"duration\": \"0:04:27\", \"hyperdrive_id\": \"595634fe-649f-4c02-877d-b358149585fb\", \"arguments\": null, \"param_C\": 0.01, \"param_max_iter\": 100, \"best_metric\": 0.90907987}, {\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_6\", \"run_number\": 1662967516, \"metric\": 0.90968682, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T07:25:28.322341Z\", \"end_time\": \"2022-09-12T07:25:47.110711Z\", \"created_time\": \"2022-09-12T07:25:16.939019Z\", \"created_time_dt\": \"2022-09-12T07:25:16.939019Z\", \"duration\": \"0:00:30\", \"hyperdrive_id\": \"595634fe-649f-4c02-877d-b358149585fb\", \"arguments\": null, \"param_C\": 1, \"param_max_iter\": 50, \"best_metric\": 0.90968682}, {\"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb_7\", \"run_number\": 1662967517, \"metric\": 0.90907987, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T07:25:28.67607Z\", \"end_time\": \"2022-09-12T07:25:47.403883Z\", \"created_time\": \"2022-09-12T07:25:17.048267Z\", \"created_time_dt\": \"2022-09-12T07:25:17.048267Z\", \"duration\": \"0:00:30\", \"hyperdrive_id\": \"595634fe-649f-4c02-877d-b358149585fb\", \"arguments\": null, \"param_C\": 0.01, \"param_max_iter\": 50, \"best_metric\": 0.90968682}], \"children_metrics\": {\"categories\": [0], \"series\": {\"Regularization Strength:\": [{\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"markers\", \"name\": \"Regularization Strength:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.001, 1.0, 0.01, 1.0, 0.01]}, {\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"lines\", \"name\": \"Regularization Strength:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.001, 1.0, 1.0, 1.0, 1.0]}], \"Max iterations:\": [{\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"markers\", \"name\": \"Max iterations:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [50, 100, 100, 50, 50]}, {\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"lines\", \"name\": \"Max iterations:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [50, 100, 100, 100, 100]}], \"Accuracy\": [{\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"markers\", \"name\": \"Accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.9079873755765963, 0.9088370963826171, 0.9090798737557659, 0.909686817188638, 0.9090798737557659]}, {\"categories\": [1662967216, 1662967217, 1662967218, 1662967516, 1662967517], \"mode\": \"lines\", \"name\": \"Accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.9079873755765963, 0.9088370963826171, 0.9090798737557659, 0.909686817188638, 0.909686817188638]}]}, \"metricName\": null, \"primaryMetricName\": \"Accuracy\", \"showLegend\": false}, \"run_metrics\": [{\"name\": \"best_child_by_primary_metric\", \"run_id\": \"HD_595634fe-649f-4c02-877d-b358149585fb\", \"categories\": [0], \"series\": [{\"data\": [{\"time_elapse\": [300, 391], \"metric_value\": [0.9094440398154892, 0.9100509832483612], \"metric_name\": [\"Accuracy\", \"Accuracy\"], \"run_id\": [\"HD_595634fe-649f-4c02-877d-b358149585fb_2\", \"HD_595634fe-649f-4c02-877d-b358149585fb_4\"], \"final\": [false, true]}]}]}], \"run_logs\": \"[2022-09-12T07:20:15.826930][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\\n[2022-09-12T07:20:16.6063199Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_0' \\n[2022-09-12T07:20:16.7339534Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_1' \\n[2022-09-12T07:20:16.8400476Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_2' \\n[2022-09-12T07:20:16.916349][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T07:20:16.9650886Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_3' \\n[2022-09-12T07:20:16.9954058Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_1' \\n[2022-09-12T07:20:17.0479606Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_2' \\n[2022-09-12T07:20:17.1568889Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_3' \\n[2022-09-12T07:20:18.5103084Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_0' \\n[2022-09-12T07:25:16.231250][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\\n[2022-09-12T07:25:16.5879126Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_4' \\n[2022-09-12T07:25:16.6628990Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_5' \\n[2022-09-12T07:25:16.8064120Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_6' \\n[2022-09-12T07:25:16.856328][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T07:25:16.9088160Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_7' \\n[2022-09-12T07:25:16.9888309Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_4' \\n[2022-09-12T07:25:17.0096264Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_6' \\n[2022-09-12T07:25:17.0612136Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_5' \\n[2022-09-12T07:25:17.1431175Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_7' \\n[2022-09-12T07:26:48.883460][CONTROLLER][INFO]Experiment was 'ExperimentStatus.RUNNING', is 'ExperimentStatus.FINISHED'.\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.44.0\"}, \"loading\": false}" + "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b?wsid=/subscriptions/f9d5a085-54dc-4215-9ba6-dad5d86e60a0/resourcegroups/aml-quickstarts-207163/workspaces/quick-starts-ws-207163&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\", \"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"run_properties\": {\"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"created_utc\": \"2022-09-12T10:08:51.003787Z\", \"properties\": {\"primary_metric_config\": \"{\\\"name\\\":\\\"Accuracy\\\",\\\"goal\\\":\\\"maximize\\\"}\", \"resume_from\": \"null\", \"runTemplate\": \"HyperDrive\", \"azureml.runsource\": \"hyperdrive\", \"platform\": \"AML\", \"ContentSnapshotId\": \"4e3b41bf-23f3-419b-b0d0-c3e9613125db\", \"user_agent\": \"python/3.8.5 (Linux-5.15.0-1017-azure-x86_64-with-glibc2.10) msrest/0.7.1 Hyperdrive.Service/1.0.0 Hyperdrive.SDK/core.1.44.0\", \"space_size\": \"8\", \"score\": \"0.9100509832483612\", \"best_child_run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\", \"best_metric_status\": \"Succeeded\", \"best_data_container_id\": \"dcid.HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\"}, \"tags\": {\"_aml_system_max_concurrent_jobs\": \"4\", \"_aml_system_max_total_jobs\": \"10\", \"_aml_system_max_duration_minutes\": \"10080\", \"_aml_system_policy_config\": \"{\\\"name\\\":\\\"Bandit\\\",\\\"properties\\\":{\\\"evaluation_interval\\\":1,\\\"delay_evaluation\\\":5,\\\"slack_factor\\\":0.1}}\", \"_aml_system_generator_config\": \"{\\\"name\\\":\\\"RANDOM\\\",\\\"parameter_space\\\":{\\\"C\\\":[\\\"choice\\\",[[0.01,0.001,0.1,1]]],\\\"max_iter\\\":[\\\"choice\\\",[[50,100]]]},\\\"properties\\\":null}\", \"_aml_system_primary_metric_config\": \"{\\\"name\\\":\\\"Accuracy\\\",\\\"goal\\\":\\\"maximize\\\"}\", \"_aml_system_platform_config\": \"{\\\"ServiceAddress\\\": \\\"https://southcentralus.experiments.azureml.net\\\", \\\"SubscriptionId\\\": \\\"f9d5a085-54dc-4215-9ba6-dad5d86e60a0\\\", \\\"ResourceGroupName\\\": \\\"aml-quickstarts-207163\\\", \\\"WorkspaceName\\\": \\\"quick-starts-ws-207163\\\", \\\"ExperimentName\\\": \\\"udacity-project\\\", \\\"Definition\\\": {\\\"Configuration\\\": null, \\\"Attribution\\\": null, \\\"TelemetryValues\\\": {\\\"amlClientType\\\": \\\"azureml-sdk-train\\\", \\\"amlClientModule\\\": \\\"[Scrubbed]\\\", \\\"amlClientFunction\\\": \\\"[Scrubbed]\\\", \\\"tenantId\\\": \\\"660b3398-b80e-49d2-bc5b-ac1dc93b5254\\\", \\\"amlClientRequestId\\\": \\\"340d0e41-99a3-4caf-a12b-ef1d66f2c390\\\", \\\"amlClientSessionId\\\": \\\"57e75686-c6c7-4d01-9c99-48460be30024\\\", \\\"subscriptionId\\\": \\\"f9d5a085-54dc-4215-9ba6-dad5d86e60a0\\\", \\\"estimator\\\": \\\"NoneType\\\", \\\"samplingMethod\\\": \\\"RANDOM\\\", \\\"terminationPolicy\\\": \\\"Bandit\\\", \\\"primaryMetricGoal\\\": \\\"maximize\\\", \\\"maxTotalRuns\\\": 10, \\\"maxConcurrentRuns\\\": 4, \\\"maxDurationMinutes\\\": 10080, \\\"vmSize\\\": null}, \\\"Overrides\\\": {\\\"Script\\\": \\\"train.py\\\", \\\"Command\\\": \\\"\\\", \\\"UseAbsolutePath\\\": false, \\\"Arguments\\\": [], \\\"SourceDirectoryDataStore\\\": null, \\\"Framework\\\": 0, \\\"Communicator\\\": 0, \\\"Target\\\": \\\"optimize-pipeline-cluster\\\", \\\"DataReferences\\\": {}, \\\"Data\\\": {}, \\\"OutputData\\\": {}, \\\"Datacaches\\\": [], \\\"JobName\\\": null, \\\"MaxRunDurationSeconds\\\": 2592000, \\\"NodeCount\\\": 1, \\\"InstanceTypes\\\": [], \\\"Priority\\\": null, \\\"CredentialPassthrough\\\": false, \\\"Identity\\\": null, \\\"Environment\\\": {\\\"Name\\\": \\\"sklearn-env\\\", \\\"AutoRebuild\\\": true, \\\"Python\\\": {\\\"InterpreterPath\\\": \\\"python\\\", \\\"UserManagedDependencies\\\": false, \\\"CondaDependencies\\\": {\\\"dependencies\\\": [\\\"python=3.6.2\\\", \\\"scikit-learn\\\", \\\"numpy\\\", \\\"pandas\\\", {\\\"pip\\\": [\\\"azureml-defaults\\\"]}]}, \\\"BaseCondaEnvironment\\\": null}, \\\"EnvironmentVariables\\\": {\\\"EXAMPLE_ENV_VAR\\\": \\\"EXAMPLE_VALUE\\\"}, \\\"Docker\\\": {\\\"BaseImage\\\": \\\"mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220708.v1\\\", \\\"Platform\\\": {\\\"Os\\\": \\\"Linux\\\", \\\"Architecture\\\": \\\"amd64\\\"}, \\\"BaseDockerfile\\\": null, \\\"BaseImageRegistry\\\": {\\\"Address\\\": null, \\\"Username\\\": null, \\\"Password\\\": null}, \\\"Enabled\\\": false, \\\"Arguments\\\": []}, \\\"Spark\\\": {\\\"Repositories\\\": [], \\\"Packages\\\": [], \\\"PrecachePackages\\\": true}, \\\"InferencingStackVersion\\\": null}, \\\"History\\\": {\\\"OutputCollection\\\": true, \\\"DirectoriesToWatch\\\": [\\\"logs\\\"], \\\"EnableMLflowTracking\\\": true, \\\"snapshotProject\\\": true}, \\\"Spark\\\": {\\\"Configuration\\\": {\\\"spark.app.name\\\": \\\"Azure ML Experiment\\\", \\\"spark.yarn.maxAppAttempts\\\": \\\"1\\\"}}, \\\"ParallelTask\\\": {\\\"MaxRetriesPerWorker\\\": 0, \\\"WorkerCountPerNode\\\": 1, \\\"TerminalExitCodes\\\": null, \\\"Configuration\\\": {}}, \\\"BatchAi\\\": {\\\"NodeCount\\\": 0}, \\\"AmlCompute\\\": {\\\"Name\\\": null, \\\"VmSize\\\": null, \\\"RetainCluster\\\": false, \\\"ClusterMaxNodeCount\\\": null}, \\\"AISuperComputer\\\": {\\\"InstanceType\\\": \\\"D2\\\", \\\"FrameworkImage\\\": null, \\\"ImageVersion\\\": \\\"pytorch-1.7.0\\\", \\\"Location\\\": null, \\\"AISuperComputerStorageData\\\": null, \\\"Interactive\\\": false, \\\"ScalePolicy\\\": null, \\\"VirtualClusterArmId\\\": null, \\\"TensorboardLogDirectory\\\": null, \\\"SSHPublicKey\\\": null, \\\"SSHPublicKeys\\\": null, \\\"EnableAzmlInt\\\": true, \\\"Priority\\\": \\\"Medium\\\", \\\"SLATier\\\": \\\"Standard\\\", \\\"UserAlias\\\": null}, \\\"KubernetesCompute\\\": {\\\"InstanceType\\\": null}, \\\"Tensorflow\\\": {\\\"WorkerCount\\\": 1, \\\"ParameterServerCount\\\": 1}, \\\"Mpi\\\": {\\\"ProcessCountPerNode\\\": 1}, \\\"PyTorch\\\": {\\\"CommunicationBackend\\\": \\\"nccl\\\", \\\"ProcessCount\\\": null}, \\\"Hdi\\\": {\\\"YarnDeployMode\\\": 2}, \\\"ContainerInstance\\\": {\\\"Region\\\": null, \\\"CpuCores\\\": 2.0, \\\"MemoryGb\\\": 3.5}, \\\"ExposedPorts\\\": null, \\\"Docker\\\": {\\\"UseDocker\\\": false, \\\"SharedVolumes\\\": true, \\\"ShmSize\\\": \\\"2g\\\", \\\"Arguments\\\": []}, \\\"Cmk8sCompute\\\": {\\\"Configuration\\\": {}}, \\\"CommandReturnCodeConfig\\\": {\\\"ReturnCode\\\": 0, \\\"SuccessfulReturnCodes\\\": []}, \\\"EnvironmentVariables\\\": {}, \\\"ApplicationEndpoints\\\": {}, \\\"Parameters\\\": []}, \\\"SnapshotId\\\": \\\"4e3b41bf-23f3-419b-b0d0-c3e9613125db\\\", \\\"Snapshots\\\": [], \\\"SourceCodeDataReference\\\": null, \\\"ParentRunId\\\": null, \\\"DataContainerId\\\": null, \\\"RunType\\\": null, \\\"DisplayName\\\": null, \\\"EnvironmentAssetId\\\": null, \\\"Properties\\\": {}, \\\"Tags\\\": {}, \\\"AggregatedArtifactPath\\\": null}, \\\"ParentRunId\\\": \\\"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\\\"}\", \"_aml_system_resume_child_runs\": \"null\", \"_aml_system_all_jobs_generated\": \"true\", \"_aml_system_cancellation_requested\": \"false\", \"_aml_system_progress_metadata_evaluation_timestamp\": \"\\\"2022-09-12T10:08:51.636454\\\"\", \"_aml_system_progress_metadata_digest\": \"\\\"c6d2e237756b2611b16c5d416126eeb61adb842b8691c2d89823d1fc9f7a9378\\\"\", \"_aml_system_progress_metadata_active_timestamp\": \"\\\"2022-09-12T10:08:51.636454\\\"\", \"_aml_system_optimizer_state_artifact\": \"null\", \"_aml_system_outdated_optimizer_state_artifacts\": \"\\\"[]\\\"\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0\": \"{\\\"C\\\": 1, \\\"max_iter\\\": 50}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\": \"{\\\"C\\\": 0.1, \\\"max_iter\\\": 100}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2\": \"{\\\"C\\\": 0.01, \\\"max_iter\\\": 100}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_3\": \"{\\\"C\\\": 0.01, \\\"max_iter\\\": 50}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4\": \"{\\\"C\\\": 0.001, \\\"max_iter\\\": 50}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5\": \"{\\\"C\\\": 0.001, \\\"max_iter\\\": 100}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_6\": \"{\\\"C\\\": 1, \\\"max_iter\\\": 100}\", \"_aml_system_HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7\": \"{\\\"C\\\": 0.1, \\\"max_iter\\\": 50}\", \"_aml_system_final_best_metric_update_retry_count\": \"1\"}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-09-12T10:29:00.241011Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/hyperdrive.txt\": \"https://mlstrg207163.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b/azureml-logs/hyperdrive.txt?sv=2019-07-07&sr=b&sig=bu9nUwW5vg5r2c2rkKFKwBcCZh4umm9mx8krwJwEfEU%3D&skoid=60cd9028-d8f1-4bf3-ac06-c886a38b0737&sktid=660b3398-b80e-49d2-bc5b-ac1dc93b5254&skt=2022-09-12T09%3A58%3A54Z&ske=2022-09-13T18%3A08%3A54Z&sks=b&skv=2019-07-07&st=2022-09-12T10%3A46%3A02Z&se=2022-09-12T18%3A56%3A02Z&sp=r\"}, \"log_groups\": [[\"azureml-logs/hyperdrive.txt\"]], \"run_duration\": \"0:20:09\", \"run_number\": \"1662977331\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}, \"hyper_parameters\": {\"C\": [\"choice\", [[0.01, 0.001, 0.1, 1]]], \"max_iter\": [\"choice\", [[50, 100]]]}}, \"child_runs\": [{\"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0\", \"run_number\": 1662977333, \"metric\": 0.90968682, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T10:23:08.700325Z\", \"end_time\": \"2022-09-12T10:27:10.211336Z\", \"created_time\": \"2022-09-12T10:08:53.423159Z\", \"created_time_dt\": \"2022-09-12T10:08:53.423159Z\", \"duration\": \"0:18:16\", \"hyperdrive_id\": \"9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"arguments\": null, \"param_C\": 1, \"param_max_iter\": 50, \"best_metric\": 0.90968682}, {\"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4\", \"run_number\": 1662978322, \"metric\": 0.90798738, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T10:25:29.056443Z\", \"end_time\": \"2022-09-12T10:25:49.882757Z\", \"created_time\": \"2022-09-12T10:25:22.979133Z\", \"created_time_dt\": \"2022-09-12T10:25:22.979133Z\", \"duration\": \"0:00:26\", \"hyperdrive_id\": \"9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"arguments\": null, \"param_C\": 0.001, \"param_max_iter\": 50, \"best_metric\": 0.90968682}, {\"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5\", \"run_number\": 1662978382, \"metric\": 0.90810876, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T10:26:28.009405Z\", \"end_time\": \"2022-09-12T10:26:47.798172Z\", \"created_time\": \"2022-09-12T10:26:22.635566Z\", \"created_time_dt\": \"2022-09-12T10:26:22.635566Z\", \"duration\": \"0:00:25\", \"hyperdrive_id\": \"9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"arguments\": null, \"param_C\": 0.001, \"param_max_iter\": 100, \"best_metric\": 0.90968682}, {\"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7\", \"run_number\": 1662978442, \"metric\": 0.90944404, \"status\": \"Completed\", \"run_type\": \"azureml.scriptrun\", \"training_percent\": null, \"start_time\": \"2022-09-12T10:27:32.271441Z\", \"end_time\": \"2022-09-12T10:27:54.440109Z\", \"created_time\": \"2022-09-12T10:27:22.836662Z\", \"created_time_dt\": \"2022-09-12T10:27:22.836662Z\", \"duration\": \"0:00:31\", \"hyperdrive_id\": \"9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"arguments\": null, \"param_C\": 0.1, \"param_max_iter\": 50, \"best_metric\": 0.90968682}], \"children_metrics\": {\"categories\": [0], \"series\": {\"Regularization Strength:\": [{\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"markers\", \"name\": \"Regularization Strength:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [1.0, 0.001, 0.001, 0.1]}, {\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"lines\", \"name\": \"Regularization Strength:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [1.0, 1.0, 1.0, 1.0]}], \"Max iterations:\": [{\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"markers\", \"name\": \"Max iterations:\", \"stepped\": false, \"type\": \"scatter\", \"data\": [50, 50, 100, 50]}, {\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"lines\", \"name\": \"Max iterations:_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [50, 50, 100, 100]}], \"Accuracy\": [{\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"markers\", \"name\": \"Accuracy\", \"stepped\": false, \"type\": \"scatter\", \"data\": [0.909686817188638, 0.9079873755765963, 0.9081087642631707, 0.9094440398154892]}, {\"categories\": [1662977333, 1662978322, 1662978382, 1662978442], \"mode\": \"lines\", \"name\": \"Accuracy_max\", \"stepped\": true, \"type\": \"scatter\", \"data\": [0.909686817188638, 0.909686817188638, 0.909686817188638, 0.909686817188638]}]}, \"metricName\": null, \"primaryMetricName\": \"Accuracy\", \"showLegend\": false}, \"run_metrics\": [{\"name\": \"best_child_by_primary_metric\", \"run_id\": \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\", \"categories\": [0], \"series\": [{\"data\": [{\"time_elapse\": [963, 1114, 1114], \"metric_value\": [0.9090798737557659, 0.9100509832483612, 0.9100509832483612], \"metric_name\": [\"Accuracy\", \"Accuracy\", \"Accuracy\"], \"run_id\": [\"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2\", \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\", \"HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\"], \"final\": [false, false, true]}]}]}], \"run_logs\": \"[2022-09-12T10:08:51.736473][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\\n[2022-09-12T10:08:52.5267455Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0' \\n[2022-09-12T10:08:52.6485848Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1' \\n[2022-09-12T10:08:52.7690083Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2' \\n[2022-09-12T10:08:52.828224][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T10:08:52.8812340Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_3' \\n[2022-09-12T10:08:53.1482890Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1' \\n[2022-09-12T10:08:53.1592295Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_3' \\n[2022-09-12T10:08:53.2249787Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2' \\n[2022-09-12T10:08:53.5098252Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0' \\n[2022-09-12T10:25:22.257033][GENERATOR][INFO]Trying to sample '1' jobs from the hyperparameter space\\n[2022-09-12T10:25:22.5354523Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4' \\n[2022-09-12T10:25:22.490636][GENERATOR][INFO]Successfully sampled '1' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T10:25:23.0595703Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4' \\n[2022-09-12T10:26:22.227469][GENERATOR][INFO]Trying to sample '1' jobs from the hyperparameter space\\n[2022-09-12T10:26:22.474906][GENERATOR][INFO]Successfully sampled '1' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T10:26:22.5022966Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5' \\n[2022-09-12T10:26:22.7083880Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5' \\n[2022-09-12T10:27:22.212399][GENERATOR][INFO]Trying to sample '2' jobs from the hyperparameter space\\n[2022-09-12T10:27:22.5826384Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_6' \\n[2022-09-12T10:27:22.7023959Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7' \\n[2022-09-12T10:27:22.659250][GENERATOR][INFO]Successfully sampled '2' jobs, they will soon be submitted to the execution target.\\n[2022-09-12T10:27:22.7994695Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_6' \\n[2022-09-12T10:27:22.9155070Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7' \\n[2022-09-12T10:27:52.193293][GENERATOR][INFO]Trying to sample '2' jobs from the hyperparameter space\\n[2022-09-12T10:27:52.250995][GENERATOR][WARNING]Could not sample any more jobs from the space.\\n[2022-09-12T10:29:00.427172][CONTROLLER][INFO]Experiment was 'ExperimentStatus.RUNNING', is 'ExperimentStatus.FINISHED'.\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.44.0\"}, \"loading\": false}" }, "metadata": {} }, { "output_type": "stream", "name": "stdout", - "text": "RunId: HD_595634fe-649f-4c02-877d-b358149585fb\nWeb View: https://ml.azure.com/runs/HD_595634fe-649f-4c02-877d-b358149585fb?wsid=/subscriptions/976ee174-3882-4721-b90a-b5fef6b72f24/resourcegroups/aml-quickstarts-207125/workspaces/quick-starts-ws-207125&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n\nStreaming azureml-logs/hyperdrive.txt\n=====================================\n\n[2022-09-12T07:20:15.826930][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\n[2022-09-12T07:20:16.6063199Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_0' \n[2022-09-12T07:20:16.7339534Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_1' \n[2022-09-12T07:20:16.8400476Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_2' \n[2022-09-12T07:20:16.916349][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\n[2022-09-12T07:20:16.9650886Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_3' \n[2022-09-12T07:20:16.9954058Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_1' \n[2022-09-12T07:20:17.0479606Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_2' \n[2022-09-12T07:20:17.1568889Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_3' \n[2022-09-12T07:20:18.5103084Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_0' \n[2022-09-12T07:25:16.231250][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\n[2022-09-12T07:25:16.5879126Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_4' \n[2022-09-12T07:25:16.6628990Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_5' \n[2022-09-12T07:25:16.8064120Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_6' \n[2022-09-12T07:25:16.856328][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\n[2022-09-12T07:25:16.9088160Z][SCHEDULER][INFO]Scheduling job, id='HD_595634fe-649f-4c02-877d-b358149585fb_7' \n[2022-09-12T07:25:16.9888309Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_4' \n[2022-09-12T07:25:17.0096264Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_6' \n[2022-09-12T07:25:17.0612136Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_5' \n[2022-09-12T07:25:17.1431175Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_595634fe-649f-4c02-877d-b358149585fb_7' \n[2022-09-12T07:26:48.883460][CONTROLLER][INFO]Experiment was 'ExperimentStatus.RUNNING', is 'ExperimentStatus.FINISHED'.\n\nExecution Summary\n=================\nRunId: HD_595634fe-649f-4c02-877d-b358149585fb\nWeb View: https://ml.azure.com/runs/HD_595634fe-649f-4c02-877d-b358149585fb?wsid=/subscriptions/976ee174-3882-4721-b90a-b5fef6b72f24/resourcegroups/aml-quickstarts-207125/workspaces/quick-starts-ws-207125&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n\n" + "text": "RunId: HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\nWeb View: https://ml.azure.com/runs/HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b?wsid=/subscriptions/f9d5a085-54dc-4215-9ba6-dad5d86e60a0/resourcegroups/aml-quickstarts-207163/workspaces/quick-starts-ws-207163&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n\nStreaming azureml-logs/hyperdrive.txt\n=====================================\n\n[2022-09-12T10:08:51.736473][GENERATOR][INFO]Trying to sample '4' jobs from the hyperparameter space\n[2022-09-12T10:08:52.5267455Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0' \n[2022-09-12T10:08:52.6485848Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1' \n[2022-09-12T10:08:52.7690083Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2' \n[2022-09-12T10:08:52.828224][GENERATOR][INFO]Successfully sampled '4' jobs, they will soon be submitted to the execution target.\n[2022-09-12T10:08:52.8812340Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_3' \n[2022-09-12T10:08:53.1482890Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1' \n[2022-09-12T10:08:53.1592295Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_3' \n[2022-09-12T10:08:53.2249787Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_2' \n[2022-09-12T10:08:53.5098252Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_0' \n[2022-09-12T10:25:22.257033][GENERATOR][INFO]Trying to sample '1' jobs from the hyperparameter space\n[2022-09-12T10:25:22.5354523Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4' \n[2022-09-12T10:25:22.490636][GENERATOR][INFO]Successfully sampled '1' jobs, they will soon be submitted to the execution target.\n[2022-09-12T10:25:23.0595703Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_4' \n[2022-09-12T10:26:22.227469][GENERATOR][INFO]Trying to sample '1' jobs from the hyperparameter space\n[2022-09-12T10:26:22.474906][GENERATOR][INFO]Successfully sampled '1' jobs, they will soon be submitted to the execution target.\n[2022-09-12T10:26:22.5022966Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5' \n[2022-09-12T10:26:22.7083880Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_5' \n[2022-09-12T10:27:22.212399][GENERATOR][INFO]Trying to sample '2' jobs from the hyperparameter space\n[2022-09-12T10:27:22.5826384Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_6' \n[2022-09-12T10:27:22.7023959Z][SCHEDULER][INFO]Scheduling job, id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7' \n[2022-09-12T10:27:22.659250][GENERATOR][INFO]Successfully sampled '2' jobs, they will soon be submitted to the execution target.\n[2022-09-12T10:27:22.7994695Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_6' \n[2022-09-12T10:27:22.9155070Z][SCHEDULER][INFO]Successfully scheduled a job. Id='HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_7' \n[2022-09-12T10:27:52.193293][GENERATOR][INFO]Trying to sample '2' jobs from the hyperparameter space\n[2022-09-12T10:27:52.250995][GENERATOR][WARNING]Could not sample any more jobs from the space.\n[2022-09-12T10:29:00.427172][CONTROLLER][INFO]Experiment was 'ExperimentStatus.RUNNING', is 'ExperimentStatus.FINISHED'.\n\nExecution Summary\n=================\nRunId: HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b\nWeb View: https://ml.azure.com/runs/HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b?wsid=/subscriptions/f9d5a085-54dc-4215-9ba6-dad5d86e60a0/resourcegroups/aml-quickstarts-207163/workspaces/quick-starts-ws-207163&tid=660b3398-b80e-49d2-bc5b-ac1dc93b5254\n\n" }, { "output_type": "execute_result", "execution_count": 5, "data": { - "text/plain": "{'runId': 'HD_595634fe-649f-4c02-877d-b358149585fb',\n 'target': 'optimize-pipeline-cluster',\n 'status': 'Completed',\n 'startTimeUtc': '2022-09-12T07:20:15.174547Z',\n 'endTimeUtc': '2022-09-12T07:26:48.694318Z',\n 'services': {},\n 'properties': {'primary_metric_config': '{\"name\":\"Accuracy\",\"goal\":\"maximize\"}',\n 'resume_from': 'null',\n 'runTemplate': 'HyperDrive',\n 'azureml.runsource': 'hyperdrive',\n 'platform': 'AML',\n 'ContentSnapshotId': '56c44205-9676-4c8a-9e38-129639e93ca7',\n 'user_agent': 'python/3.8.5 (Linux-5.15.0-1017-azure-x86_64-with-glibc2.10) msrest/0.7.1 Hyperdrive.Service/1.0.0 Hyperdrive.SDK/core.1.44.0',\n 'space_size': '8',\n 'score': '0.9100509832483612',\n 'best_child_run_id': 'HD_595634fe-649f-4c02-877d-b358149585fb_4',\n 'best_metric_status': 'Succeeded',\n 'best_data_container_id': 'dcid.HD_595634fe-649f-4c02-877d-b358149585fb_4'},\n 'inputDatasets': [],\n 'outputDatasets': [],\n 'runDefinition': {'configuration': None,\n 'attribution': None,\n 'telemetryValues': {'amlClientType': 'azureml-sdk-train',\n 'amlClientModule': '[Scrubbed]',\n 'amlClientFunction': '[Scrubbed]',\n 'tenantId': '660b3398-b80e-49d2-bc5b-ac1dc93b5254',\n 'amlClientRequestId': 'a95be214-a556-458d-a450-ec0bd6ef72f7',\n 'amlClientSessionId': '6eae51e4-f234-4e3f-aa70-2434c0c7cdc2',\n 'subscriptionId': '976ee174-3882-4721-b90a-b5fef6b72f24',\n 'estimator': 'NoneType',\n 'samplingMethod': 'RANDOM',\n 'terminationPolicy': 'Bandit',\n 'primaryMetricGoal': 'maximize',\n 'maxTotalRuns': 10,\n 'maxConcurrentRuns': 4,\n 'maxDurationMinutes': 10080,\n 'vmSize': None},\n 'snapshotId': '56c44205-9676-4c8a-9e38-129639e93ca7',\n 'snapshots': [],\n 'sourceCodeDataReference': None,\n 'parentRunId': None,\n 'dataContainerId': None,\n 'runType': None,\n 'displayName': None,\n 'environmentAssetId': None,\n 'properties': {},\n 'tags': {},\n 'aggregatedArtifactPath': None},\n 'logFiles': {'azureml-logs/hyperdrive.txt': 'https://mlstrg207125.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_595634fe-649f-4c02-877d-b358149585fb/azureml-logs/hyperdrive.txt?sv=2019-07-07&sr=b&sig=%2FOEi4Yx%2FZ0ejHLwuKgJT1sFp61FN6cg1Xs81bh%2F4U%2Bk%3D&skoid=19345451-8081-46cd-8160-586311b11bff&sktid=660b3398-b80e-49d2-bc5b-ac1dc93b5254&skt=2022-09-12T07%3A04%3A09Z&ske=2022-09-13T15%3A14%3A09Z&sks=b&skv=2019-07-07&st=2022-09-12T07%3A16%3A54Z&se=2022-09-12T15%3A26%3A54Z&sp=r'},\n 'submittedBy': 'ODL_User 207125'}" + "text/plain": "{'runId': 'HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b',\n 'target': 'optimize-pipeline-cluster',\n 'status': 'Completed',\n 'startTimeUtc': '2022-09-12T10:08:51.058504Z',\n 'endTimeUtc': '2022-09-12T10:29:00.241011Z',\n 'services': {},\n 'properties': {'primary_metric_config': '{\"name\":\"Accuracy\",\"goal\":\"maximize\"}',\n 'resume_from': 'null',\n 'runTemplate': 'HyperDrive',\n 'azureml.runsource': 'hyperdrive',\n 'platform': 'AML',\n 'ContentSnapshotId': '4e3b41bf-23f3-419b-b0d0-c3e9613125db',\n 'user_agent': 'python/3.8.5 (Linux-5.15.0-1017-azure-x86_64-with-glibc2.10) msrest/0.7.1 Hyperdrive.Service/1.0.0 Hyperdrive.SDK/core.1.44.0',\n 'space_size': '8',\n 'score': '0.9100509832483612',\n 'best_child_run_id': 'HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1',\n 'best_metric_status': 'Succeeded',\n 'best_data_container_id': 'dcid.HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1'},\n 'inputDatasets': [],\n 'outputDatasets': [],\n 'runDefinition': {'configuration': None,\n 'attribution': None,\n 'telemetryValues': {'amlClientType': 'azureml-sdk-train',\n 'amlClientModule': '[Scrubbed]',\n 'amlClientFunction': '[Scrubbed]',\n 'tenantId': '660b3398-b80e-49d2-bc5b-ac1dc93b5254',\n 'amlClientRequestId': '340d0e41-99a3-4caf-a12b-ef1d66f2c390',\n 'amlClientSessionId': '57e75686-c6c7-4d01-9c99-48460be30024',\n 'subscriptionId': 'f9d5a085-54dc-4215-9ba6-dad5d86e60a0',\n 'estimator': 'NoneType',\n 'samplingMethod': 'RANDOM',\n 'terminationPolicy': 'Bandit',\n 'primaryMetricGoal': 'maximize',\n 'maxTotalRuns': 10,\n 'maxConcurrentRuns': 4,\n 'maxDurationMinutes': 10080,\n 'vmSize': None},\n 'snapshotId': '4e3b41bf-23f3-419b-b0d0-c3e9613125db',\n 'snapshots': [],\n 'sourceCodeDataReference': None,\n 'parentRunId': None,\n 'dataContainerId': None,\n 'runType': None,\n 'displayName': None,\n 'environmentAssetId': None,\n 'properties': {},\n 'tags': {},\n 'aggregatedArtifactPath': None},\n 'logFiles': {'azureml-logs/hyperdrive.txt': 'https://mlstrg207163.blob.core.windows.net/azureml/ExperimentRun/dcid.HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b/azureml-logs/hyperdrive.txt?sv=2019-07-07&sr=b&sig=BTkYQ%2BA0KBgs8vTTQLA%2BJBSkFkn6reBZN%2FLfEbQWkVM%3D&skoid=60cd9028-d8f1-4bf3-ac06-c886a38b0737&sktid=660b3398-b80e-49d2-bc5b-ac1dc93b5254&skt=2022-09-12T09%3A58%3A54Z&ske=2022-09-13T18%3A08%3A54Z&sks=b&skv=2019-07-07&st=2022-09-12T10%3A19%3A03Z&se=2022-09-12T18%3A29%3A03Z&sp=r'},\n 'submittedBy': 'ODL_User 207163'}" }, "metadata": {} - }, - { - "output_type": "error", - "ename": "KeyError", - "evalue": "'log_files'", - "traceback": [ - "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/ipywidgets/widgets/widget.py:756\u001b[0m, in \u001b[0;36mWidget._handle_msg\u001b[0;34m(self, msg)\u001b[0m\n\u001b[1;32m 754\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbuffer_paths\u001b[39m\u001b[38;5;124m'\u001b[39m \u001b[38;5;129;01min\u001b[39;00m data:\n\u001b[1;32m 755\u001b[0m _put_buffers(state, data[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbuffer_paths\u001b[39m\u001b[38;5;124m'\u001b[39m], msg[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbuffers\u001b[39m\u001b[38;5;124m'\u001b[39m])\n\u001b[0;32m--> 756\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mset_state\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstate\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 758\u001b[0m \u001b[38;5;66;03m# Handle a state request.\u001b[39;00m\n\u001b[1;32m 759\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m method \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mrequest_state\u001b[39m\u001b[38;5;124m'\u001b[39m:\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/ipywidgets/widgets/widget.py:625\u001b[0m, in \u001b[0;36mWidget.set_state\u001b[0;34m(self, sync_data)\u001b[0m\n\u001b[1;32m 622\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m name \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mkeys:\n\u001b[1;32m 623\u001b[0m from_json \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtrait_metadata(name, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mfrom_json\u001b[39m\u001b[38;5;124m'\u001b[39m,\n\u001b[1;32m 624\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_trait_from_json)\n\u001b[0;32m--> 625\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mset_trait(name, from_json(sync_data[name], \u001b[38;5;28mself\u001b[39m))\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/contextlib.py:120\u001b[0m, in \u001b[0;36m_GeneratorContextManager.__exit__\u001b[0;34m(self, type, value, traceback)\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mtype\u001b[39m \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 119\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 120\u001b[0m \u001b[38;5;28;43mnext\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mgen\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 121\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mStopIteration\u001b[39;00m:\n\u001b[1;32m 122\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;01mFalse\u001b[39;00m\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/traitlets/traitlets.py:1371\u001b[0m, in \u001b[0;36mHasTraits.hold_trait_notifications\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 1369\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m changes \u001b[38;5;129;01min\u001b[39;00m cache\u001b[38;5;241m.\u001b[39mvalues():\n\u001b[1;32m 1370\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m change \u001b[38;5;129;01min\u001b[39;00m changes:\n\u001b[0;32m-> 1371\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnotify_change\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchange\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/ipywidgets/widgets/widget.py:686\u001b[0m, in \u001b[0;36mWidget.notify_change\u001b[0;34m(self, change)\u001b[0m\n\u001b[1;32m 683\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m name \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mkeys \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_should_send_property(name, \u001b[38;5;28mgetattr\u001b[39m(\u001b[38;5;28mself\u001b[39m, name)):\n\u001b[1;32m 684\u001b[0m \u001b[38;5;66;03m# Send new state to front-end\u001b[39;00m\n\u001b[1;32m 685\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39msend_state(key\u001b[38;5;241m=\u001b[39mname)\n\u001b[0;32m--> 686\u001b[0m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mWidget\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnotify_change\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchange\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/traitlets/traitlets.py:1386\u001b[0m, in \u001b[0;36mHasTraits.notify_change\u001b[0;34m(self, change)\u001b[0m\n\u001b[1;32m 1384\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mnotify_change\u001b[39m(\u001b[38;5;28mself\u001b[39m, change):\n\u001b[1;32m 1385\u001b[0m \u001b[38;5;124;03m\"\"\"Notify observers of a change event\"\"\"\u001b[39;00m\n\u001b[0;32m-> 1386\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_notify_observers\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchange\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/traitlets/traitlets.py:1431\u001b[0m, in \u001b[0;36mHasTraits._notify_observers\u001b[0;34m(self, event)\u001b[0m\n\u001b[1;32m 1428\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(c, EventHandler) \u001b[38;5;129;01mand\u001b[39;00m c\u001b[38;5;241m.\u001b[39mname \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 1429\u001b[0m c \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mgetattr\u001b[39m(\u001b[38;5;28mself\u001b[39m, c\u001b[38;5;241m.\u001b[39mname)\n\u001b[0;32m-> 1431\u001b[0m \u001b[43mc\u001b[49m\u001b[43m(\u001b[49m\u001b[43mevent\u001b[49m\u001b[43m)\u001b[49m\n", - "File \u001b[0;32m/anaconda/envs/azureml_py38/lib/python3.8/site-packages/azureml/widgets/_userrun/_run_details.py:627\u001b[0m, in \u001b[0;36m_UserRunDetails._on_selected_run_log_change\u001b[0;34m(self, change)\u001b[0m\n\u001b[1;32m 625\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_on_selected_run_log_change\u001b[39m(\u001b[38;5;28mself\u001b[39m, change):\n\u001b[1;32m 626\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mselected_run_log \u001b[38;5;241m=\u001b[39m change\u001b[38;5;241m.\u001b[39mnew\n\u001b[0;32m--> 627\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_get_run_logs_async(\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mwidget_instance\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_properties\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mlog_files\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m,\n\u001b[1;32m 628\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mwidget_instance\u001b[38;5;241m.\u001b[39mrun_properties[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstatus\u001b[39m\u001b[38;5;124m'\u001b[39m],\n\u001b[1;32m 629\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39merror, change\u001b[38;5;241m.\u001b[39mnew)\n", - "\u001b[0;31mKeyError\u001b[0m: 'log_files'" - ] } ], "execution_count": 5, "metadata": { + "gather": { + "logged": 1662978569143 + }, "jupyter": { - "source_hidden": false, - "outputs_hidden": false + "outputs_hidden": false, + "source_hidden": false }, "nteract": { "transient": { "deleting": false } - }, - "gather": { - "logged": 1662967614920 } } }, @@ -256,13 +232,13 @@ { "output_type": "stream", "name": "stdout", - "text": "Best Run Id: HD_595634fe-649f-4c02-877d-b358149585fb_4\n\n Best Accuracy: 0.9100509832483612\n" + "text": "Best Run Id: HD_9b05eee2-7913-4c8d-9016-80784dc0ad7b_1\n\n Best Accuracy: 0.9100509832483612\n" } ], "execution_count": 6, "metadata": { "gather": { - "logged": 1662967616807 + "logged": 1662978573399 }, "jupyter": { "outputs_hidden": false, @@ -303,7 +279,7 @@ "execution_count": 7, "metadata": { "gather": { - "logged": 1662967624244 + "logged": 1662978580738 } } }, @@ -328,7 +304,7 @@ "execution_count": 8, "metadata": { "gather": { - "logged": 1662967624303 + "logged": 1662978580783 }, "jupyter": { "outputs_hidden": false, @@ -357,23 +333,26 @@ "output_type": "display_data", "data": { "text/plain": "", - "text/html": "
ExperimentIdTypeStatusDetails PageDocs Page
udacity-projectAutoML_00678b7e-98fc-49a0-93bd-cd708443687bautomlPreparingLink to Azure Machine Learning studioLink to Documentation
" + "text/html": "
ExperimentIdTypeStatusDetails PageDocs Page
udacity-projectAutoML_ab7fc1d6-ed22-4d6c-b282-3bce39da90d0automlPreparingLink to Azure Machine Learning studioLink to Documentation
" }, "metadata": {} }, { "output_type": "stream", "name": "stdout", - "text": "Current status: DatasetEvaluation. Gathering dataset statistics.\nCurrent status: FeaturesGeneration. Generating features for the dataset.\nCurrent status: DatasetFeaturization. Beginning to fit featurizers and featurize the dataset.\nCurrent status: DatasetFeaturizationCompleted. Completed fit featurizers and featurizing the dataset.\nCurrent status: DatasetBalancing. Performing class balancing sweeping\nCurrent status: DatasetCrossValidationSplit. Generating individually featurized CV splits.\n\n********************************************************************************************\nDATA GUARDRAILS: \n\nTYPE: Class balancing detection\nSTATUS: ALERTED\nDESCRIPTION: To decrease model bias, please cancel the current run and fix balancing problem.\n Learn more about imbalanced data: https://aka.ms/AutomatedMLImbalancedData\nDETAILS: Imbalanced data can lead to a falsely perceived positive effect of a model's accuracy because the input data has bias towards one class.\n+------------------------------+--------------------------------+--------------------------------------+\n|Size of the smallest class |Name/Label of the smallest class|Number of samples in the training data|\n+==============================+================================+======================================+\n|3692 |yes |32950 |\n+------------------------------+--------------------------------+--------------------------------------+\n\n********************************************************************************************\n\nTYPE: Missing feature values imputation\nSTATUS: PASSED\nDESCRIPTION: No feature missing values were detected in the training data.\n Learn more about missing value imputation: https://aka.ms/AutomatedMLFeaturization\n\n********************************************************************************************\n\nTYPE: High cardinality feature detection\nSTATUS: PASSED\nDESCRIPTION: Your inputs were analyzed, and no high cardinality features were detected.\n Learn more about high cardinality feature handling: https://aka.ms/AutomatedMLFeaturization\n\n********************************************************************************************\nCurrent status: ModelSelection. Beginning model selection.\n\n********************************************************************************************\nITER: The iteration being evaluated.\nPIPELINE: A summary description of the pipeline being evaluated.\nDURATION: Time taken for the current iteration.\nMETRIC: The result of computing score on the fitted pipeline.\nBEST: The best observed score thus far.\n********************************************************************************************\n\n ITER PIPELINE DURATION METRIC BEST\n 0 MaxAbsScaler LightGBM 0:00:32 0.9125 0.9125\n 1 MaxAbsScaler XGBoostClassifier 0:00:38 0.9110 0.9125\n 2 MaxAbsScaler ExtremeRandomTrees 0:00:32 0.7319 0.9125\n 3 SparseNormalizer XGBoostClassifier 0:00:34 0.9132 0.9132\n 4 MaxAbsScaler LightGBM 0:00:29 0.9123 0.9132\n 5 MaxAbsScaler SGD 0:00:29 0.8608 0.9134\n 14 StandardScalerWrapper XGBoostClassifier 0:00:34 0.9127 0.9134\n 15 SparseNormalizer RandomForest 0:00:40 0.8172 0.9134\n 16 StandardScalerWrapper LogisticRegression 0:00:33 0.9084 0.9134\n 17 StandardScalerWrapper RandomForest 0:00:35 0.9015 0.9134\n 18 StandardScalerWrapper XGBoostClassifier 0:00:36 0.9124 0.9134\n 19 TruncatedSVDWrapper RandomForest 0:01:34 0.8230 0.9134\n 20 TruncatedSVDWrapper RandomForest 0:02:10 0.8341 0.9134\n 21 StandardScalerWrapper XGBoostClassifier 0:00:32 0.9117 0.9134\n 22 StandardScalerWrapper LightGBM 0:00:34 0.9128 0.9134\n 23 MaxAbsScaler LightGBM 0:00:35 0.8880 0.9134\n 24 StandardScalerWrapper XGBoostClassifier 0:00:50 0.9133 0.9134\n 25 StandardScalerWrapper XGBoostClassifier 0:00:41 0.8880 0.9134\n 26 MaxAbsScaler LightGBM 0:00:32 0.9096 0.9134\n 27 StandardScalerWrapper XGBoostClassifier 0:00:53 0.9101 0.9134\n 28 StandardScalerWrapper ExtremeRandomTrees 0:01:03 0.8880 0.9134\n 29 MaxAbsScaler LightGBM 0:00:31 0.9018 0.9134\n 30 VotingEnsemble 0:00:42 0.9168 0.9168\n 31 StackEnsemble 0:00:45 0.9149 0.9168\nStopping criteria reached at iteration 32. Ending experiment.\n********************************************************************************************\nCurrent status: BestRunExplainModel. Best run model explanations started\nCurrent status: ModelExplanationDataSetSetup. Model explanations data setup completed\nCurrent status: PickSurrogateModel. Choosing LightGBM as the surrogate model for explanations\nCurrent status: EngineeredFeatureExplanations. Computation of engineered features started\nCurrent status: EngineeredFeatureExplanations. Computation of engineered features completed\nCurrent status: RawFeaturesExplanations. Computation of raw features started\nCurrent status: RawFeaturesExplanations. Computation of raw features completed\nCurrent status: BestRunExplainModel. Best run model explanations completed\n********************************************************************************************\n" + "text": "Current status: DatasetEvaluation. Gathering dataset statistics.\nCurrent status: FeaturesGeneration. Generating features for the dataset.\nCurrent status: DatasetFeaturization. Beginning to fit featurizers and featurize the dataset.\nCurrent status: DatasetFeaturizationCompleted. Completed fit featurizers and featurizing the dataset.\nCurrent status: DatasetBalancing. Performing class balancing sweeping\nCurrent status: DatasetCrossValidationSplit. Generating individually featurized CV splits.\n\n********************************************************************************************\nDATA GUARDRAILS: \n\nTYPE: Class balancing detection\nSTATUS: ALERTED\nDESCRIPTION: To decrease model bias, please cancel the current run and fix balancing problem.\n Learn more about imbalanced data: https://aka.ms/AutomatedMLImbalancedData\nDETAILS: Imbalanced data can lead to a falsely perceived positive effect of a model's accuracy because the input data has bias towards one class.\n+------------------------------+--------------------------------+--------------------------------------+\n|Size of the smallest class |Name/Label of the smallest class|Number of samples in the training data|\n+==============================+================================+======================================+\n|3692 |yes |32950 |\n+------------------------------+--------------------------------+--------------------------------------+\n\n********************************************************************************************\n\nTYPE: Missing feature values imputation\nSTATUS: PASSED\nDESCRIPTION: No feature missing values were detected in the training data.\n Learn more about missing value imputation: https://aka.ms/AutomatedMLFeaturization\n\n********************************************************************************************\n\nTYPE: High cardinality feature detection\nSTATUS: PASSED\nDESCRIPTION: Your inputs were analyzed, and no high cardinality features were detected.\n Learn more about high cardinality feature handling: https://aka.ms/AutomatedMLFeaturization\n\n********************************************************************************************\nCurrent status: ModelSelection. Beginning model selection.\n\n********************************************************************************************\nITER: The iteration being evaluated.\nPIPELINE: A summary description of the pipeline being evaluated.\nDURATION: Time taken for the current iteration.\nMETRIC: The result of computing score on the fitted pipeline.\nBEST: The best observed score thus far.\n********************************************************************************************\n\n ITER PIPELINE DURATION METRIC BEST\n 0 MaxAbsScaler LightGBM 0:00:28 0.9125 0.9125\n 1 MaxAbsScaler XGBoostClassifier 0:00:34 0.9110 0.9125\n 2 MaxAbsScaler ExtremeRandomTrees 0:00:30 0.7373 0.9125\n 3 SparseNormalizer XGBoostClassifier 0:00:31 0.9132 0.9132\n 4 MaxAbsScaler LightGBM 0:00:27 0.9123 0.9132\n 5 MaxAbsScaler LightGBM 0:00:27 0.8883 0.9132\n 6 StandardScalerWrapper XGBoostClassifier 0:00:28 0.9077 0.9132\n 7 MaxAbsScaler LogisticRegression 0:00:29 0.9086 0.9132\n 8 StandardScalerWrapper ExtremeRandomTrees 0:00:28 0.8895 0.9132\n 9 StandardScalerWrapper XGBoostClassifier 0:00:28 0.9115 0.9132\n 10 SparseNormalizer LightGBM 0:00:27 0.9041 0.9132\n 11 StandardScalerWrapper XGBoostClassifier 0:00:29 0.9134 0.9134\n 12 MaxAbsScaler LogisticRegression 0:00:29 0.9084 0.9134\n 13 MaxAbsScaler SGD 0:00:27 0.8795 0.9134\n 14 StandardScalerWrapper XGBoostClassifier 0:00:29 0.9127 0.9134\n 15 SparseNormalizer RandomForest 0:00:37 0.8193 0.9134\n 16 StandardScalerWrapper LogisticRegression 0:00:27 0.9084 0.9134\n 17 StandardScalerWrapper RandomForest 0:00:31 0.9015 0.9134\n 18 StandardScalerWrapper XGBoostClassifier 0:00:30 0.9124 0.9134\n 19 TruncatedSVDWrapper RandomForest 0:01:30 0.8245 0.9134\n 20 TruncatedSVDWrapper RandomForest 0:02:04 0.8327 0.9134\n 21 StandardScalerWrapper XGBoostClassifier 0:00:29 0.9117 0.9134\n 22 StandardScalerWrapper LightGBM 0:00:31 0.9128 0.9134\n 23 StandardScalerWrapper XGBoostClassifier 0:01:34 0.9071 0.9134\n 24 MaxAbsScaler LightGBM 0:00:27 0.8880 0.9134\n 25 StandardScalerWrapper XGBoostClassifier 0:00:28 0.8880 0.9134\n 26 MaxAbsScaler LightGBM 0:00:28 0.9096 0.9134\n 27 StandardScalerWrapper XGBoostClassifier 0:00:44 0.9133 0.9134\n 28 StandardScalerWrapper ExtremeRandomTrees 0:00:59 0.8880 0.9134\n 29 MaxAbsScaler LightGBM 0:00:27 0.8910 0.9134\n 30 VotingEnsemble 0:00:38 0.9171 0.9171\n 31 StackEnsemble 0:00:41 0.9157 0.9171\nStopping criteria reached at iteration 32. Ending experiment.\n********************************************************************************************\nCurrent status: BestRunExplainModel. Best run model explanations started\nCurrent status: ModelExplanationDataSetSetup. Model explanations data setup completed\nCurrent status: PickSurrogateModel. Choosing LightGBM as the surrogate model for explanations\nCurrent status: EngineeredFeatureExplanations. Computation of engineered features started\nCurrent status: EngineeredFeatureExplanations. Computation of engineered features completed\nCurrent status: RawFeaturesExplanations. Computation of raw features started\nCurrent status: RawFeaturesExplanations. Computation of raw features completed\nCurrent status: BestRunExplainModel. Best run model explanations completed\n********************************************************************************************\n" }, { "output_type": "stream", "name": "stderr", - "text": "2022-09-12:07:55:12,280 INFO [explanation_client.py:334] Using default datastore for uploads\n" + "text": "2022-09-12:10:55:50,518 INFO [explanation_client.py:334] Using default datastore for uploads\n" } ], "execution_count": 9, "metadata": { + "gather": { + "logged": 1662980157355 + }, "jupyter": { "outputs_hidden": false, "source_hidden": false @@ -382,9 +361,6 @@ "transient": { "deleting": false } - }, - "gather": { - "logged": 1662969320950 } } }, @@ -403,24 +379,57 @@ "execution_count": 10, "metadata": { "gather": { - "logged": 1662969321909 + "logged": 1662980158370 + } + } + }, + { + "cell_type": "markdown", + "source": [ + "#### 3. Delete the compute cluster" + ], + "metadata": { + "nteract": { + "transient": { + "deleting": false + } } } }, { "cell_type": "code", - "source": [], - "outputs": [], - "execution_count": null, + "source": [ + "# Fetch or create the compute resource\n", + "try:\n", + " instance = ComputeTarget(workspace=ws, name=cluster_name)\n", + "\n", + " instance.delete()\n", + " instance.wait_for_completion(show_output=True)\n", + " print('Deleted compute resource')\n", + "\n", + "except ComputeTargetException:\n", + " print('Already deleted!')" + ], + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": "InProgressCurrent provisioning state of AmlCompute is \"Deleting\"\n\n..........Current provisioning state of AmlCompute is \"Deleting\"\n\n..........Current provisioning state of AmlCompute is \"Deleting\"\n\n...........Current provisioning state of AmlCompute is \"Deleting\"\n\n.....\nSucceededProvisioning operation finished, operation \"Succeeded\"\nAlready deleted!\n" + } + ], + "execution_count": 11, "metadata": { "jupyter": { - "source_hidden": false, - "outputs_hidden": false + "outputs_hidden": false, + "source_hidden": false }, "nteract": { "transient": { "deleting": false } + }, + "gather": { + "logged": 1662980385394 } } } @@ -446,15 +455,20 @@ "nbconvert_exporter": "python", "file_extension": ".py" }, - "nteract": { - "version": "nteract-front-end@1.0.0" - }, "microsoft": { "host": { "AzureML": { "notebookHasBeenCompleted": true } } + }, + "nteract": { + "version": "nteract-front-end@1.0.0" + }, + "vscode": { + "interpreter": { + "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" + } } }, "nbformat": 4,