Mdip/aws-s3-bucket

examples/Tutorial_3-AWS+S3.ipynb

@@ -0,0 +1,272 @@
+{
+ "cells": [
+  {
+   "attachments": {},
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Connecting StochSS-Compute to S3\n",
+    "This tutorial shows how you can configure StochSS-Compute to use an S3 bucket for its Results cache."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys, os\n",
+    "sys.path.insert(1, os.path.abspath(os.path.join(os.getcwd(), '../')))\n",
+    "\n",
+    "from stochss_compute.cloud import EC2Cluster\n",
+    "from stochss_compute import RemoteSimulation\n",
+    "\n",
+    "import gillespy2"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 1. Configuration\n",
+    "1. First, create an AWS account [here](https://aws.amazon.com/).\n",
+    "2. In order to make the AWS API calls to your account, you need an AWS access key and access key ID.  \n",
+    "From the IAM dashboard, click 'Manage access keys'.  \n",
+    "Then, under the Access keys tab, click 'Create New Access Key'.  \n",
+    "This file can only be downloaded once, but if something happens you can just make a new one.  \n",
+    "This file contains the Access Key ID and a Secret Access Key.\n",
+    "3. The simplest way to configure API calls is to download and install [AWS Command Line Interface](https://aws.amazon.com/cli/).  \n",
+    "Then, run `aws configure`.  \n",
+    "You will be asked for your AWS Access Key ID, your AWS Secret Access Key, and default region name (listed [here](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html#Concepts.RegionsAndAvailabilityZones.Regions)), such as `us-east-2`.  \n",
+    "If you prefer not to install this, you can set the environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION`.  \n",
+    "For a full list of environment variables you can set, see [here](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#using-environment-variables)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Uncomment the two lines below if using environment variables to configure AWS API calls. \n",
+    "# from dotenv import load_dotenv # To install: python -m pip install python-dotenv\n",
+    "# load_dotenv() # Loads from a file named .env by default"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 2. Launch\n",
+    "- Instantiate a cluster object. \n",
+    "- `stochss_compute.cloud` will first attempt to re-load an already running cluster by resource name, so you can continue where you left off."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster = EC2Cluster()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Launch a StochSS-Compute instance, providing an AWS instance type.\n",
+    "- For more information about instance types, see [here](https://aws.amazon.com/ec2/instance-types/).\n",
+    "- Make sure you are aware of AWS pricing policies before proceeding. See [here](https://aws.amazon.com/ec2/pricing/) and [here](https://aws.amazon.com/ec2/pricing/on-demand/) for more information."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.launch_single_node_instance('t2.micro')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 3. Run\n",
+    "- Create your model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def create_michaelis_menten(parameter_values=None):\n",
+    "    # Intialize the Model with a name of your choosing.\n",
+    "    model = gillespy2.Model(name=\"Michaelis_Menten\")\n",
+    "\n",
+    "    \"\"\"\n",
+    "    Variables (GillesPy2.Species) can be anything that participates in or is produced by a reaction channel.\n",
+    "\n",
+    "    - name: A user defined name for the species.\n",
+    "    - initial_value: A value/population count of species at start of simulation.\n",
+    "    \"\"\"\n",
+    "    A = gillespy2.Species(name=\"A\", initial_value=301)\n",
+    "    B = gillespy2.Species(name=\"B\", initial_value=120)\n",
+    "    C = gillespy2.Species(name=\"C\", initial_value=0)\n",
+    "    D = gillespy2.Species(name=\"D\", initial_value=0)\n",
+    "\n",
+    "    # Add the Variables to the Model.\n",
+    "    model.add_species([A, B, C, D])\n",
+    "\n",
+    "    \"\"\"\n",
+    "    Parameters are constant values relevant to the system, such as reaction kinetic rates.\n",
+    "\n",
+    "    - name: A user defined name for reference.\n",
+    "    - expression: Some constant value.\n",
+    "    \"\"\"\n",
+    "    rate1 = gillespy2.Parameter(name=\"rate1\", expression=0.0017)\n",
+    "    rate2 = gillespy2.Parameter(name=\"rate2\", expression=0.5)\n",
+    "    rate3 = gillespy2.Parameter(name=\"rate3\", expression=0.1)\n",
+    "\n",
+    "    # Add the Parameters to the Model.\n",
+    "    model.add_parameter([rate1, rate2, rate3])\n",
+    "\n",
+    "    \"\"\"\n",
+    "    Reactions are the reaction channels which cause the system to change over time.\n",
+    "\n",
+    "    - name: A user defined name for the reaction.\n",
+    "    - reactants: A dictionary with participant reactants as keys, and consumed per reaction as value.\n",
+    "    - products: A dictionary with reaction products as keys, and number formed per reaction as value.\n",
+    "    - rate: A parameter rate constant to be applied to the propensity of this reaction firing.\n",
+    "    - propensity_function: Can be used instead of rate in order to declare a custom propensity function in string format.\n",
+    "    \"\"\"\n",
+    "    r1 = gillespy2.Reaction(\n",
+    "            name=\"r1\",\n",
+    "            reactants={'A': 1, 'B': 1}, \n",
+    "            products={'C': 1},\n",
+    "            rate='rate1'\n",
+    "        )\n",
+    "\n",
+    "    r2 = gillespy2.Reaction(\n",
+    "            name=\"r2\",\n",
+    "            reactants={'C': 1}, \n",
+    "            products={'A': 1, 'B': 1},\n",
+    "            rate='rate2'\n",
+    "        )\n",
+    "\n",
+    "    r3 = gillespy2.Reaction(\n",
+    "            name=\"r3\",\n",
+    "            reactants={'C': 1}, \n",
+    "            products={'B': 1, 'D': 1},\n",
+    "            rate='rate3'\n",
+    "        )\n",
+    "\n",
+    "    # Add the Reactions to the Model.\n",
+    "    model.add_reaction([r1, r2, r3])\n",
+    "\n",
+    "    # Define the timespan of the model.\n",
+    "    tspan = gillespy2.TimeSpan.linspace(t=100, num_points=100)\n",
+    "    \n",
+    "    # Set the timespan of the Model.\n",
+    "    model.timespan(tspan)\n",
+    "    return model"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Run it"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model = create_michaelis_menten()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "simulation = RemoteSimulation(model, server=cluster, solver=gillespy2.TauHybridSolver)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "results = simulation.run()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "- Wait for/fetch results"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "results.plot()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 4. Clean Up\n",
+    "- Deletes all cluster resources that were created by `launch_single_node_instance()`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.clean_up()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.1+"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "ef56f9d787682a6ac61c29852cd545f557bd05dd1aa39793e08a6cd6ebb9b699"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

examples/s3_trust_policy.json

@@ -0,0 +1,17 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "ListObjectsInBucket",
+      "Effect": "Allow",
+      "Action": ["s3:ListBucket"],
+      "Resource": ["arn:aws:s3:::sssc-cache"]
+    },
+    {
+      "Sid": "AllObjectActions",
+      "Effect": "Allow",
+      "Action": "s3:*Object",
+      "Resource": ["arn:aws:s3:::sssc-cache/*"]
+    }
+  ]
+}

stochss_compute/cloud/ec2.py

@@ -474,6 +474,11 @@ def _launch_head_node(self, instance_type):
             ],
             'UserData': launch_commands,
         }
+        # if self._remote_config.s3config is not None:
+        #     kwargs['IamInstanceProfile'] = {
+        #         'Arn': self._remote_config.s3_config.instance_role_arn,
+        #         'Name': 'string'
+        #     }
 
         self.log.info(
             'Launching StochSS-Compute server instance. This might take a minute.......')

stochss_compute/cloud/ec2_config.py

@@ -49,6 +49,9 @@ class EC2RemoteConfig:
 
     :param ami: Custom AMI to use, like 'ami-09d3b3274b6c5d4aa'. See `here <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html>`_.
     :type ami: str
+
+    :param s3_config: If this configuration is supplied, s3 caching will be enabled.
+    :type s3_config: S3Config
     '''
     _AMIS = {
         'us-east-1': 'ami-09d3b3274b6c5d4aa',
@@ -67,6 +70,7 @@ def __init__(self,
                  api_port=29681,
                  region=None,
                  ami=None,
+                 s3_config=None,
                  ):
         if suffix is not None:
             suffix = f'-{suffix}'
@@ -81,6 +85,7 @@ def __init__(self,
         self.api_port = api_port
         self.region = region
         self.ami = ami
+        self.s3_config = s3_config
 
 
 class EC2LocalConfig:

stochss_compute/cloud/s3_config.py

@@ -0,0 +1,11 @@
+class S3Config:
+    '''
+    Configuration for AWS Simple Storage Service
+    '''
+    def __init__(self,
+                 instance_role_arn,
+                 s3_bucket_uri,
+                 ) -> None:
+        self.instance_role_arn = instance_role_arn
+        self.s3_bucket_uri = s3_bucket_uri
+