Skip to content

Latest commit

 

History

History
400 lines (322 loc) · 11.2 KB

File metadata and controls

400 lines (322 loc) · 11.2 KB

AWS Lambda Deployment Guide - GradientBoosting Anomaly Detector

This guide walks through deploying the trained GradientBoosting model to AWS Lambda for real-time anomaly detection scoring.

Note: This guide covers manual step-by-step deployment. For one-click deployment, use CloudBackend/aws-lambda/deploy.sh instead.

Overview

The deployment consists of:

  1. S3 Model Storage: Store trained model artifacts (model.pkl, scaler.pkl)
  2. Lambda Function: Handler that loads model and scores incoming metrics
  3. API Gateway: HTTP endpoint for Wear app to call
  4. IAM Permissions: Role allowing Lambda to access S3

Prerequisites

  • AWS Account with CLI configured
  • Trained model artifacts in MLPipeline/models/lambda_export/ (generated by train_pipeline_sklearn.sh)
  • AWS CLI installed and configured
  • Python 3.9 (Lambda runtime)

Step 1: Create S3 Bucket for Models

# Create bucket (replace with your region)
aws s3 mb s3://health-ml-models-$(date +%s) --region ap-south-2

# Verify
aws s3 ls

Store the bucket name for later steps. Set it as environment variable:

export MODEL_BUCKET="health-ml-models-xxxxxxxxxxxx"

Step 2: Upload Trained Model to S3

cd /Users/ramadugudhanush/Documents/CAP_STONE/MLPipeline

# Upload GradientBoosting model and scaler to S3
aws s3 cp models/saved_models/best_anomaly_gradientboosting.pkl \
  s3://$MODEL_BUCKET/gradientboosting/model.pkl

aws s3 cp models/saved_models/best_anomaly_scaler.pkl \
  s3://$MODEL_BUCKET/gradientboosting/scaler.pkl

# Verify upload
aws s3 ls s3://$MODEL_BUCKET/gradientboosting/

Output should show:

2026-03-06 10:30:45       1234567 model.pkl
2026-03-06 10:30:46          5678 scaler.pkl

Step 3: Create Lambda Execution Role

# Create role trust policy
cat > /tmp/trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

# Create role
aws iam create-role \
  --role-name HealthAnomalyDetectorRole \
  --assume-role-policy-document file:///tmp/trust-policy.json

# Attach basic Lambda execution policy
aws iam attach-role-policy \
  --role-name HealthAnomalyDetectorRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

# Create S3 access policy
cat > /tmp/s3-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::health-ml-models-*/*"
    }
  ]
}
EOF

# Attach S3 policy
aws iam put-role-policy \
  --role-name HealthAnomalyDetectorRole \
  --policy-name S3ModelAccess \
  --policy-document file:///tmp/s3-policy.json

# Wait for role to be available (important!)
sleep 10

Step 4: Package Lambda Function with Dependencies

cd /tmp
mkdir lambda-package
cd lambda-package

# Copy handler
cp /Users/ramadugudhanush/Documents/CAP_STONE/CloudBackend/aws-lambda/lambda_inference_sklearn.py .

# Install dependencies
pip install -r /Users/ramadugudhanush/Documents/CAP_STONE/CloudBackend/aws-lambda/requirements.txt -t .

# Create deployment package
zip -r ../lambda-function.zip .

# Verify size (should be ~50-100MB with sklearn)
ls -lh ../lambda-function.zip

Step 5: Create Lambda Function

# Get AWS account ID
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

# Get role ARN
export ROLE_ARN="arn:aws:iam::${AWS_ACCOUNT_ID}:role/HealthAnomalyDetectorRole"

# Create function
# NOTE: The inference Lambda now uses Docker container deployment (ECR).
# For one-click deployment, use deploy.sh instead.
# Manual container deployment:
#   docker build --platform linux/amd64 -f Dockerfile.inference -t health-inference-lambda:latest .
#   docker tag health-inference-lambda:latest 145023117892.dkr.ecr.ap-south-2.amazonaws.com/health-inference-lambda:latest
#   docker push 145023117892.dkr.ecr.ap-south-2.amazonaws.com/health-inference-lambda:latest
aws lambda create-function \
  --function-name HealthAnomalyInference \
  --runtime python3.11 \
  --role $ROLE_ARN \
  --handler lambda_inference_sklearn.lambda_handler \
  --zip-file fileb:///tmp/lambda-function.zip \
  --timeout 30 \
  --memory-size 512 \
  --environment Variables="{
    MODEL_BUCKET=$MODEL_BUCKET,
    MODEL_KEY=gradientboosting/model.pkl,
    SCALER_KEY=gradientboosting/scaler.pkl
  }"

echo "✅ Lambda function created: arn:aws:lambda:*:${AWS_ACCOUNT_ID}:function:HealthAnomalyDetector"

Verify creation:

aws lambda get-function --function-name HealthAnomalyDetector

Step 6: Create API Gateway Endpoint

# Create REST API
export API_ID=$(aws apigateway create-rest-api \
  --name "HealthAnomalyAPI" \
  --description "Anomaly detection scoring endpoint" \
  --query 'id' \
  --output text)

echo "API ID: $API_ID"

# Get root resource
export ROOT_ID=$(aws apigateway get-resources \
  --rest-api-id $API_ID \
  --query 'items[0].id' \
  --output text)

# Create /score resource
export RESOURCE_ID=$(aws apigateway create-resource \
  --rest-api-id $API_ID \
  --parent-id $ROOT_ID \
  --path-part score \
  --query 'id' \
  --output text)

# Create POST method
aws apigateway put-method \
  --rest-api-id $API_ID \
  --resource-id $RESOURCE_ID \
  --http-method POST \
  --authorization-type NONE

# Get Lambda function ARN
export LAMBDA_ARN="arn:aws:lambda:$(aws configure get region):${AWS_ACCOUNT_ID}:function:HealthAnomalyDetector"

# Create integration
aws apigateway put-integration \
  --rest-api-id $API_ID \
  --resource-id $RESOURCE_ID \
  --http-method POST \
  --type AWS_PROXY \
  --integration-http-method POST \
  --uri "arn:aws:apigateway:$(aws configure get region):lambda:path/2015-03-31/functions/${LAMBDA_ARN}/invocations"

# Create method response
aws apigateway put-method-response \
  --rest-api-id $API_ID \
  --resource-id $RESOURCE_ID \
  --http-method POST \
  --status-code 200

# Grant API Gateway permission to invoke Lambda
aws lambda add-permission \
  --function-name HealthAnomalyDetector \
  --statement-id AllowAPIGateway \
  --action lambda:InvokeFunction \
  --principal apigateway.amazonaws.com \
  --source-arn "arn:aws:execute-api:$(aws configure get region):${AWS_ACCOUNT_ID}:${API_ID}/*"

# Deploy API
export STAGE=$(aws apigateway create-deployment \
  --rest-api-id $API_ID \
  --stage-name prod \
  --query 'id' \
  --output text)

# Get endpoint URL
export API_ENDPOINT="https://${API_ID}.execute-api.$(aws configure get region).amazonaws.com/prod/score"

echo "✅ API Gateway deployed: $API_ENDPOINT"

Step 7: Test the Deployment

Option A: Using curl

curl -X POST https://u8tkgz3vsf.execute-api.ap-south-2.amazonaws.com/prod/health-data/ingest \
  -H "Content-Type: application/json" \
  -d '{
    "metrics": [
      {
        "metric_id": "test-1",
        "heart_rate": 72,
        "steps": 150,
        "calories": 25,
        "distance": 0.15
      },
      {
        "metric_id": "test-2",
        "heart_rate": 150,
        "steps": 200,
        "calories": 50,
        "distance": 0.3
      }
    ]
  }'

Expected response:

{
  "results": [
    {
      "metric_id": "test-1",
      "is_anomaly": false,
      "cloud_score": 0.25,
      "anomaly_reasons": [],
      "feature_contributions": {}
    },
    {
      "metric_id": "test-2",
      "is_anomaly": true,
      "cloud_score": 0.85,
      "anomaly_reasons": ["Elevated heart rate: 150 BPM is above normal range (50–100 BPM)"],
      "feature_contributions": {"heartRate": 0.72, "steps": 0.15, "calories": 0.09, "distance": 0.04}
    }
  ],
  "timestamp": "2024-01-15T10:35:20Z",
  "model_threshold": 0.552534
}

Option B: Using AWS CLI

aws lambda invoke \
  --function-name HealthAnomalyDetector \
  --payload '{"metrics":[{"metric_id":"test","heart_rate":72,"steps":150,"calories":25,"distance":0.15}]}' \
  response.json

cat response.json

Step 8: Configure Wear App to Use Lambda Endpoint

Update WearOSApp/app/src/main/java/com/healthmonitor/config/ApiConfig.kt:

object ApiConfig {
    const val BASE_URL = "https://u8tkgz3vsf.execute-api.ap-south-2.amazonaws.com/prod/"
    // ... rest of config
}

Or use the Settings screen in the Wear app to dynamically override the endpoint at runtime.

Monitoring and Troubleshooting

View Lambda Logs

# Stream logs in real-time
aws logs tail /aws/lambda/HealthAnomalyDetector --follow

# View recent errors
aws logs filter-log-events \
  --log-group-name /aws/lambda/HealthAnomalyDetector \
  --filter-pattern "ERROR"

Common Issues

Issue Solution
ResourceNotFoundException for S3 model Verify bucket name matches MODEL_BUCKET env var
PermissionError accessing S3 Check IAM role has S3 GetObject permission
ModuleNotFoundError: sklearn Ensure lambda-function.zip includes site-packages (run pip install -t .)
MT19937 is not a known BitGenerator numpy version mismatch — models trained with numpy 2.x require numpy>=2.0.0 in the container
Inference returns 400 Payload format mismatch — the handler supports both direct invocation (raw JSON) and API Gateway events (body wrapper)
Timeout (>30s) Increase Lambda timeout and memory (Settings → Memory)

Performance Metrics

# View invocation duration and memory usage
aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name Duration \
  --dimensions Name=FunctionName,Value=HealthAnomalyDetector \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 300 \
  --statistics Average,Maximum

Cost Estimation

  • Model Storage (S3): ~$0.023/month for 50MB
  • Lambda Invocations: $0.20 per 1M requests
    • 1,000 metrics/day = $6/month
    • 100,000 metrics/day = $600/month
  • Data Transfer: $0.09/GB (S3 → Lambda egress)
    • Minimal for metric scoring

Cleanup (Optional)

# Delete Lambda function
aws lambda delete-function --function-name HealthAnomalyDetector

# Delete API Gateway
aws apigateway delete-rest-api --rest-api-id $API_ID

# Delete IAM role
aws iam delete-role-policy --role-name HealthAnomalyDetectorRole --policy-name S3ModelAccess
aws iam detach-role-policy --role-name HealthAnomalyDetectorRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
aws iam delete-role --role-name HealthAnomalyDetectorRole

# Delete S3 bucket
aws s3 rb s3://$MODEL_BUCKET --force

Next Steps

  1. ✅ Train model locally (bash train_pipeline_sklearn.sh)
  2. ✅ Upload model to S3
  3. ✅ Deploy Lambda function
  4. ✅ Create API Gateway endpoint
  5. 🔄 Connect Wear app to Lambda endpoint
  6. 🔄 Monitor and tune anomaly thresholds in production

References