Skip to content

Troubleshooting

eberrigan edited this page Jun 24, 2025 · 6 revisions

Troubleshooting LabLink Deployments

Troubleshooting workflow

Starting in local repo folder lablink/lablink-allocator

  1. Initialize allocator terraform state locally using AWS CLI
terraform init -backend-config=backend-dev.hcl -reconfigure
  1. Plan terraform resources
AWS_PROFILE=711387140753_PowerUserAccess terraform plan -var="resource_suffix=dev"

Check resources. πŸ”‘ Create tls_private_key.lablink_key – a new 4096-bit RSA key

aws_key_pair.lablink_key_pair – named lablink-key-dev

aws_security_group.allow_http – named allow_http_dev, open ports 22 and 80

aws_instance.lablink_allocator_server – tagged as lablink_allocator_server_dev, AMI ami-00c257e12d6828491, type t2.micro

aws_eip.lablink_allocator_ip – elastic IP with tag lablink_allocator_ip_dev

aws_eip_association.lablink_allocator_ip_assoc – binds the EIP to the instance

βœ… Outputs ec2_key_name = "lablink-key-dev"

ec2_public_ip = dynamically assigned, known after apply

private_key_pem = sensitive output (good practice)
  1. Apply changes
terraform apply -var="resource_suffix=dev"
  1. Get pem key for SSH into EC2 instance
terraform output -raw private_key_pem > ~/lablink-key-dev.pem
chmod 600 ~/lablink-key-dev.pem
  1. Check admin website at "<ec2_public_ip>/admin" or ssh
ssh -i ~/lablink-key-dev.pem ubuntu@<ec2_public_ip>
  1. When you are ready, destroy resources
terraform destroy -var="resource_suffix=dev"
  1. If you started any VMs, you can terminate those using the "Terraform Destroy" github action workflow using "Run workflow" at https://github.com/talmolab/lablink/actions/workflows/lablink-allocator-destroy.yml

Docker Permission Error

permission denied while trying to connect to the Docker daemon socket

Solution:

sudo usermod -aG docker $USER
newgrp docker

Or use sudo docker ... commands.


Can't SSH to Instance

  • Ensure correct PEM key is used
  • Instance must have port 22 open in its security group
  • Check EC2 public IP from Terraform output

Flask Server Not Responding

  • Check container logs:

    sudo docker ps
    sudo docker logs <container_id>
    cat /var/log/cloud-init-output.log
  • Interact with container

    sudo docker exec -it <container_id> /bin/bash 
    
  • Test with:

    curl localhost:80

You should see the HTML page for the Flask app.

  • Ping server

    nc -vz <ec2_public_ip> 80
    
  • Check http vs https


VM Not Spawning

  • Check the allocator container logs
  • Verify AWS credentials (access key, secret, token) are passed into the container
  • Confirm that terraform apply was triggered inside the container
Clone this wiki locally