-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
akastav
committed
Apr 19, 2021
1 parent
7878903
commit 99b23a6
Showing
142 changed files
with
18,855 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
> :warning: working on the readme in progress | ||
# SAK-incubator | ||
|
||
The [sak-incubator](https://github.com/provectus/sak-incubator/tree/main) repository contains Terraform modules that pass the verification and evaluation stage. After adapting the module to the project, it will get its own repository of the form sak - < module name> and a fixed version. You can offer your modules here | ||
|
||
## Using SAK Modules | ||
|
||
To use modules in your cluster, include some in your project by uncommenting them in the `modules.tf` file, set variables for these modules in the `example.tfvars` file, and deploy your cluster. | ||
To add or destroy a module, add/remove it in the modules.tf file and run: | ||
``` | ||
terraform plan -out plan && terraform apply plan | ||
``` | ||
## All SAK Modules | ||
|
||
SAK Modules: | ||
|
||
* [Core Modules](#core) | ||
* [Optional Modules](#optional) | ||
|
||
Some of the SAK modules are core - you can't deploy a cluster without them. Core modules are in bold in the list below. Other modules are optional. | ||
|
||
* [airflow](https://github.com/provectus/swiss-army-kube/tree/master/modules/airflow) | ||
* [cicd](https://github.com/provectus/swiss-army-kube/tree/master/modules/cicd) | ||
+ [argo](https://github.com/provectus/swiss-army-kube/tree/master/modules/cicd/argo) | ||
+ [jenkins](https://github.com/provectus/swiss-army-kube/tree/master/modules/cicd/jenkins) | ||
* [ingress](https://github.com/provectus/swiss-army-kube/tree/master/modules/ingress) | ||
+ [alb-ingress](https://github.com/provectus/swiss-army-kube/tree/master/modules/ingress/alb-ingress) | ||
+ [nginx](https://github.com/provectus/swiss-army-kube/tree/master/modules/ingress/nginx) | ||
* [kubeflow](https://github.com/provectus/swiss-army-kube/tree/master/modules/kubeflow) | ||
* **[kubernetes](https://github.com/provectus/swiss-army-kube/tree/master/modules/kubernetes)** | ||
* [logging](https://github.com/provectus/swiss-army-kube/tree/master/modules/logging) | ||
+ [efk](https://github.com/provectus/swiss-army-kube/tree/master/modules/logging/efk) | ||
+ [loki](https://github.com/provectus/swiss-army-kube/tree/master/modules/logging/loki) | ||
* [monitoring](https://github.com/provectus/swiss-army-kube/tree/master/modules/monitoring) | ||
+ [prometheus](https://github.com/provectus/swiss-army-kube/tree/master/modules/monitoring/prometheus) | ||
* **[network](https://github.com/provectus/swiss-army-kube/tree/master/modules/network)** | ||
* [rds](https://github.com/provectus/swiss-army-kube/tree/master/modules/rds) | ||
* [scaling](https://github.com/provectus/swiss-army-kube/tree/master/modules/scaling) | ||
* [storage](https://github.com/provectus/swiss-army-kube/tree/master/modules/storage) | ||
+ [efs](https://github.com/provectus/swiss-army-kube/tree/master/modules/storage/efs) | ||
+ [fsx](https://github.com/provectus/swiss-army-kube/tree/master/modules/storage/fsx) | ||
* **[system](https://github.com/provectus/swiss-army-kube/tree/master/modules/system)** | ||
|
||
<a name="core"></a> | ||
### Core Modules | ||
|
||
#### 1. Kubernetes | ||
|
||
Kubernetes module is used to deploy the EKS cluster in Amazon. It creates an autoscaling group (ASG) of EC2 instances in selected accessibility zones and runs containers on those instances, maintaining and scaling them. | ||
|
||
#### 2. Network | ||
|
||
Network module is a VPC module for creating networks, load balancers, and gateways. | ||
|
||
#### 3. System | ||
|
||
System module configures an EKS cluster with addons and Helm charts - cert-manager (ExternalDNS), external-dns, saled-secrets, kube-state-metrics. Cert-manager is a native Kubernetes certificate management addon to automate issuance and management of TLS certificates. ExternalDNS addon makes Kubernetes resources discoverable via public DNS servers. kube-state-metrics Helm Chart listens to the Kubernetes API server and generates metrics about the state of the objects (deployments, nodes and pods). sealed-secrets manages secretes. | ||
|
||
<a name="optional"></a> | ||
### Optional Modules | ||
|
||
Other (non-core) modules are optional. You can include them in your project by uncommenting them in the `modules.tf` file and setting variables for them in the `example.tfvars` file. You can also add your own modules to include in your cluster deployments. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
data "aws_region" "current" {} | ||
|
||
locals { | ||
|
||
reuse_existing_acm_arn = var.existing_acm_arn != "" | ||
create_self_signed_acm_certificate = var.existing_acm_arn == "" && var.self_sign_acm_certificate | ||
create_normal_acm_certificate = var.existing_acm_arn == "" && !var.self_sign_acm_certificate | ||
|
||
aws_region = var.aws_region == "" ? data.aws_region.current.name : var.aws_region | ||
|
||
} | ||
|
||
|
||
provider "aws" { | ||
alias = "certificate" | ||
region = local.aws_region | ||
} | ||
|
||
|
||
# normal acm certificate | ||
module "acm_certificate" { | ||
source = "terraform-aws-modules/acm/aws" | ||
version = "v2.0" | ||
|
||
count = local.create_normal_acm_certificate ? 1 : 0 | ||
domain_name = var.domain_name | ||
subject_alternative_names = var.subject_alternative_names | ||
zone_id = var.zone_id | ||
validate_certificate = var.validate_certificate | ||
|
||
providers = { | ||
aws = aws.certificate | ||
} | ||
|
||
tags = var.tags | ||
} | ||
|
||
|
||
|
||
# self-signed certificate | ||
resource "tls_private_key" "self_signed_cert" { | ||
count = local.create_self_signed_acm_certificate ? 1 : 0 | ||
algorithm = "RSA" | ||
} | ||
|
||
resource "tls_self_signed_cert" "self_signed_cert" { | ||
count = local.create_self_signed_acm_certificate ? 1 : 0 | ||
key_algorithm = "RSA" | ||
private_key_pem = tls_private_key.self_signed_cert[0].private_key_pem | ||
|
||
subject { | ||
common_name = var.domain_name | ||
organization = var.domain_name | ||
} | ||
|
||
validity_period_hours = var.self_signed_certificate_validity_period | ||
|
||
allowed_uses = [ | ||
"key_encipherment", | ||
"digital_signature", | ||
"server_auth", | ||
] | ||
} | ||
|
||
resource "aws_acm_certificate" "self_signed_cert" { | ||
count = local.create_self_signed_acm_certificate ? 1 : 0 | ||
private_key = tls_private_key.self_signed_cert[0].private_key_pem | ||
certificate_body = tls_self_signed_cert.self_signed_cert[0].cert_pem | ||
|
||
provider = aws.certificate | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
|
||
output "arn" { | ||
value = local.reuse_existing_acm_arn ? var.existing_acm_arn : (var.self_sign_acm_certificate ? aws_acm_certificate.self_signed_cert[0].arn : module.acm_certificate[0].this_acm_certificate_arn) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
|
||
variable "domain_name" { | ||
type = string | ||
description = "A domain name for which the certificate should be issued" | ||
default = "" | ||
} | ||
|
||
variable "subject_alternative_names" { | ||
type = list(string) | ||
description = "A list of domains that should be SANs in the issued certificate" | ||
default = [] | ||
} | ||
|
||
variable "zone_id" { | ||
type = string | ||
} | ||
|
||
variable "validate_certificate" { | ||
type = bool | ||
default = true | ||
} | ||
|
||
variable "tags" { | ||
type = map(string) | ||
description = "A set of tags" | ||
default = {} | ||
} | ||
|
||
variable "existing_acm_arn" { | ||
type = string | ||
description = "The ARN of an ACM certificate to attach to the Load Balancer" | ||
default = "" | ||
} | ||
|
||
variable "self_sign_acm_certificate" { | ||
type = bool | ||
description = "Set to true in order to create a self-signed ACM certificates instead of letting ACM create and validate them" | ||
default = false | ||
} | ||
|
||
variable "aws_region" { | ||
type = string | ||
description = "Region in which to create the certificate" | ||
default = "" | ||
|
||
} | ||
|
||
variable "self_signed_certificate_validity_period" { | ||
type = number | ||
description = "Number of hours the self-signed certificate should be valid" | ||
default = 2400 //100 days | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
This chart bootstraps an Apache Airflow deployment on a Kubernetes cluster using the Helm package manager. | ||
|
||
Bitnami charts can be used with Kubeapps for deployment and management of Helm Charts in clusters. This Helm chart has been tested on top of Bitnami Kubernetes Production Runtime (BKPR). Deploy BKPR to get automated TLS certificates, logging and monitoring for your applications. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
# Create namespace | ||
resource "kubernetes_namespace" "airflow" { | ||
depends_on = [ | ||
var.module_depends_on | ||
] | ||
metadata { | ||
name = "airflow" | ||
} | ||
} | ||
|
||
resource "helm_release" "airflow" { | ||
depends_on = [ | ||
var.module_depends_on | ||
] | ||
name = "airflow" | ||
repository = "https://charts.bitnami.com/bitnami" | ||
chart = "airflow" | ||
version = "6.3.7" | ||
namespace = kubernetes_namespace.airflow.metadata[0].name | ||
recreate_pods = true | ||
timeout = 1200 | ||
|
||
|
||
values = [templatefile("${path.module}/values/airflow.yaml", | ||
{ | ||
airflow_url = "airflow.${var.domains[0]}" | ||
airflow_username = var.airflow_username | ||
airflow_password = var.airflow_password != "" ? var.airflow_password : random_password.airflow_password.result | ||
airflow_fernetKey = var.airflow_fernetKey | ||
postgresql_local = var.airflow_postgresql_local | ||
postgresql_host = var.airflow_postgresql_host | ||
postgresql_port = var.airflow_postgresql_port | ||
postgresql_username = var.airflow_postgresql_username | ||
postgresql_password = var.airflow_postgresql_local ? random_password.airflow_postgresql_password.result : var.airflow_postgresql_password | ||
postgresql_database = var.airflow_postgresql_database | ||
redis_local = var.airflow_redis_local | ||
redis_host = var.airflow_redis_host | ||
redis_port = var.airflow_redis_port | ||
redis_username = var.airflow_redis_username | ||
redis_password = var.airflow_redis_local ? random_password.airflow_redis_password.result : var.airflow_redis_password | ||
}) | ||
] | ||
} | ||
|
||
#Password generator | ||
resource "random_password" "airflow_password" { | ||
length = 16 | ||
special = true | ||
override_special = "!#%&*()-_=+[]{}<>:?" | ||
} | ||
|
||
resource "aws_ssm_parameter" "airflow_password" { | ||
name = "/airflow/${var.cluster_name}/${var.airflow_username}" | ||
type = "SecureString" | ||
value = random_password.airflow_password.result | ||
} | ||
|
||
resource "random_password" "airflow_postgresql_password" { | ||
length = 16 | ||
special = true | ||
override_special = "!#%&*()-_=+[]{}<>:?" | ||
} | ||
|
||
resource "aws_ssm_parameter" "airflow_postgresql_password" { | ||
name = "/airflow/${var.cluster_name}/${var.airflow_postgresql_username}" | ||
type = "SecureString" | ||
value = random_password.airflow_postgresql_password.result | ||
} | ||
|
||
resource "random_password" "airflow_redis_password" { | ||
length = 16 | ||
special = true | ||
override_special = "!#%&*()-_=+[]{}<>:?" | ||
} | ||
|
||
resource "aws_ssm_parameter" "airflow_redis_password" { | ||
name = "/airflow/${var.cluster_name}/${var.airflow_redis_username}" | ||
type = "SecureString" | ||
value = random_password.airflow_redis_password.result | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
output "airflow" { | ||
value = helm_release.airflow | ||
} |
Oops, something went wrong.