[AWS Terraform]EKS 구축

구성도

 

 

디렉토리 tree 구조

 

 

01. 기본 설정

변수 파일 작성

# 99_var.tf
variable "region" {
  type    = string
  default = "ap-northeast-2"
}

variable "cidr" {
  type    = string
  default = "10.0.0.0/16"
}

variable "rocidr" {
  type    = string
  default = "0.0.0.0/0"
}

variable "name" {
  type    = string
  default = "eks"
}

 

프로바이더, region 등 기본 aws terraform 설정

# 01_init.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# 01_region.tf
provider "aws" {
    region = var.region  
}

 

키페어 생성

IAM에서 생성한 엑세스 키 중 public key로 키페어 생성

(엑세스 키 만드는 방법은 이 포스팅 참조)

# 02_key.tf
resource "aws_key_pair" "pmh_key" {
  key_name = "${var.name}-key"
  public_key = file("./pmh.pub")

 

 

02. VPC

# 03_vpc.tf
resource "aws_vpc" "eks_vpc" {
  cidr_block           = var.cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.name}-vpc"
  }
}

 

03. 인터넷 게이트웨이

# 04_ig.tf
resource "aws_internet_gateway" "eks_ig" {
  vpc_id = aws_vpc.eks_vpc.id

  tags = {
    Name = "${var.name}-ig"
  }
}

 

04. 서브넷

퍼블릭 서브넷 1쌍: eks를 설치할 bastion을 둘 예정

프라이빗 서브넷 2쌍: master node와 worker node를 둘 예정

# 05_subnet.tf
resource "aws_subnet" "eksnet_bastion" {
  count      = 2
  vpc_id     = aws_vpc.eks_vpc.id
  cidr_block = "10.0.${count.index}.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "c"}"

  tags = {
    Name = "${var.name}-pub-${count.index == 0 ? "a" : "c"}"
  }
}

resource "aws_subnet" "eksnet_ma" {
  count      = 2
  vpc_id     = aws_vpc.eks_vpc.id
  cidr_block = "10.0.${count.index + 2}.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "c"}"

  tags = {
    Name = "${var.name}-ma-${count.index == 0 ? "a" : "c"}"
  }
}

resource "aws_subnet" "eksnet_work" {
  count      = 2
  vpc_id     = aws_vpc.eks_vpc.id
  cidr_block = "10.0.${count.index + 4}.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "c"}"

  tags = {
    Name = "${var.name}-work-${count.index == 0 ? "a" : "c"}"
  }
}

 

05. 라우팅 테이블

# 06_rt.tf
resource "aws_route_table" "eks_eksnet_bastion_rt" {
  vpc_id = aws_vpc.eks_vpc.id

  route {
    cidr_block = var.rocidr
    gateway_id = aws_internet_gateway.eks_ig.id
  }

  tags = {
    Name = "${var.name}-bastion-rt"
  }
}

resource "aws_route_table" "eks_node_rt" {
  vpc_id = aws_vpc.eks_vpc.id

  route {
    cidr_block = var.rocidr
    gateway_id = aws_internet_gateway.eks_ig.id
  }

  tags = {
    Name = "${var.name}-node-rt"
  }
}

 

# 07_rtass.tf
resource "aws_route_table_association" "eks_bastion_rtass" {
  count          = 2
  subnet_id      = aws_subnet.eksnet_bastion[count.index].id
  route_table_id = aws_route_table.eks_eksnet_bastion_rt.id
}
resource "aws_route_table_association" "eks_man_rtaas" {
  count          = 2
  subnet_id      = aws_subnet.eksnet_ma[count.index].id
  route_table_id = aws_route_table.eks_node_rt.id
}
resource "aws_route_table_association" "eks_work_rtaas" {
  count          = 2
  subnet_id      = aws_subnet.eksnet_work[count.index].id
  route_table_id = aws_route_table.eks_node_rt.id
}

 

 

06. EKS 클러스터

코드 참고:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_cluster

Terraform Registry

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon

Terraform Registry

 

eks_coredns는 성능 저하 문제로 제외하고 cni, proxy만 최신 버전으로 구성

 

# 09_eksclu.tf
resource "aws_eks_cluster" "eks_clu" {
  name     = "${var.name}-clu"
  role_arn = aws_iam_role.eks_clurole.arn

  vpc_config {
    subnet_ids              = concat(aws_subnet.eksnet_ma[*].id, aws_subnet.eksnet_work[*].id)
    endpoint_private_access = true
    endpoint_public_access  = true
    security_group_ids      = [aws_security_group.eks_secu.id]
  }

  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKSVPCResourceController,
  ]
}

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "eks_clurole" {
  name               = "eks-clurole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_clurole.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_clurole.name
}

resource "aws_eks_addon" "eks_cni" {
  cluster_name                = aws_eks_cluster.eks_clu.name
  addon_name                  = "vpc-cni"
  addon_version               = "v1.13.2-eksbuild.1"
  resolve_conflicts_on_create = "OVERWRITE"
}

resource "aws_eks_addon" "eks_proxy" {
  cluster_name                = aws_eks_cluster.eks_clu.name
  addon_name                  = "kube-proxy"
  addon_version               = "v1.27.3-eksbuild.1"
  resolve_conflicts_on_create = "OVERWRITE"
}

 

 

07. EKS 노드 그룹

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group

Terraform Registry

# 10_eksnode.tf
resource "aws_eks_node_group" "eks_ma_node" {
  cluster_name    = aws_eks_cluster.eks_clu.name
  node_group_name = "${var.name}-master-node"
  node_role_arn   = aws_iam_role.eks_noderole.arn
  subnet_ids      = concat(aws_subnet.eksnet_ma[*].id)
  capacity_type   = "ON_DEMAND"
  disk_size       = 10
  instance_types  = ["t3.micro"]

  scaling_config {
    desired_size = 2
    max_size     = 2
    min_size     = 1
  }

  update_config {
    max_unavailable = 1
  }

  # Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
  # Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-AmazonEC2ContainerRegistryReadOnly,
  ]
}

resource "aws_eks_node_group" "eks_work_node" {
  cluster_name    = aws_eks_cluster.eks_clu.name
  node_group_name = "${var.name}-worker-node"
  node_role_arn   = aws_iam_role.eks_noderole.arn
  subnet_ids      = concat(aws_subnet.eksnet_work[*].id)
  capacity_type   = "ON_DEMAND"
  disk_size       = 10
  instance_types  = ["t3.micro"]

  scaling_config {
    desired_size = 2
    max_size     = 2
    min_size     = 1
  }

  update_config {
    max_unavailable = 1
  }

  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-AmazonEC2ContainerRegistryReadOnly,
  ]
}

resource "aws_iam_role" "eks_noderole" {
  name = "eks-noderole"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_noderole.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_noderole.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.eks_noderole.name
}

 

 

08. eks 인스턴스 생성

프라이빗에 두는게 좋지만 간단한 구성을 위해 퍼블릭 서브넷에 위치.

11_ec2.tf
resource "aws_instance" "pmh_bastion" {
  ami                    = "ami-0ea4d4b8dc1e46212"
  instance_type          = "t2.micro"
  key_name               = "${var.name}-key"
  vpc_security_group_ids = [aws_security_group.eks_secu.id]
  availability_zone      = "${var.region}a"
  private_ip             = "10.0.0.10"
  user_data = templatefile("./eks.sh", {
    region     = var.region
    name       = "${var.name}-clu"
    access_key = var.access_key
    secret_key = var.secret_key
  })
  subnet_id                   = aws_subnet.eksnet_bastion[0].id
  associate_public_ip_address = true

  tags = {
    Name = "${var.name}-bastion"
  }
  depends_on = [ 
    aws_eks_cluster.eks_clu
    ]
}

output "pub_ip" {
  value = aws_instance.pmh_bastion.public_ip
}