AWS网络负载均衡器HealthChecks的行为不如预期

发布于 2025-02-07 15:56:30 字数 5942 浏览 5 评论 0原文

我有一个问题,网络负载平衡器健康检查员显示目标是不健康的。目标是AWS Fargate实例。

HealthCheck端口和应用程序端口为8080。如果我将ECS Fargate Cluster Service放置在端口8080 Open的安全组中,一切正常。 HealthChecks成功,应用程序有效。

问题是我不希望从公共互联网访问该应用程序。因此,当我尝试限制IP地址访问时,我会遇到问题。因为健康检查端口和 应用程序访问全部发生在端口8080上,我必须限制它,以便健康检查仅来自网络负载均衡器的私有IP地址。

根据AWS文档,这应该是可能的: https:// docs.aws.amazon.com/elasticloadbalcing/latest/network/target-group-register-targets.html#target-security-groups 根据本文档,应该可以将VPC CIDR添加到安全组规则中。我这样做了,健康检查员失败了。我还试图将网络负载平衡器的私有IP地址添加到安全组中,并且它们也失败了。

我完全没有想法。

有人可以验证是否有可能将HealthCheck流量限制在网络负载均衡器的私人IP地址范围内?

这是大多数Terraform代码:

## security groups. This is the security group that the Fargate cluster service is placed in

resource "aws_security_group" "nlb_sg_health_test" {
  vpc_id = aws_vpc.test_vpc.id

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = [
      "10.167.0.0/16"
    ]
    description = "Allow connection from NLB to app healthcheck"
  }

  egress {
    protocol    = "-1"
    from_port   = 0
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }

}

resource "aws_vpc" "test_vpc" {
    cidr_block                       = "10.167.0.0/16"

    tags = {
     "Name" = "${var.prefix}"
   }
}

## private subnets

resource "aws_subnet" "test_eu_west_private_1a" {
  cidr_block              = "10.167.10.0/24"
  map_public_ip_on_launch = false
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}a"

  tags = {
   "Name" = "${var.prefix} private eu-west-1a"
 }
}


resource "aws_subnet" "test_eu_west_private_1b" {
  cidr_block              = "10.167.20.0/24"
  map_public_ip_on_launch = false
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}b"

  tags = {
   "Name" = "${var.prefix} private eu-west-1b"
 }
}

resource "aws_route_table" "test_private" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
   "Name" = "${var.prefix} private a"
 }
}

resource "aws_route_table_association" "test_main_a" {
  subnet_id      = aws_subnet.test_eu_west_private_1a.id
  route_table_id = aws_route_table.test_private.id
}

resource "aws_route_table_association" "test_main_b" {
  subnet_id      = aws_subnet.test_eu_west_private_1b.id
  route_table_id = aws_route_table.test_private.id
}

## public subnets

resource "aws_subnet" "test_eu_west_public_1a" {
  cidr_block              = "10.167.1.0/24"
  map_public_ip_on_launch = true
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}a"

  tags = {
   "Name" = "${var.prefix} public eu-west-1a"
 }
}


resource "aws_subnet" "test_eu_west_public_1b" {
  cidr_block              = "10.167.2.0/24"
  map_public_ip_on_launch = true
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}b"

  tags = {
   "Name" = "${var.prefix} public eu-west-1b"
 }
}

resource "aws_route_table" "test_public" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
   "Name" = "${var.prefix} public a"
 }
}

resource "aws_route_table_association" "test_public_a" {
  subnet_id      = aws_subnet.test_eu_west_public_1a.id
  route_table_id = aws_route_table.test_public.id
}

resource "aws_route_table_association" "test_public_b" {
  subnet_id      = aws_subnet.test_eu_west_public_1b.id
  route_table_id = aws_route_table.test_public.id
}


resource "aws_internet_gateway" "test_igw" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
    Name = "${var.prefix} IGW"
  }
}

resource "aws_route" "public_internet_access_a" {
  route_table_id            = aws_route_table.test_public.id
  destination_cidr_block    = "0.0.0.0/0"
  gateway_id                = aws_internet_gateway.test_igw.id
}

resource "aws_route" "public_internet_access_b" {
  route_table_id            = aws_route_table.test_public.id
  destination_cidr_block    = "0.0.0.0/0"
  gateway_id                = aws_internet_gateway.test_igw.id
}

## NLB

resource "aws_lb_target_group" "test_http_nlb_target_group" {
  name     = "${var.prefix}-target-group-http"
  port     = 8080
  protocol = "TCP"
  vpc_id   = aws_vpc.test_vpc.id
  target_type = "ip"
  preserve_client_ip = true
}

resource "aws_lb_listener" "test_https_nlb_listener" {
  load_balancer_arn = aws_lb.test_nlb.arn
  port              = 443
  protocol          = "TLS"
  ssl_policy        = "ELBSecurityPolicy-2016-08"
  certificate_arn   = aws_acm_certificate.app.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.test_http_nlb_target_group.arn
  }
}

resource "aws_lb" "test_nlb" {
  name               = "${var.prefix}-nlb"
  load_balancer_type = "network"

  subnet_mapping {
    subnet_id     = aws_subnet.test_eu_west_public_1a.id
    allocation_id = aws_eip.nlb_public_ip_a.id
  }

  subnet_mapping {
    subnet_id     = aws_subnet.test_eu_west_public_1b.id
    allocation_id = aws_eip.nlb_public_ip_b.id
  }
}

## ECS

resource "aws_ecs_service" "main" {
  name            = "${var.prefix}"
  cluster         = aws_ecs_cluster.test_cluster.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 1
  launch_type     = "FARGATE"
  enable_execute_command = true
  health_check_grace_period_seconds = 300

  network_configuration {
    security_groups  = [
      aws_security_group.nlb_sg_health_test.id
    ]
    subnets          = [
      aws_subnet.test_eu_west_private_1a.id,
      aws_subnet.test_eu_west_private_1b.id
    ]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.test_http_nlb_target_group.id
    container_name   = var.test_frontend_container_name
    container_port   = 8080
  }
}

I have an issue where the Network Load Balancer healthcheck is showing targets as unhealthy. The targets are AWS Fargate instances.

The healthcheck port and application port is 8080. Everything works perfectly if I place the ECS Fargate cluster service to a security group with 0.0.0.0/0 to port 8080 open.
Healthchecks are successful and the application works.

The problem is that I do not want the application to be accessed from the public internet. So when I try to restrict access with ip-addresses I get issues. Because the healthcheck port and
application access all happen on port 8080 I have to restrict it so that the healthcheck works only from the private ip-address of the Network load balancer.

According to AWS documentation this should be possible:
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html#target-security-groups
According to this documentation it should be possible to add the VPC CIDR to the security group rules. I did that and the healthchecks fail. I also tried to add the private ip-addresses of the network load balancer in to the security group and they fail as well.

I am completely out of ideas.

Could someone verify that this is even possible to limit healthcheck traffic to the private ip address range of the Network Load Balancer?

Here is most of the Terraform code:

## security groups. This is the security group that the Fargate cluster service is placed in

resource "aws_security_group" "nlb_sg_health_test" {
  vpc_id = aws_vpc.test_vpc.id

  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = [
      "10.167.0.0/16"
    ]
    description = "Allow connection from NLB to app healthcheck"
  }

  egress {
    protocol    = "-1"
    from_port   = 0
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }

}

resource "aws_vpc" "test_vpc" {
    cidr_block                       = "10.167.0.0/16"

    tags = {
     "Name" = "${var.prefix}"
   }
}

## private subnets

resource "aws_subnet" "test_eu_west_private_1a" {
  cidr_block              = "10.167.10.0/24"
  map_public_ip_on_launch = false
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}a"

  tags = {
   "Name" = "${var.prefix} private eu-west-1a"
 }
}


resource "aws_subnet" "test_eu_west_private_1b" {
  cidr_block              = "10.167.20.0/24"
  map_public_ip_on_launch = false
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}b"

  tags = {
   "Name" = "${var.prefix} private eu-west-1b"
 }
}

resource "aws_route_table" "test_private" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
   "Name" = "${var.prefix} private a"
 }
}

resource "aws_route_table_association" "test_main_a" {
  subnet_id      = aws_subnet.test_eu_west_private_1a.id
  route_table_id = aws_route_table.test_private.id
}

resource "aws_route_table_association" "test_main_b" {
  subnet_id      = aws_subnet.test_eu_west_private_1b.id
  route_table_id = aws_route_table.test_private.id
}

## public subnets

resource "aws_subnet" "test_eu_west_public_1a" {
  cidr_block              = "10.167.1.0/24"
  map_public_ip_on_launch = true
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}a"

  tags = {
   "Name" = "${var.prefix} public eu-west-1a"
 }
}


resource "aws_subnet" "test_eu_west_public_1b" {
  cidr_block              = "10.167.2.0/24"
  map_public_ip_on_launch = true
  vpc_id                  = aws_vpc.test_vpc.id
  availability_zone       = "${data.aws_region.current.name}b"

  tags = {
   "Name" = "${var.prefix} public eu-west-1b"
 }
}

resource "aws_route_table" "test_public" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
   "Name" = "${var.prefix} public a"
 }
}

resource "aws_route_table_association" "test_public_a" {
  subnet_id      = aws_subnet.test_eu_west_public_1a.id
  route_table_id = aws_route_table.test_public.id
}

resource "aws_route_table_association" "test_public_b" {
  subnet_id      = aws_subnet.test_eu_west_public_1b.id
  route_table_id = aws_route_table.test_public.id
}


resource "aws_internet_gateway" "test_igw" {
  vpc_id = aws_vpc.test_vpc.id

  tags = {
    Name = "${var.prefix} IGW"
  }
}

resource "aws_route" "public_internet_access_a" {
  route_table_id            = aws_route_table.test_public.id
  destination_cidr_block    = "0.0.0.0/0"
  gateway_id                = aws_internet_gateway.test_igw.id
}

resource "aws_route" "public_internet_access_b" {
  route_table_id            = aws_route_table.test_public.id
  destination_cidr_block    = "0.0.0.0/0"
  gateway_id                = aws_internet_gateway.test_igw.id
}

## NLB

resource "aws_lb_target_group" "test_http_nlb_target_group" {
  name     = "${var.prefix}-target-group-http"
  port     = 8080
  protocol = "TCP"
  vpc_id   = aws_vpc.test_vpc.id
  target_type = "ip"
  preserve_client_ip = true
}

resource "aws_lb_listener" "test_https_nlb_listener" {
  load_balancer_arn = aws_lb.test_nlb.arn
  port              = 443
  protocol          = "TLS"
  ssl_policy        = "ELBSecurityPolicy-2016-08"
  certificate_arn   = aws_acm_certificate.app.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.test_http_nlb_target_group.arn
  }
}

resource "aws_lb" "test_nlb" {
  name               = "${var.prefix}-nlb"
  load_balancer_type = "network"

  subnet_mapping {
    subnet_id     = aws_subnet.test_eu_west_public_1a.id
    allocation_id = aws_eip.nlb_public_ip_a.id
  }

  subnet_mapping {
    subnet_id     = aws_subnet.test_eu_west_public_1b.id
    allocation_id = aws_eip.nlb_public_ip_b.id
  }
}

## ECS

resource "aws_ecs_service" "main" {
  name            = "${var.prefix}"
  cluster         = aws_ecs_cluster.test_cluster.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 1
  launch_type     = "FARGATE"
  enable_execute_command = true
  health_check_grace_period_seconds = 300

  network_configuration {
    security_groups  = [
      aws_security_group.nlb_sg_health_test.id
    ]
    subnets          = [
      aws_subnet.test_eu_west_private_1a.id,
      aws_subnet.test_eu_west_private_1b.id
    ]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.test_http_nlb_target_group.id
    container_name   = var.test_frontend_container_name
    container_port   = 8080
  }
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

腹黑女流氓 2025-02-14 15:56:30

来自 aws documentation

对于方案,请选择面向互联网或内部的。面向Internet的负载平衡器路由通过Internet向目标请求。内部负载平衡器路由使用私人IP地址向目标请求。

From AWS documentation

For Scheme, choose Internet-facing or Internal. An internet-facing load balancer routes requests from clients to targets over the internet. An internal load balancer routes requests to targets using private IP addresses.

心病无药医 2025-02-14 15:56:30

我不知道这是否可以作为答案,但是我的AWS设置有问题。我用Terraform在不同的VPC中重新创建了整个基础架构,现在一切都按预期工作。

因此,要回答我问题的一部分:是的,如AWS文档中所述,可以将健康检查流量从网络负载平衡器到Fargate实例。即使“互联网面向”网络负载平衡器也可以做到这一点。只需将负载平衡器的VPC CIDR或私有IP地址添加到安全组中即可。

我不确定我的原始设置中有什么问题,但是在重新创建所有内容之后,它起作用了。

I dont know if this can be accepted as an answer, but there was something wrong with my AWS setup. I recreated to entire infrastructure with Terraform in a different VPC and now everything works as expected.

So to answer one part of my question: Yes, it is possible to limit the health check traffic from the network load balancer to the Fargate instance as described in the AWS documentation. This can be done even with the "internet facing" network load balancer. Just add the VPC CIDR or private ip-addresses of the load balancer to the security group.

I am not sure what the problem was in my original setup, but after recreating everything now it works.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文