Centralized EKS Logging Across AWS Accounts Using Fluent Bit, Pod Identity and OpenSearch

Jul 23, 20254 min read

Centralized log collection is essential in distributed Kubernetes environments. Amazon EKS combined with Fluent Bit offers a lightweight and flexible log forwarding solution. In this guide, we configure Fluent Bit to run on an EKS cluster in one AWS account (Account A) and forward logs to an OpenSearch domain hosted in another account (Account B).

We’ll secure the communication using EKS Pod Identity and a cross-account IAM role protected with an external_id. This ensures that no long-lived credentials are exposed and the trust relationship cannot be abused.

What is Fluent Bit?

Fluent Bit is a lightweight, high-performance log processor and forwarder. Designed with efficiency in mind, it’s ideal for environments like Kubernetes, where resource constraints matter. It collects logs from various sources (e.g., files, containers, syslog), applies filtering and transformation, and then routes them to different destinations such as OpenSearch, Elasticsearch, or cloud services.

What is OpenSearch?

OpenSearch is an open-source search and analytics engine derived from Elasticsearch 7.10. It’s developed and maintained by AWS and the community. OpenSearch supports full-text search, distributed indexing, and real-time analytics — making it an excellent backend for log aggregation, monitoring, and observability platforms.

Architecture overview

account A: hosts the EKS cluster and Fluent Bit
account B: hosts the OpenSearch domain
Fluent Bit pods use a pod identity to assume a cross-account role that grants access to write logs into OpenSearch

To prevent the confused deputy problem, the trust policy in account B includes an external_id that must be known and passed by the caller (Fluent Bit).

centralized-eks-logging-across-aws-accounts-using-fluent-bit-pod-identity-and-open-search.svg

Prerequisites

EKS cluster deployed in account A with Pod Identity Agent installed
OpenSearch domain deployed in account B
Terraform v1.6+
kubectl access to the cluster
helm

Creating the pod identity role (account A)

resource "aws_iam_policy" "log_writer_assuming_policy" {
  name = "LogWriterAssumingPolicy"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "sts:AssumeRole",
          "sts:TagSession"
        ]
        Resource = "arn:aws:iam::ACCOUNT_B_ID:role/fluentbit-opensearch-writer"
      }
    ]
  })
}

module "fluentbit_pod_identity" {
  source  = "terraform-aws-modules/eks-pod-identity/aws"
  version = "1.11.0"

  name = "fluentbit-pod-identity"

  additional_policy_arns = {
    LogWriterAssumingPolicy = aws_iam_policy.log_writer_assuming_policy.arn
  }

  associations = {
    main = {
      namespace       = "middlewares"
      service_account = "fluentbit"
      cluster_name    = var.cluster_name
    }
  }
}

The log_writer_assuming_policy is a policy that allows sts:AssumeRole on the cross-account writer role in account B.

EKS Pod Identity uses session tags (therefore sts:TagSession action) to pass metadata during the AssumeRole operation — specifically:

Kubernetes namespace
Service account name
Cluster name

These are passed as session tags when the EKS control plane assumes the IAM role on behalf of the pod. AWS uses those tags internally to enforce the correct role-to-service-account association, ensuring that only the intended pods can assume that role.

Creating the OpenSearch writer role (account B)

This role trusts only account A to assume it and requires an external ID for additional protection:

data "aws_iam_policy_document" "trust_policy" {
  statement {
    effect = "Allow"

    principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::ACCOUNT_A_ID:role/fluentbit-pod-identity"]
    }

    actions = ["sts:AssumeRole"]

    condition {
      test     = "StringEquals"
      variable = "sts:ExternalId"
      values   = ["shared-external-id-between-a-and-b-accounts"]
    }
  }
}

resource "aws_iam_role" "fluentbit_writer" {
  name               = "fluentbit-opensearch-writer"
  assume_role_policy = data.aws_iam_policy_document.trust_policy.json
}

Attaching OpenSearch permissions

This policy grants Fluent Bit permission to write to the OpenSearch domain (cluster):

resource "aws_iam_policy" "write_to_opensearch" {
  name = "opensearch-write-access"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "es:ESHttpPost",
          "es:ESHttpPut"
        ]
        Resource = "arn:aws:es:AWS_REGION:ACCOUNT_B_ID:domain/OPENSEARCH_DOMAIN_NAME/*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "attach_writer_policy" {
  role       = aws_iam_role.fluentbit_writer.name
  policy_arn = aws_iam_policy.write_to_opensearch.arn
}

Deploying Fluent Bit on EKS with helm

We use the official Helm chart from the eks-charts repo. It supports pod identity and AWS-signed requests to OpenSearch.

Make sure the IAM role from account B is correctly assumed by Fluent Bit using the pod identity role from account A. Also, pass the external ID as a Helm value. Remember that AWS_Role_ARN has to be an ARN of role in account B. Service account name should be the same as we set up in fluentbit_pod_identity module.

helm repo add eks-charts https://aws.github.io/eks-charts
helm repo update

helm upgrade --install fluentbit eks-charts/aws-for-fluent-bit \
  --namespace middlewares \
  --create-namespace \
  --set serviceAccount.name="fluentbit-service-account-name" \
  --set output.opensearch.enabled=true \
  --set output.opensearch.host="vpc-my-domain.region.es.amazonaws.com" \
  --set output.opensearch.port=443 \
  --set output.opensearch.awsAuth=true \
  --set output.opensearch.awsExternalId="shared-external-id-between-a-and-b-accounts" \
  --set output.opensearch.AWS_Role_ARN="arn:aws:iam::ACCOUNT_B_ID:role/fluentbit-opensearch-writer" \
  --set output.opensearch.index="index-name" \
  --set output.opensearch.type="_doc"

Verifying the setup

Check the logs of the Fluent Bit pods:

kubectl logs -n middlewares -l app.kubernetes.io/name=fluentbit

You should see lines indicating successful writes:

[ info] [output:es:es.0] HTTP status=200

You can also search in the OpenSearch dashboard to confirm logs are being indexed in eks-logs.

Conclusion

This setup allows Fluent Bit to securely ship logs from EKS in account A to OpenSearch in account B. By using EKS Pod Identity and a cross-account IAM role with an external ID, we avoid exposing secrets while still maintaining a strong trust boundary.

This architecture is well-suited for multi-account environments and aligns with AWS best practices for identity and access control.