Firefly States Redactor

The Firefly states redactor is a self-hosted solution for securely handling Terraform state files. It fetches state files from remote sources, scans for sensitive data, and redacts secrets before mirroring the files to an S3 bucket. This helps organizations ensure that sensitive information is not exposed in their infrastructure state management workflows.

Features

• Fetches Terraform state files from supported remote sources (e.g., S3, Terraform Cloud, ArgoCD).

• Identifies and redacts sensitive data within state files.

• Mirrors redacted state files to a designated S3 bucket.

• Integrates with Gitleaks to further scan for secrets.

• Can be deployed as a Kubernetes CronJob or as an ECS Fargate task.

Architecture

The redactor is deployed as a Kubernetes CronJob or ECS Fargate task that runs every 2 hours by default. It is designed for EKS clusters and uses IAM roles for access to S3. The redactor can also be run on ECS Fargate for organizations preferring AWS-native orchestration.

Prerequisites

• An EKS cluster (for Kubernetes deployment) or an ECS cluster (for AWS Fargate deployment).

• An S3 bucket for storing redacted state files.

• IAM role with the following permissions:

  • s3:GetBucket, s3:ListBucket, s3:GetObject, s3:PutObject.

  • (Optional) kms:Decrypt if the bucket is encrypted.

• Credentials for the remote state provider (e.g., Terraform Cloud token, ArgoCD token).

Installation (Kubernetes Helm)

To install the states redactor using Helm:

helm repo add firefly-redactor https://gofireflyio.github.io/states-redactor
helm install states-redactor firefly-redactor/firefly-redactor -f values.yaml --namespace=firefly --create-namespace

Configuration Examples (values.yaml)

Terraform Cloud

serviceAccount:
  annotations: {
     "gofirefly.io/component": firefly-redactor,
     "eks.amazonaws.com/role-arn": aws:aws:iam::123456789:role/my-role
  }
firefly:
  accountId: GIVEN-BY-FIREFLY
  crawlerId: GIVEN-BY-FIREFLY
  location:
    tfc:
      organization: example
      address: example-tfc-enteprise.com
  type: tfc

credentials:
  tfcToken: MY-ORGANIZATION-TOKEN
  tfcCustomDomain: https://example-tfc-enteprise.com

redactorMirrorBucketName: my-mirror-bucket
redactorMirrorBucketRegion: us-east-1
logging:
  remoteHash: GIVEN-BY-FIREFLY

S3 Bucket

serviceAccount:
  annotations: {
     "gofirefly.io/component": firefly-redactor,
     "eks.amazonaws.com/role-arn": aws:aws:iam::123456789:role/my-role
  }
firefly:
  accountId: GIVEN-BY-FIREFLY
  crawlerId: GIVEN-BY-FIREFLY
  location:
    s3:
     isLocal: true
     bucket: my-bucket
     region: us-east-1
  type: s3

redactorMirrorBucketName: my-mirror-bucket
redactorMirrorBucketRegion: us-east-1
logging:
  remoteHash: GIVEN-BY-FIREFLY

Running on ECS (Terraform Module)

You can also run the states redactor on ECS Fargate using the provided Terraform module:

module "states-redactor-ecs" {
  source = "github.com/gofireflyio/states-redactor//terraform/ecs"
  aws_region = "us-west-2"

  firefly_account_id = "<ACCOUNT_ID>"             // Given by Firefly
  firefly_crawler_id = "<CRAWLER_ID>"             // Given by Firefly
  firefly_remote_log_hash = "<REMOTE_LOG_HASH>"   // Given by Firefly

  redacted_bucket_name = "tfstate-target-bucket"
  source_bucket_name = "tfstate-source-bucket"
  source_bucket_region = "us-west-2"

  container_cpu = 256
  container_memory = 512
  schedule_expression = "rate(2 hours)"

  security_groups = ["sg-1234"]
  subnets = ["subnet-1234", "subnet-5678"]
  assign_public_ip = false // If false, add VPC endpoints to reach the ECR
  ecs_cluster_arn = "arn:aws:ecs:us-west-2:0123456789:cluster/firefly-states-redactor" // If empty, will create a cluster
}

References

• states-redactor GitHub repository

• Gitleaks

Need Help?

Reach out to your Firefly Customer Success manager or email [email protected] for assistance with your states redactor configuration.