Workflows & Guardrails

Welcome to the Firefly Workflows and Guardrails documentation. This guide provides a comprehensive overview of these powerful features, designed to streamline your Infrastructure as Code (IaC) deployments using Terraform and OpenTofu, while ensuring they adhere to your organization's standards.

Understanding Firefly Workflows

Firefly Workflows is a robust system designed to manage and automate your Terraform and OpenTofu deployments. It provides a centralized platform to control how your infrastructure code is planned and applied, integrating seamlessly with your Version Control System (VCS) and CI/CD practices.

What are Workflows?

In Firefly, a Workflow represents the end-to-end process of deploying a specific IaC stack. Each Workflow is tied to a Workspace, which encapsulates:

• IaC Code: The Terraform or OpenTofu configuration files defining a set of resources.

• VCS Integration: Connection to your Git repository (e.g., GitHub, GitLab) where the IaC code is hosted.

• Variables: Configuration values, including sensitive secrets, required for the deployment.

• Execution Environment: The setup used to run terraform plan and terraform apply commands.

• Run History: A log of all plan and apply operations performed for that Workspace.

Workflows automate the critical steps of an IaC deployment:

1. Plan on Change: When changes are proposed to your IaC (typically via a pull request in your VCS), Firefly automatically triggers a terraform plan (or tofu plan). This shows you the potential impact of the changes before they are made.

2. Apply on Merge/Approval: Once changes are approved and merged into your designated primary branch, Firefly triggers a terraform apply (or tofu apply) to enact those changes in your cloud environment.

Key Benefits of Firefly Workflows:

• Automation: Reduces manual effort and potential for human error in deployments.

• Consistency: Ensures a standardized deployment process across all your projects and teams.

• Visibility: Provides a clear view of all ongoing and past deployments, their status, and detailed logs.

• Collaboration: Facilitates teamwork by integrating with VCS workflows (pull/merge requests) and providing shared visibility.

• Control: Allows for fine-grained configuration of how and when deployments occur.

Options for Using Workflows in Firefly

Firefly offers flexibility in how you can implement and utilize Workflows, catering to different needs and existing setups:

1. Firefly-Managed Workflows (with Firefly Runners):

   • Concept: Firefly provides and manages the execution environment (runners) for your Terraform/OpenTofu operations. You define your Workspace in Firefly, point it to your IaC code in your VCS, and Firefly handles the rest – pulling the code, running plan and apply using its own secure and managed infrastructure.

   • Best For: Teams looking for a turnkey solution to automate IaC deployments without the overhead of managing their own CI/CD runners or complex pipeline scripts. Ideal for simplifying setup and getting started quickly.

   • How it Works: You create a new Workflow/Workspace directly in the Firefly UI, configuring details like VCS repository, branch, Terraform version, variables, and cloud provider authentication. Firefly's internal runners then execute the necessary Terraform/OpenTofu commands based on triggers from your VCS (e.g., new pull request, merge to main branch).

   • Refer to the Creating New Firefly Workflows documentation for a detailed guide.
2. Integrate into Existing CI/CD Pipelines (Visualization and Monitoring):

   • Concept: If you already have an established CI/CD pipeline (e.g., Jenkins, GitHub Actions, GitLab CI, Azure Pipelines) that handles your Terraform/OpenTofu deployments, you can integrate Firefly to gain enhanced visualization, monitoring, and Guardrail enforcement for these existing processes.

   • Best For: Teams with mature CI/CD setups who want to leverage Firefly's insights, PR commenting, and Guardrail capabilities without overhauling their current deployment mechanisms.

   • How it Works: You continue to use your own CI/CD runners and pipeline scripts to execute terraform plan and terraform apply. You add small steps to your pipeline using the fireflyci tool (a CLI or Docker image provided by Firefly). This tool sends the plan and apply logs and metadata from your CI/CD jobs to Firefly. Firefly then displays this information in its UI, providing a centralized view and allowing Guardrails to be evaluated.

   • Refer to the Integrating Existing CI/CD Pipelines with Firefly Workflows documentation for detailed instructions.
3. Self-Hosted Runners (Work in Progress - WIP):

   • Concept: This model will offer a hybrid approach where you host and manage the execution runners within your own environment, but these runners are specifically designated for and orchestrated by Firefly Workflows.

   • Best For: Organizations that require deployments to run within their own network boundaries for security or compliance reasons, while still benefiting from Firefly's Workflow management, UI, and Guardrail features.

   • How it Works (Anticipated): You would install and configure Firefly's runner agent in your infrastructure. When a Workflow is triggered, Firefly would delegate the plan or apply task to one of your self-hosted runners. These runners would then execute the Terraform/OpenTofu commands, reporting status and logs back to Firefly.

   • Detailed documentation for self-hosted runners will be available as the feature matures and becomes generally available. This option combines the control of self-hosting with the convenience of Firefly's management plane.

Supported IaC Engines: Firefly Workflows are designed to work seamlessly with both:

• Terraform

• OpenTofu

Understanding Firefly Guardrails

Firefly Guardrails are a critical component for ensuring that your IaC deployments are safe, compliant, and align with your organization's policies. They act as automated checks that evaluate your terraform plan output against a set of predefined rules.

What are Guardrails?

Guardrails are policies you define within Firefly to govern your infrastructure changes. When a terraform plan is generated by a Workflow (either Firefly-managed or an integrated external CI/CD pipeline sending data to Firefly), Guardrails analyze this plan to detect any violations of your configured rules.

If a violation is detected, Guardrails can:

• Block the Deployment: Prevent the terraform apply from proceeding, thus stopping non-compliant changes before they reach your infrastructure.

• Alert an Administrator: Notify relevant personnel about the violation.

• Allow Override (Flexible Block): In some cases, authorized users can override a violation to allow a deployment to proceed, providing flexibility for exceptional circumstances.

Key Benefits of Firefly Guardrails:

• Proactive Policy Enforcement: Catch and prevent policy violations before they become issues in your live environment.

• Risk Mitigation: Reduce the risk of security breaches, cost overruns, misconfigurations, and compliance failures.

• Standardization: Enforce consistent configurations and best practices across all your cloud resources.

• Automation: Automate policy checks, removing the need for manual reviews for many common policy requirements.

• Developer Empowerment: Provide developers with immediate feedback on policy compliance within their existing VCS and CI/CD workflows (e.g., via PR comments).

Guardrail Rule Types

Firefly provides several types of Guardrail rules to cover various aspects of IaC governance:

1. Cost Rules:

   • Purpose: Control cloud spending by setting thresholds on estimated cost changes from a plan.

   • Example: Block any plan that estimates a cost increase of more than $100 or 10%.

2. Policy Rules:

   • Purpose: Enforce adherence to security best practices, compliance standards, or operational guidelines (often using a policy engine like OPA).

   • Example: Ensure all S3 buckets have encryption enabled, or that no EC2 instances are created with public IP addresses.

3. Resource Rules:

   • Purpose: Control specific actions (create, update, delete) on particular resource types, in certain regions, or for specific resource instances.

   • Example: Prevent the deletion of a critical database instance or disallow the creation of resources in a non-approved cloud region.

4. Tag Rules:

   • Purpose: Enforce consistent tagging policies for better resource organization, cost tracking, and automation.

   • Example: Require all resources to have an Environment tag, or ensure the CostCenter tag uses an approved value.

Integrating Guardrails with Workflows

Guardrails are an integral part of the Firefly Workflow ecosystem:

• During plan phase: After a terraform plan is generated, Firefly subjects it to evaluation by all applicable Guardrail rules defined for that Workspace's scope.

• Feedback Loop: Violations are reported back through various channels: in the Firefly UI (often in a dedicated "Guardrails Step" for the run), as comments on pull requests, and via notifications (Slack, email, etc.).

• AI-Powered Remediation: For many violations, Firefly provides AI-generated suggestions and even code snippets to help developers quickly understand and fix the non-compliant IaC.

For a detailed guide on creating and managing these rules, refer to the Creating Guardrail Rules in Firefly documentation.

Related Guides and Examples

• Creating New Firefly Workflows: Step-by-step guide for setting up Firefly-managed workflows.
• Integrating Existing CI/CD Pipelines with Firefly Workflows: How to connect your current CI/CD pipelines to Firefly for visualization and monitoring.
• Creating Guardrail Rules in Firefly: How to define and enforce policies and best practices.
• Firefly Workflows Example Pipelines (GitHub): Real-world pipeline YAMLs and integration templates for Jenkins, GitHub Actions, and more.

Conclusion

Firefly Workflows and Guardrails together provide a comprehensive solution for automating and governing your Infrastructure as Code deployments. Whether you opt for Firefly-managed runners for simplicity, integrate Firefly into your existing CI/CD for enhanced visibility, or anticipate using self-hosted runners for greater control, Firefly empowers you to deploy infrastructure with speed, confidence, and adherence to your organizational standards.

This documentation set will guide you through setting up and utilizing these features to their full potential. We encourage you to explore the specific guides for a deeper understanding of each component.