Creating Workflows

Related Guides and Examples:

• Overview: Firefly Workflows and Guardrails: High-level concepts and options.

• Integrating Existing CI/CD Pipelines with Firefly Workflows: For hybrid or external CI/CD integration.

• Creating Guardrail Rules in Firefly: How to define and enforce policies and best practices.

• Creating Projects: How to create projects to group IaC orchestration resources and control access.

• Creating Variable Sets: Managing reusable variable collections.

• Triggering Workspace Deployment: How to trigger a workspace deployment from the Firefly UI.

• Firefly Workflows Example Pipelines: Real-world pipeline and integration templates for Jenkins, GitHub Actions, and more.

Firefly Workflows offer a streamlined and powerful way to automate your Terraform, OpenTofu, and Terragrunt deployments directly within the Firefly platform. When you choose to create a new IaC pipeline with Firefly Runners, you leverage Firefly's fully managed infrastructure to execute your IaC operations (plan and apply). This approach provides built-in drift detection, policy guardrails, automatic logging, and a secure, scalable experience for your infrastructure deployments. For organizations with specific security or compliance needs, Firefly also supports self-hosted runners, allowing you to run workflows within your own environment while still benefiting from Firefly's orchestration and governance features.

This document provides a comprehensive guide to creating and configuring a new Firefly Runners-managed workflow.

Introduction to Firefly Runners

Firefly Runners are ideal for:

• Fully Managed IaC Operations: Firefly handles the entire execution environment, eliminating the need to configure and maintain your own runners or complex pipeline scripts.

• Built-in Drift Detection: Automatic periodic scanning to detect infrastructure drift, configuration changes and deployment failures.

• Policy Guardrails: Integrated policy enforcement with automated checks and violations reporting.

• Secure Execution: Firefly provides a secure, isolated environment for running your Terraform/OpenTofu/Terragrunt operations.

• Automated Logging: Comprehensive logging and audit trails for all IaC operations.

• VCS Integration: Seamless integration with your Version Control System (VCS) like GitHub, GitLab, etc.

• Flexible Runner Options: Choose between managed runners for simplicity or self-hosted runners for enhanced security, compliance, and network control.

Each workflow in Firefly corresponds to a Workspace. A Workspace represents a specific IaC stack (a collection of Terraform/OpenTofu/Terragrunt files defining a set of resources) and its deployment configuration.

Key Concepts for Firefly Runners:

• Workspace: The core entity representing your IaC configuration, its associated VCS repository, variables, and run history.

• Firefly Runners: Firefly offers two types of runners for executing your IaC commands (plan, apply, and others):

  • Managed Runners: Firefly's fully managed, cloud-hosted infrastructure that securely runs your IaC operations without any setup or maintenance required on your end.

  • Self-Hosted Runners: For organizations with strict security, compliance, or network requirements, Firefly supports self-hosted runners. These allow you to run all IaC operations within your own environment or network, giving you full control over execution, networking, and data residency. Self-hosted runners are ideal when you need to keep sensitive credentials or resources within your own infrastructure.

• VCS Integration: Connects Firefly to your Git provider (e.g., GitHub) to trigger workflows based on code changes (pull requests, merges).

• Pull Request (PR) Trigger: Optionally, creating a PR in your connected repository automatically triggers a plan for the corresponding Workspace.

• Merge Trigger: Optionally, merging a PR into the Workspace's default branch automatically triggers a apply (or waits for manual approval, depending on Workspace settings).

• Periodic Plans: Optionally, scheduled drift detection that runs periodic plan operations to identify infrastructure changes and alert on deployment failures.

Prerequisites

Before creating a Firefly Runners workflow, ensure you have:

• An active Firefly account with appropriate permissions to create Workflows and Workspaces.

• Your Terraform, OpenTofu, or Terragrunt code hosted in a supported Version Control System (e.g., GitHub, GitLab, Azure DevOps, etc.).

• Your VCS provider integrated with Firefly. This is done at an organizational level within Firefly settings.

• Understanding of your IaC project structure (working directory, variable files, etc.).

Creating a New Firefly Runners Workflow: Step-by-Step

Follow the wizard in the Firefly UI to create a new workspace with Firefly Runners.

Procedure:

Step 1: Navigate to Workspaces:

In the Firefly UI, click on Workflows > Workspaces from the main navigation menu.

Step 2: Add New Workspace:

Click on the + Add New Workspace button.

Step 3: Select IaC Provisioning Engine:

Choose the Infrastructure as Code engine your project uses:

• Terraform

• OpenTofu

• Terragrunt

Click Next.

Step 4: Choose Workflow Type:

Select from the two available options:

Create new IaC pipeline (Firefly Runners) Recommended

Firefly will fully manage IaC operations using its execution engine, with built-in drift detection, policy guardrails, and automatic logging. This option supports both managed runners (Firefly's cloud infrastructure) and self-hosted runners (your own infrastructure) for maximum flexibility and security.

Integrate into an existing IaC pipeline

If you already have a CI pipeline for your IaC projects, Firefly can seamlessly integrate into it. By choosing this option, Firefly will help you add the necessary steps and configurations to your existing pipeline.

Deprecated: Generate third-party IaC pipeline option is deprecated. This option is no longer available. Please use the Create new IaC pipeline (Firefly Runners) option instead.

Step 5: General Configuration

Configure your workspace's basic information and organization settings.

Name (Required): Choose a clear, descriptive name that identifies your workspace's purpose and environment. Workspace names must be unique within your assigned project. Description (Optional): Add context about the workspace's purpose, team ownership, or infrastructure it manages. Labels (Optional): Use labels to categorize and filter workspaces across your organization. Project Assignment (Optional): Assign this workspace to a project for enhanced organization and access control. If you don't assign a project, the workspace will be accessible to all Firefly users in your organization.

Benefits of Project Assignment:

• Access Control: Limit workspace access to specific teams or users.

• Variable Inheritance: Automatically inherit project-level variables.

• Resource Organization: Group related workspaces together.

Click Next to proceed to VCS configuration.

Step 6: VCS Configuration

Connect your workspace to your Version Control System and specify where your infrastructure code is located.

VCS Integration (Required): Select your Version Control System provider from the available integrations. Code Repository (Required): Choose the repository containing your Infrastructure as Code files. Default Branch (Required): Select the primary branch where your production-ready infrastructure code resides. This branch will be used for automatic deployments when merging pull requests if configured in the execution settings. Working Directory (Optional): Specify the directory path within your repository where your root infrastructure module is located. Leave blank if your IaC files are in the repository root. The working directory should contain your root module files (typically main.tf, variables.tf, and outputs.tf).

Click Next to configure variables.

Step 7: Variables Configuration (Optional)

Define the variables needed for your infrastructure deployment. Variables can be inherited from variable sets and project variables or defined specifically for this workspace.

Variable Sets (Optional): Attach existing variable sets to reuse common configurations across multiple workspaces.

What are Variable Sets?

• Pre-defined collections of variables that can be shared across workspaces.

• Ideal for common configurations like AWS regions, instance types, or team-specific settings.

• Can include both regular and sensitive variables.

Note: Variable sets are managed separately in Firefly. If you need to create new variable sets, you can do so after workspace creation. Check the Creating Variable Sets guide for more information.

Inline Variables (Optional): Define variables specific to this workspace that supplement or override inherited variables.

• Name: Variable name.

• Value: Variable value.

• Sensitive: Mark sensitive values appropriately. If you mark a variable as sensitive, it will be encrypted and hidden from logs and the UI.

• Environment Variable: Export as environment variable. If you export a variable as an environment variable, it will be available to your workspace as an environment variable.

Variable Precedence

When variables are defined in multiple places, Firefly follows this precedence order (highest to lowest):

• Inline Variables (this workspace)

• Variable Sets (attached to workspace)

• Project Variables (inherited from project)

• Organization Variables (inherited from organization)

• Default Values (defined in IaC code)

Click Next to configure execution settings.

Step 8: Execution Configuration

Configure when and how your workspace should execute infrastructure operations.

Run on Pull Request (Recommended): Enable this option to automatically run plan when pull requests are opened against your default branch.

Benefits of Run on Pull Request:

• Early Feedback: See infrastructure changes before merging.

• Code Review: Review plan output alongside code changes.

• Prevent Errors: Catch issues before they reach production.

• Compliance: Ensure changes meet policy requirements.

What Happens when Run on Pull Request is enabled:

1. Developer opens a pull request.

2. Firefly automatically triggers a plan operation.

3. plan results are posted as PR comments.

4. Policy violations are flagged if any exist.

5. Reviewers can see infrastructure impact before approving.

Run on Merge (For Production Workspaces): Enable this option to automatically run apply when pull requests are merged into your default branch.

Benefits of Run on Merge:

• Continuous Deployment: Immediate infrastructure updates.

• Consistency: Ensures every merge results in deployment.

• Reduced Manual Work: Eliminates manual deployment steps.

• Audit Trail: Complete history of all changes.

What Happens when Run on Merge is enabled:

1. Pull request is merged into default branch.

2. Firefly automatically triggers an apply operation.

3. Infrastructure is updated according to the merged changes.

4. Notifications are sent to configured channels.

Caution: Only enable auto-apply for well-tested workspaces. Consider manual approval for production environments.

Set Periodic Plan (Optional): Enable this option to automatically detect configuration drift and deployment failures.

Configuration Options for Set Periodic Plan:

Simple Scheduling:

• Once every X hours: Enter interval (e.g., 5, 12, 24).

• Minimum frequency: Every hour.

• Common patterns: Every 6 hours, Daily (24 hours), Twice daily (12 hours).

Advanced Scheduling:

• Click the rotation icon to use cron expressions.

• Examples:

  • 0 */6 * * * - Every 6 hours.

  • 0 9 * * 1-5 - Daily at 9 AM, Monday through Friday.

  • 0 2 * * 0 - Weekly on Sunday at 2 AM.

Benefits of Set Periodic Plan:

• Early Detection: Identify manual changes to your infrastructure or deployment failures.

• Compliance: Ensure infrastructure matches your code.

• Monitoring: Track deployment health over time.

• Alerting: Get notified when drift is detected or deployment fails.

Apply Rules

Choose how apply operations are handled:

Manual Apply (Recommended for Production)

• Requires manual approval within Firefly after plan generation.

• Provides an additional safety gate before infrastructure changes.

• Allows for final review of execution plan.

• Ideal for production environments and critical infrastructure.

Auto Apply (For Development/Staging)

• Automatically applies changes after successful plan.

• Enables fully automated deployment pipeline.

• Suitable for development and staging environments.

• Reduces deployment time and manual intervention.

Terraform/OpenTofu Version (Required): Select the version of your Infrastructure as Code tool that matches your project requirements.

Click Next to complete workspace setup.

Step 9: Completion

Congratulations! Your Firefly Runners workspace has been successfully created and is ready to manage your infrastructure deployments.

What Happens Next

Immediate Actions:

1. Initial State Check: Firefly automatically runs a baseline plan to establish your workspace's initial state.

2. Configuration Verification: Review your workspace dashboard to confirm all settings are correct.

3. Integration Testing: Your VCS integration is now active and will respond to pull requests and merges.

Recommended Next Steps

1. Test Your Configuration

• Create a test pull request to verify the PR trigger works.

• Review the generated plan output.

• Ensure all variables are correctly configured.

2. Set Up Notifications

• Configure notifications to your desired channels.

• Get alerted when plans fail or succeed.

• Keep stakeholders informed of infrastructure changes.

3. Add Guardrails

• Implement policy-as-code rules to enforce compliance.

• Set up cost controls and security policies.

• Prevent deployments of resources with no tags.

Quick Reference

Your Workspace Configuration:

• Automatic Triggers: Pull requests and merges will trigger IaC operations.

• Drift Detection: Periodic plans will monitor for configuration changes.

• Security: Sensitive variables are encrypted and redacted from logs.

• Audit Trail: Complete history of all infrastructure changes.

Common Actions:

• Manual Deployment: Use the Deploy button to run IaC operations manually.

• Edit Variables: Update workspace variables without recreating the workspace.

• View Logs: Access detailed execution logs for troubleshooting.

• Manage Periodic Plans: Adjust drift detection frequency as needed.

Additional Resources

• Triggering Workspace Deployment: How to trigger a workspace deployment from the Firefly UI.

• Creating Guardrail Rules: How to define and enforce policies and best practices.

• Creating Variable Sets: Managing reusable variable collections.

• Creating Projects: How to create projects to group IaC orchestration resources and control access.

Managing Your Firefly Runners Workspace

After creation, you can manage your workspace through various actions:

Edit Workspace

• Navigate to your workspace, in the actions menu click Edit.

• Modify any configuration set during creation.

• Click Save to save the changes.

Edit Variables

• Navigate to your workspace, in the actions menu click Edit Variables.

• Attach variable sets or define inline variables specifically for this workspace. These variables will override any inherited ones.

• Click Save to save the changes.

Set Periodic Plan

• Navigate to your workspace, in the actions menu click Set Periodic Plan.

• Enable and configure scheduled plan runs to detect infrastructure drift over time.

• Click Save to set the time interval for the periodic plan.

Deploy Workspace

• Navigate to your workspace, in the actions menu click Deploy.

• Trigger a plan and optionally an apply execution for the workspace using your selected branch and settings. You can also add additional CLI arguments for Terraform/OpenTofu/Terragrunt.

• Click Deploy to trigger the execution.

Delete Workspace

• Navigate to your workspace, in the actions menu click Delete.

• Permanently delete this workspace from Firefly. You can optionally run a destroy before deletion to clean up infrastructure. This action cannot be undone.

• Click Delete to confirm the deletion.

Best Practices

• Workspace Organization: Use descriptive names and labels for easy identification.

• Variable Management: Leverage variable sets for consistency across workspaces.

• Security: Always use sensitive variables for credentials and secrets.

• Runner Selection: Choose managed runners for simplicity, self-hosted for security/compliance requirements.

• Monitoring: Enable periodic plans for proactive drift detection and deployment failures.

• Policy Enforcement: Implement guardrails to maintain compliance.

• Documentation: Maintain clear descriptions and labels for team collaboration.

For hybrid or CI/CD integration, see the Integrating Existing CI/CD Pipelines with Firefly Workflows guide. For real-world pipeline examples, visit the workflows-examples GitHub repository.

By following this guide, you can effectively create and manage Firefly Runners workflows, providing a fully integrated, scalable, and secure infrastructure automation experience.