Databricks

Firefly integrates with Databricks to pull in information about your workspace resources directly into your Firefly Inventory.

Prerequisites

• A Databricks workspace.

• A Databricks service principal, with a Client ID and Client Secret.

• Workspace access assigned to the service principal is the minimum needed to connect. Additional entitlements are needed to discover certain object types; see Setup Procedure.

Setup Procedure

1. Create and Configure a Service Principal in Databricks

1. In your Databricks workspace, go to Settings > Identity and access.

2. Under Service principals, click Manage > Add service principal > Add new.

3. Open the service principal and go to Configuration. Workspace access is the minimum entitlement; add Databricks SQL access to inventory SQL objects. Unity Catalog objects require grants on the securables that should be visible.

4. Under Secrets, click Generate secret. Copy the Client ID and Client Secret. The secret is shown only once.

5. For Workspace URL, use the https:// origin from your browser's address bar, with no path or port.

2. Configure in Firefly

1. In Firefly, go to Settings > Integrations.

2. Click Add New > Databricks.

3. Enter a Nickname.

4. Enter your Workspace URL.

5. Enter your Client ID.

6. Enter your Client Secret.

7. Click Next.

8. Click Done.

Supported Assets

Workspace Access

Asset Type	Identifier
Groups	databricks_group
Users	databricks_user
Service Principals	databricks_service_principal
Group Members	databricks_group_member

Permissions

Asset Type	Identifier
Permissions	databricks_permissions

Compute Policies

Asset Type	Identifier
Cluster Policies	databricks_cluster_policy

Instance Pools

Asset Type	Identifier
Instance Pools	databricks_instance_pool

SQL

Asset Type	Identifier
SQL Endpoints	databricks_sql_endpoint
Queries	databricks_query

Jobs

Asset Type	Identifier
Jobs	databricks_job

Pipelines

Asset Type	Identifier
Pipelines (Delta Live Tables)	databricks_pipeline

Repos

Asset Type	Identifier
Repos	databricks_repo

Secrets

Asset Type	Identifier
Secret Scopes	databricks_secret_scope
Secret ACLs	databricks_secret_acl

Unity Catalog

Asset Type	Identifier
Catalogs	databricks_catalog
Schemas	databricks_schema
Grants	databricks_grants

Storage Governance

Asset Type	Identifier
Storage Credentials	databricks_storage_credential
External Locations	databricks_external_location
Volumes	databricks_volume

Data Sharing

Asset Type	Identifier
Shares	databricks_share
Recipients	databricks_recipient

ML Serving

Asset Type	Identifier
Model Serving	databricks_model_serving

Asset type identifiers use the databricks_ prefix, matching the naming used by the underlying Databricks provider.

Discovered Assets

Databricks assets appear in your Inventory with the Discovered status.

Discovered means Firefly has found the asset in your workspace and indexed it, but has not evaluated it against Infrastructure-as-Code. It sits alongside the other inventory states: Managed, Unmanaged, Drifted, and Ghost.

Because Discovered assets are not evaluated against IaC, the following information is hidden for them:

• Drift status

• IaC type

• VCS repository and stack

Discovered does not restrict what you can do with the asset. Databricks assets support codification: see Codification.

Codification

Codification is supported for Databricks assets. Select the asset in your Inventory and choose Codify to generate IaC for it.

Learn more in Codification.

Applications

Databricks can be used in the Application Experience. Databricks assets can be included in Applications, so they are grouped and managed together with the infrastructure they support.

For Databricks, Configuration Backup Only is the supported protection level: scheduled snapshots of the application's configuration for point-in-time restore. Recovery Automation is not currently supported for Databricks.

Managing the Integration

To manage an existing integration, go to Settings > Integrations > Databricks and open the integration menu:

• Edit: update the integration's Name, Workspace URL, Client ID, or Client Secret.

• Delete: remove the integration.

• Scan Assets: trigger an on-demand asset scan.

Limitations

• No drift detection for Databricks assets. They are indexed for visibility but are not evaluated against IaC.

• Metastores (databricks_metastore), workspace configuration (databricks_workspace_conf), SQL dashboards (databricks_sql_dashboard), vector search, and data-sharing provider objects (databricks_provider) are not currently discoverable.

Additional Resources

• Databricks Documentation: Official Databricks documentation, including OAuth machine-to-machine authentication

• Cloud Asset Inventory: How Firefly classifies and displays discovered assets