Skip to content

Allow lifecycle.depends_on to target the grants or permissions of a resource - #7435

Queued
andrewnester wants to merge 1 commit into
mainfrom
depends-on-grants
Queued

Allow lifecycle.depends_on to target the grants or permissions of a resource#7435
andrewnester wants to merge 1 commit into
mainfrom
depends-on-grants

Conversation

@andrewnester

@andrewnester andrewnester commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Changes

Allow lifecycle.depends_on to target the grants or permissions of a resource

Why

lifecycle.depends_on[].resource previously accepted only a resource key (foo or jobs.foo). Users who need a resource to wait for another resource's grants or permissions had no way to express that, so the dependency could only be ordered against the parent resource itself.

Accept a qualified key followed by the sub-resource, for example schemas.foo.grants or jobs.foo.permissions. A new
Resources.ResolveDependsOn helper maps these references to deployment plan node keys, and both the direct engine and bundle validation use it. A three-segment reference is accepted only when the resource type has a matching grants or permissions sub-resource, otherwise validation reports "no such resource".

Tests

Added acceptance tests

@andrewnester
andrewnester requested a review from denik November 10, 2026 08:24
@andrewnester
andrewnester requested review from a team as code owners November 10, 2026 08:24
@github-actions github-actions Bot added the DABs DABs related issues label Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: d03eac97a | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=2 unreleased=0 coverage=2 failing=0
test branch main (1e44bb4) latest (v1.25.0)
TestAccept/bundle/lifecycle/depends-on-validation/DMS= ✅ ✅ ✅
TestAccept/bundle/lifecycle/depends-on-validation/DMS=true ✅ ✅ ✅
TestAccept/bundle/lifecycle/depends-on/DMS= ✅ ❌ ❌
TestAccept/bundle/lifecycle/depends-on/DMS=true ✅ ❌ ❌
TestAccept/bundle/lifecycle/depends-on/DMS=: branch ✅ | main (1e44bb4) ❌ | latest (v1.25.0) ❌

main (1e44bb4):

acceptance_test.go:1346: Diff:
        --- bundle/lifecycle/depends-on/output.txt
        +++ /tmp/TestAcceptbundlelifecycledepends-onDMS=3137330015/002/output.txt
        @@ -1,73 +1,13 @@
         
         === Plan records the edges
         >>> [CLI] bundle plan -o json
        -{
        -  "resources.jobs.a_job": [
        -    {
        -      "node": "resources.jobs.z_job",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.jobs.z_job.permissions",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.pipelines.z_pipeline",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.schemas.z_schema.grants",
        -      "label": "depends_on"
        -    }
        -  ],
        -  "resources.jobs.z_job": null,
        -  "resources.jobs.z_job.permissions": [
        -    {
        -      "node": "resources.jobs.z_job",
        -      "label": "${resources.jobs.z_job.id}"
        -    }
        -  ],
        -  "resources.pipelines.z_pipeline": null,
        -  "resources.schemas.z_schema": null,
        -  "resources.schemas.z_schema.grants": [
        -    {
        -      "node": "resources.schemas.z_schema",
        -      "label": "${resources.schemas.z_schema.id}"
        -    }
        -  ]
        -}
        +Error: invalid lifecycle.depends_on: no such resource: jobs.z_job.permissions
        +  at resources.jobs.a_job.lifecycle.depends_on[2].resource
        +  in databricks.yml:13:23
         
        -=== Deploy creates dependencies first
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...
        -Created jobs.a_job
        -Created jobs.z_job
        -Created jobs.z_job.permissions
        -Created pipelines.z_pipeline
        -Created schemas.z_schema
        -Created schemas.z_schema.grants
        -Files: 1 uploaded, 0 deleted
        -Resources: 6 created, 0 changed, 0 deleted, 0 unchanged
        +Error: invalid lifecycle.depends_on: no such resource: schemas.z_schema.grants
        +  at resources.jobs.a_job.lifecycle.depends_on[3].resource
        +  in databricks.yml:14:23
         
        ->>> print_requests.py //jobs //pipelines //unity-catalog
        -POST /api/2.2/jobs/create a_job
         
        -=== Destroy deletes dependents first
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.a_job
        -  delete resources.jobs.z_job
        -  delete resources.pipelines.z_pipeline
        -  delete resources.schemas.z_schema
        -
        -This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
        -  delete resources.schemas.z_schema
        -
        -This action will result in the deletion of the following Lakeflow Spark Declarative Pipelines along with the
        -Streaming Tables (STs) and Materialized Views (MVs) managed by them. Set 'cascade_on_destroy: false' on a pipeline to retain datasets on pipeline deletion:
        -  delete resources.pipelines.z_pipeline
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default
        -
        -Destroy: 4 deleted
        -a_job deleted first
        +Exit code: 1

latest (v1.25.0): same output as main (1e44bb4)

TestAccept/bundle/lifecycle/depends-on/DMS=true: branch ✅ | main (1e44bb4) ❌ | latest (v1.25.0) ❌

main (1e44bb4):

acceptance_test.go:1346: Diff:
        --- bundle/lifecycle/depends-on/output.txt
        +++ /tmp/TestAcceptbundlelifecycledepends-onDMS=true31619053/002/output.txt
        @@ -1,73 +1,13 @@
         
         === Plan records the edges
         >>> [CLI] bundle plan -o json
        -{
        -  "resources.jobs.a_job": [
        -    {
        -      "node": "resources.jobs.z_job",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.jobs.z_job.permissions",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.pipelines.z_pipeline",
        -      "label": "depends_on"
        -    },
        -    {
        -      "node": "resources.schemas.z_schema.grants",
        -      "label": "depends_on"
        -    }
        -  ],
        -  "resources.jobs.z_job": null,
        -  "resources.jobs.z_job.permissions": [
        -    {
        -      "node": "resources.jobs.z_job",
        -      "label": "${resources.jobs.z_job.id}"
        -    }
        -  ],
        -  "resources.pipelines.z_pipeline": null,
        -  "resources.schemas.z_schema": null,
        -  "resources.schemas.z_schema.grants": [
        -    {
        -      "node": "resources.schemas.z_schema",
        -      "label": "${resources.schemas.z_schema.id}"
        -    }
        -  ]
        -}
        +Error: invalid lifecycle.depends_on: no such resource: jobs.z_job.permissions
        +  at resources.jobs.a_job.lifecycle.depends_on[2].resource
        +  in databricks.yml:13:23
         
        -=== Deploy creates dependencies first
        ->>> [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...
        -Created jobs.a_job
        -Created jobs.z_job
        -Created jobs.z_job.permissions
        -Created pipelines.z_pipeline
        -Created schemas.z_schema
        -Created schemas.z_schema.grants
        -Files: 1 uploaded, 0 deleted
        -Resources: 6 created, 0 changed, 0 deleted, 0 unchanged
        +Error: invalid lifecycle.depends_on: no such resource: schemas.z_schema.grants
        +  at resources.jobs.a_job.lifecycle.depends_on[3].resource
        +  in databricks.yml:14:23
         
        ->>> print_requests.py //jobs //pipelines //unity-catalog
        -POST /api/2.2/jobs/create a_job
         
        -=== Destroy deletes dependents first
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.a_job
        -  delete resources.jobs.z_job
        -  delete resources.pipelines.z_pipeline
        -  delete resources.schemas.z_schema
        -
        -This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
        -  delete resources.schemas.z_schema
        -
        -This action will result in the deletion of the following Lakeflow Spark Declarative Pipelines along with the
        -Streaming Tables (STs) and Materialized Views (MVs) managed by them. Set 'cascade_on_destroy: false' on a pipeline to retain datasets on pipeline deletion:
        -  delete resources.pipelines.z_pipeline
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default
        -
        -Destroy: 4 deleted
        -a_job deleted first
        +Exit code: 1

latest (v1.25.0): same output as main (1e44bb4)

@andrewnester
andrewnester added this pull request to stack #7436 November 10, 2026 08:35
@andrewnester
andrewnester force-pushed the depends-on-grants branch 2 times, most recently from f61a763 to f0ccb23 Compare November 10, 2026 08:40
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: d03eac9

Run: 42837010105

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 17 6:11
✅​ aws-windows-latest-4core-16gb 273 15 3:58
✅​ azure linux-2core-8gb 270 17 5:58
✅​ azure-windows-latest-4core-16gb 272 15 3:29
🔄​ gcp linux-2core-8gb 1 270 17 7:22
✅​ gcp-windows-latest-4core-16gb 273 15 4:48
Test Name gcp linux-2core-8gb
🔄​ TestSyncIncrementalFileOverwritesFolder 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:15 gcp-windows-latest-4core-16gb TestAccept
3:09 azure-windows-latest-4core-16gb TestAccept
3:00 aws-windows-latest-4core-16gb TestAccept

denik
denik reviewed Nov 10, 2026
View reviewed changes
Comment thread bundle/config/resources.go Resolved
@andrewnester Isaac
Allow lifecycle.depends_on to target the grants or permissions of a r…
d03eac9 
…esource

lifecycle.depends_on[].resource previously accepted only a resource key
(foo or jobs.foo). Users who need a resource to wait for another resource's
grants or permissions had no way to express that, so the dependency could
only be ordered against the parent resource itself.

Accept a qualified key followed by the sub-resource, for example
schemas.foo.grants or jobs.foo.permissions. A new
Resources.ResolveDependsOn helper maps these references to deployment plan
node keys, and both the direct engine and bundle validation use it. A
three-segment reference is accepted only when the resource type has a
matching grants or permissions sub-resource, otherwise validation reports
"no such resource".

Co-authored-by: Isaac <[email protected]>
@andrewnester
andrewnester enabled stacked auto-merge November 10, 2026 14:44
@andrewnester
andrewnester added this pull request to the merge queue Nov 10, 2026
Any commits made after this event will not be merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denik denik denik left review comments
@janniklasrose janniklasrose janniklasrose approved these changes

Assignees

No one assigned

Labels

DABs DABs related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@andrewnester @eng-dev-ecosystem-bot @denik @janniklasrose