Problem
When using a Databricks Asset Bundle (DAB) project that is passing job parameters, the project reuses the parameters at a job task level. Upon executing write code, the searching for a catalog parameter doesn’t work and fails with the message, name 'catalog' is not defined
.
Example of how you may pass the job parameters
- name: "catalog"
default: 109982_catalog_test
- name: "raw_schema"
default: raw_schema
- name: "trusted_schema"
default: trusted_schema
- name: "refined_schema"
default: refined_schema
How you then pass the job parameters on the task level
- "{{catalog}}"
- "{{raw_schema}}"
- "{{trusted_schema}}"
- "{{refined_schema}}"
Cause
The format for the task level parameters is incorrect.
Solution
On the task level, use the following syntax to pass in job parameters.
- "{{ job.parameters.[catalog] }}"
- "{{ job.parameters.[raw_schema] }}"
- "{{ job.parameters.[trusted_schema] }}"
- "{{ job.parameters.[refined_schema] }}"
For more information on dynamic value references, review the Configure job parameters (AWS | Azure | GCP) documentation.