Problem

When attempting to use the %run command to execute another notebook within a Delta Live Tables (DLT) pipeline you receive an error. 

`%run is not supported in DLT pipelines.`

Cause

The %run command is not supported in DLT pipelines.

Context

DLT pipelines are designed to manage dependencies and orchestrate data transformations in a declarative manner. The %run command is an imperative way to execute code. This difference can lead to issues with reproducibility and dependency management in a DLT pipeline.

Further, when using the %run command in a DLT pipeline, the pipeline does not have visibility into code executed in a referenced notebook. The lack of visibility results in unexpected behavior because the pipeline is unable to properly manage dependencies or track changes to the code.

Last, DLT pipelines are optimized for performance and scalability, and the %run command can introduce overhead that negatively impacts the performance of the pipeline.

Solution

Refactor your code to remove the %run command and use DLT's built-in features for code reuse and dependency management instead. 

1. Identify the code in the referenced notebook that needs to be reused.

2. Extract the reusable code into a separate notebook or function.

3. In the DLT pipeline, import the reusable code using the import statement or call the function directly.

4. Ensure that all dependencies are properly managed and declared in the DLT pipeline.