Problem
You are attempting to create a row filter with a fully qualified table name (for example, Test_catalog.Test_schema.Test_table_name
) in a Databricks environment, when you encounter a NO SUCH CATALOG EXCEPTION
error message. This error message suggests that the specified catalog does not exist, even though it does.
This issue occurs specifically when you are trying to apply a row filter using the ALTER TABLE
command with a fully qualified table name. However, if you set the catalog and schema with the use
command before running the ALTER TABLE
command, the error does not occur.
Cause
The ALTER TABLE
command expects the function location to be specified when it is applied to the table. If the function location is not specified, Databricks searches for the function in the default hive_metastore
. If the default hive_metastore
does not contain the required function it leads to a NO SUCH CATALOG EXCEPTION
error.
Solution
Specify the function location when you use the ALTER TABLE
command to apply a row filter on a table.
Example syntax
%sql
ALTER TABLE <catalog>.<schema>.<table_name> SET ROW FILTER <catalog>.<schema>.<function_name> ON (<column_name>);
Using the example table name it might look like this:
%sql
ALTER TABLE Test_catalog.Test_schema.Test_table_name
SET ROW FILTER Test_catalog.Test_schema.Test_fun ON (Test_col_name);
For more information, review the ROW FILTER clause (AWS | Azure | GCP) documentation.