You are using the Databricks Connect client with Databricks Runtime 6.4 and receive an error message which states that the client does not support the cluster.
Caused by: java.lang.IllegalArgumentException: The cluster is running server version `dbr-6.4` but this client only supports Set(dbr-5.5). You can find a list of client releases at https://pypi.org/project/databricks-connect/#history, and install the right client version with `pip install -U databricks-connect==<version>`. For example, to install the latest 5.1 release, use `pip install -U databricks-connect==5.1.*`. To ignore this error and continue, set DEBUG_IGNORE_VERSION_MISMATCH=1.
Improvements were made to Databricks Runtime 6.4 which are incompatible with the Databricks Connect client 6.4.1 and below.
Upgrade the Databricks Connect client to 6.4.2.
pip install databricks-connect==6.4.2