Problem

When the owner of vector search indexes is removed, the indexes stop working and you are required to recreate them.

Cause

Removing the owner of vector search indexes breaks several dependencies, including DLT pipelines. DLT pipelines are designed to run as the owner of the index. Since the user no longer exists, it is not possible to validate the permissions, resulting in failures.

Solution

Instead of assigning ownership of vector search indexes to individual users, use a service principal. 

In the vector search client, add the following parameters. 

• workspace_url
• service_principal_client_id
• service_principal_client_secret

For details, refer to the databricks.vector_search package documentation.

Then, enable the use of service principals for vector search index ownership. Follow the instructions in the Manage service principals documentation.