Problem

When calling a Vertex AI text embedding model from a Databricks notebook, the API call fails with a permission error. Although the project ID is set to a valid project (with the API enabled and a proper Google service account is configured) the error indicates that the API has not been used in the project, and it is not the intended project for this operation.

PermissionDenied: 403 Vertex AI API has not been used in project <project-id> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/aiplatform.googleapis.com/overview?project=<project-id> then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

Cause

Your Vertex AI model is attempting to access resources from the wrong project that does not have the Vertex AI API enabled or the API lacks the required permissions. 

Trying to access an incorrect project can result from incorrect project settings or permissions within Vertex AI. It can also happen if the default project is not correctly set in the Databricks environment.

Solution

You should configure the project and credentials in your notebook before you make any Vertex AI API calls. This ensures that the correct project is used. 

You can copy and paste this sample code to use in your notebook.

%python
import google.auth
from vertexai import vertexai

creds, _ = google.auth.default(quota_project_id='<your-project-id>')
vertexai.init(project='<your-project-id>', credentials=creds)

For details on obtaining your project ID, refer to Google’s Locate the project ID documentation.

For more information on Vertex AI, refer to Google’s Vertex Generative AI SDK for Python documentation.