• Home
  • All articles
  • Unity Catalog
  • Failure to initialize configuration for storage account while registering a model serving model in Unity Catalog

Failure to initialize configuration for storage account while registering a model serving model in Unity Catalog

An external location must be provisioned during the initial Unity Catalog configuration.

Written by alberto.umana

Last published at: October 15th, 2024

Problem

You are trying to register a serving model endpoint to the Unity Catalog metastore when you get a Failure to initialize configuration for storage account error message.

Example error message

ExecutionError: An error occurred while calling o437.ls.
: Failure to initialize configuration for storage account XXXXXdbrmetastoreeus2.dfs.core.windows.net: Invalid configuration value detected for fs.azure.account.keyInvalid configuration value detected for fs.azure.account.key

MlflowException: The following failures occurred while uploading one or more artifacts to abfss://metastore@XXXXXXXXXXX.dfs.core.windows.net/5f2ca358-314a-42da-9a9b-2eca16415dd0/models/e40b317e-9f99-4d86-8384-4a4daa7845d7/versions/b1a714a7-3704-43d9-bbbd-cb81262ef98d: {'/local_disk0/repl_tmp_data/ReplId-76b2b-ace2e-48b5d-6/tmpb8d4j1_k/model/model.pkl': "HttpResponseError('(AuthorizationFailure) This request is not authorized to perform this operation.\\nRequestId:e882052a-301f-0057-45ee-fa7936000000\\nTime:2024-08-30T15:09:54.6446260Z\\nCode: AuthorizationFailure\\nMessage: This request is not authorized to perform this operation.\\nRequestId:e882052a-301f-0057-45ee-fa7936000000\\nTime:2024-08-30T15:09:54.6446260Z')", '/local_disk0/repl_tmp_data/ReplId-76b2b-ace2e-48b5d-6/tmpb8d4j1_k/model/requirements.txt': "HttpResponseError('(AuthorizationFailure) This request is not authorized to perform this operation.\\nRequestId:0328be4e-d01f-0019-65ee-fabcd3000000\\nTime:2024-08-30T15:09:54.6494435Z\\nCode: AuthorizationFailure\\nMessage: This request is not authorized to perform this operation.\\nRequestId:0328be4e-d01f-0019-65ee-fabcd3000000\\nTime:2024-08-30T15:09:54.6494435Z')", '/local_disk0/repl_tmp_data/ReplId-76b2b-ace2e-48b5d-6/tmpb8d4j1_k/model/python_env.yaml': "HttpResponseError('(AuthorizationFailure) This request is not authorized to perform this operation.\\nRequestId:f38fc42e-501f-0008-43ee-fa8bc8000000\\nTime:2024-08-30T15:09:54.6518467Z\\nCode: AuthorizationFailure\\nMessage: This request is not authorized to perform this operation.\\nRequestId:f38fc42e-501f-0008-43ee-fa8bc8000000\\nTime:2024-08-30T15:09:54.6518467Z')", '/local_disk0/repl_tmp_data/ReplId-76b2b-ace2e-48b5d-6/tmpb8d4j1_k/model/MLmodel': "HttpResponseError('(AuthorizationFailure) This request is not authorized to perform this operation.\\nRequestId:bad77956-201f-0007-75ee-fa663e000000\\nTime:2024-08-30T15:09:54.6167981Z\\nCode: AuthorizationFailure\\nMessage: This request is not authorized to perform this operation.\\nRequestId:bad77956-201f-0007-75ee-fa663e000000\\nTime:2024-08-30T15:09:54.6167981Z')", '/local_disk0/repl_tmp_data/ReplId-76b2b-ace2e-48b5d-6/tmpb8d4j1_k/model/conda.yaml': "HttpResponseError('(AuthorizationFailure) This request is not authorized to perform this operation.\\nRequestId:e5ee5a65-101f-0069-1aee-facf17000000\\nTime:2024-08-30T15:09:54.5864086Z\\nCode: AuthorizationFailure\\nMessage: This request is not authorized to perform this operation.\\nRequestId:e5ee5a65-101f-0069-1aee-facf17000000\\nTime:2024-08-30T15:09:54.5864086Z')"}

Cause

The issue can occur when an external location is not provisioned during the initial Unity Catalog configuration.

  • To register a serving model with Unity Catalog, the serving model needs to have an external storage_root defined on the Unity Catalog metastore.
  • The root metastore may have been blocked by a firewall, resulting in a failure when uploading model serving artifacts. 

Example

You can validate your storage root location with this sample code to see if you have properly configured an external location. Replace <catalog-name> and <schema-name> with your own catalog and schema when running the sample code.

%sql

DESCRIBE CATALOG EXTENDED <catalog-name>;
DESCRIBE SCHEMA EXTENDED <schema-name>;

If Storage Root and Storage Location values are empty when you get the results, an external location was not assigned to the catalog or the schema.

Screenshot showing example results of running the sample code. The Storage Root and Storage Location values are present in the results.

Solution

You cannot modify the catalog or schema of an existing metastore. You must create a new Unity Catalog metastore (AWS | Azure) with a different storage location.

After creating a new metastore, you have to re-register your model in the new location. This ensures that artifacts are uploaded to the specified catalog or schema, instead of the default root metastore storage. Storing artifacts in the default root metastore storage is not recommended.

For more details, review the Model serving with Databricks (AWS | Azure) documentation.