“Connection pool is full” error when pulling models from S3 with MLflow

Increase the maximum size of the MLflow connection pool.

Written by jairo.prado

Last published at: February 12th, 2025

Problem

You are pulling models from S3 using MLflow when you get a Connection pool is full error message.

WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: XXXXXXXX.XXX.XXXXXXXX.com. Connection pool size: 10

 

Cause

The warning message is generated when the connection pool in urllib3 reaches its maximum size. The default maximum size of the connection pool is 10, which may not be sufficient for some use cases.

 

Solution

The warning message is benign and does not result in data loss. To avoid seeing these warning messages, you can increase the maximum size of the connection pool by setting the MLFLOW_HTTP_POOL_CONNECTIONS and MLFLOW_HTTP_POOL_MAXSIZE environment variables to a value larger than 10, such as 30.

 

For more information, review the mlflow.environment_variables documentation.