Problem

You created a VPC peering connection and configured an Amazon Redshift cluster in the peer network. When you attempt to access the Redshift cluster, you get the following error:

Error message: OperationalError: could not connect to server: Connection timed out

Cause

This problem can occur if:

1. VPC peering is misconfigured.
2. The corresponding port is blocked at the network component level, due to Security Groups (SG), Network Access Control Lists (NACL), or other routing issues.

Troubleshooting

Step 1. Test the connection

Check the AWS console and make sure the Redshift cluster is online in the target VPC. Run the following Bash commands to see if the connection to the cluster can be established:

%sh nc -zv <hostname> <port>
%sh lft <hostname>:<port>
%sh telnet <hostname> <port>

The connection should succeed and show the port as open. If not, go to step 2.

xx.c1prsbaxxxxx.us-west-2.redshift.amazonaws.com [192.168.50.209] 5439 (?) open

Step 2. Check for VPC peering or DNS error

If the connection fails with either of the following errors, then the issue is a VPC peering or DNS error.

• The following error code indicates a VPC peering issue. Go to Step 3.

  xx.c1prsbaxxxxx.us-west-2.redshift.amazonaws.com [192.168.50.108] 5439 (?) : Connection timed out

  
• The following error code indicates a DNS lookup error:

  redshift.c1prsbaxxxx.us-west-2.redshift.amazonaws.com: forward host lookup failed: Unknown hos

• In this case, check that:
  • The Redshift cluster is up.
  • The hostname is typed correctly.
  • The redshift cluster IP address works, in place of the hostname.
  • Check the DNS resolution using nslookup:

    nslookup <hostname>

• If these checks appear normal, the error may lie somewhere else. Go to Step 4.

Step 3. Check the VPC peering and DNS settings

If Step 2 revealed a VPC peering or DNS issue:

1. Validate the peering configuration.

• Make sure the peering connections from requestor and acceptor VPC IDs are correct. Note the peering connection id, CIDR of Requestor and CIDR of the acceptor.
• Confirm that the peering connection is active from the target VPC.
  
• Make sure DNS resolution check is turned on for the Redshift VPC. When you're done, go to Step 4.
  

2. Check the following components from the Databricks Deployment VPC.

• Verify that the correct CIDR of the target VPC (Redshift) is added to the route table of the deployment VPC and routed to the correct target, which is the peering connection id.
  
• Check the NACL attached to the subnets and allow all traffic to Redshift, for both inbound and outbound rules.
  
• Check the Security Group of the deployment VPC. It should be an unmanaged security group. Make sure that port 5439 ( redshift ) is open to the target security group that is attached to Redshift.
  

3. Check the following components from the Redshift VPC.

• Verify the correct CIDR of the target VPC (Databricks deployment) is added to the route table of the deployment VPC and routed to correct target -peering connection id.
  
• Check NACL and allow all traffic from redshift (inbound rules and outbound rules).
  
• Check the Security Group of the Redshift security group. Make sure that port 5439 ( redshift ) is open to the target security group (the unmanaged security group inside the Databricks VPC).
  
  

Step 4. Validate and verify the connectivity between the peered VPCs

Perform the connection test again:

nc -zv <hostname> <port>

The connection test should succeed. If not, contact Databricks Support.

xx.c1prsbaxxxxx.us-west-2.redshift.amazonaws.com [192.168.50.209] 5439 (?) open