Kafka error: No resolvable bootstrap urls

Problem

You are trying to read or write data to a Kafka stream when you get an error message.

kafkashaded.org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

Caused by: kafkashaded.org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers

If you are running a notebook, the error message appears in a notebook cell.

If you are running a JAR job, the error message appears in the cluster driver and worker logs.

Cause

This error message occurs when an invalid hostname or IP address is passed to the kafka.bootstrap.servers configuration in readStream.

This means a Kafka bootstrap server is not running at the given hostname or IP address.

Solution

Contact your Kafka admin to determine the correct hostname or IP address for the Kafka bootstrap servers in your environment.

Make sure you use the correct hostname or IP address when you establish the connection between Kafka and your Apache Spark structured streaming application.