Apache Spark DStream is not supported

Problem

You are attempting to use a Spark Discretized Stream (DStream) in a Databricks streaming job, but the job is failing.

Cause

DStreams and the DStream API are not supported by Databricks.

Solution

Instead of using Spark DStream, you should migrate to Structured Streaming.

Review the Databricks Structured Streaming in production documentation for more information.