• Databricks
  • Databricks
  • View Azure Databricks documentation Azure docs
  • View Azure Databricks documentation Azure docs
  • Support
  • Feedback
  • Try Databricks
  • Help Center
  • Documentation
  • Knowledge Base
  • Forums
  • Training
Databricks Knowledge Base
  • Databricks administration
  • AWS infrastructure
  • Business intelligence tools
  • Clusters
  • Data management
  • Data sources
  • Databricks File System (DBFS)
  • Developer tools
  • Delta Lake
  • Jobs
  • Job execution
  • Libraries
  • Machine learning
  • Metastore
  • Metrics
  • Notebooks
  • Security and permissions
  • Streaming
    • Streaming with File Sink: Problems with recovery if you change checkpoint or output directories
    • How to set up Apache Kafka on Databricks
    • Handling partition column values while using an SQS queue as a streaming source
    • How to restart a structured streaming query from last written offset
    • How to switch a SNS streaming job to a new SQS queue
    • Checkpoint files not being deleted when using display()
    • Checkpoint files not being deleted when using foreachBatch()
  • Visualizations
  • Python with Apache Spark
  • R with Apache Spark
  • Scala with Apache Spark
  • SQL with Apache Spark

Updated Jun 30, 2020

Send us feedback

  • Documentation
  • Streaming

Streaming

These articles can help you with Structured Streaming and Spark Streaming (the legacy Spark streaming feature).

  • Streaming with File Sink: Problems with recovery if you change checkpoint or output directories
  • How to set up Apache Kafka on Databricks
  • Handling partition column values while using an SQS queue as a streaming source
  • How to restart a structured streaming query from last written offset
  • How to switch a SNS streaming job to a new SQS queue
  • Checkpoint files not being deleted when using display()
  • Checkpoint files not being deleted when using foreachBatch()

© Databricks 2020. All rights reserved. Apache, Apache Spark, Spark, and the Spark logo are trademarks of the Apache Software Foundation.

Send us feedback | Privacy Policy | Terms of Use