最近在测试相关内容, 以下列了一些我觉得有用的link
官方文档
Spark Structured Streaming Programming Guide
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html
databricks Structured Streaming
https://docs.databricks.com/spark/latest/structured-streaming/index.html
databricks上的一个汇总link
Anthology of Technical Assets on Apache Spark’s Structured Streaming
https://databricks.com/blog/2017/08/24/anthology-of-technical-assets-on-apache-sparks-structured-streaming.html
里面
Spark Structured Streaming A new high-level API for streaming
https://databricks.com/blog/2016/07/28/structured-streaming-in-apache-spark.html
Real-time Streaming ETL with Structured Streaming in Apache Spark 2.1 共5篇
https://databricks.com/blog/2017/01/19/real-time-streaming-etl-structured-streaming-apache-spark-2-1.html
https://databricks.com/blog/2017/02/23/working-complex-data-formats-structured-streaming-apache-spark-2-1.html
https://databricks.com/blog/2017/04/26/processing-data-in-apache-kafka-with-structured-streaming-in-apache-spark-2-2.html
https://databricks.com/blog/2017/05/08/event-time-aggregation-watermarking-apache-sparks-structured-streaming.html
https://databricks.com/blog/2017/05/18/taking-apache-sparks-structured-structured-streaming-to-production.html
CSDN上的一篇
Databrick 's Blog on Spark Structured Streaming Summary
https://blog.csdn.net/asd136912/article/details/82147657
系统示例测试
目前测试2.3.0运行自带的structured_kafka_wordcount.py会报错
bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.3.0 examples/src/main/python/sql/streaming/structured_kafka_wordcount.py testspark:9093 subscribe test1
错误信息
py4j.protocol.Py4JJavaError: An error occurred while calling o32.load.
: org.apache.kafka.common.config.ConfigException: Missing required configuration "partition.assignment.strategy" which has no default value.
at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:124)
2.2.2和2.3.2都可以
