大数据之ApacheBeam使用FlinkRunner管道参数

例子

Flink集群

mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--runner=FlinkRunner --flinkMaster= --filesToStage=target/word-count-beam-bundled-0.1.jar --inputFile=/path/to/quickstart/pom.xml --output=/tmp/counts" -Pflink-runner

管道参数说明

在丘北等地区,都构建了全面的区域性战略布局,加强发展的系统性、市场前瞻性、产品创新能力,以专注、极致的服务理念,为客户提供网站建设、做网站 网站设计制作按需搭建网站,公司网站建设,企业网站建设,高端网站设计,全网营销推广,外贸网站制作,丘北网站建设费用合理。

FieldDescriptionDefault Value
runnerThe pipeline runner to use. This option allows you to determine the pipeline runner at runtime.Set to FlinkRunnerto run using Flink.
streamingWhether streaming mode is enabled or disabled; true if enabled. Set to true if running pipelines with unbounded PCollections.false
flinkMasterThe url of the Flink JobManager on which to execute pipelines. This can either be the address of a cluster JobManager, in the form "host:port" or one of the special Strings "[local]" or "[auto]""[local]" will start a local Flink Cluster in the JVM while "[auto]" will let the system decide where to execute the pipeline based on the environment.[auto]
filesToStageJar Files to send to all workers and put on the classpath. Here you have to put the fat jar that contains your program along with all dependencies.empty
parallelismThe degree of parallelism to be used when distributing operations onto workers.1
checkpointingIntervalThe interval between consecutive checkpoints (i.e. snapshots of the current pipeline state used for fault tolerance).-1L, i.e. disabled
numberOfExecutionRetriesSets the number of times that failed tasks are re-executed. A value of 0 effectively disables fault tolerance. A value of -1 indicates that the system default value (as defined in the configuration) should be used.-1
executionRetryDelaySets the delay between executions. A value of -1 indicates that the default value should be used.-1
stateBackendSets the state backend to use in streaming mode. The default is to read this setting from the Flink config.empty, i.e. read from Flink config



网站名称:大数据之ApacheBeam使用FlinkRunner管道参数
文章出自:http://bzwzjz.com/article/pgdopg.html

其他资讯

Copyright © 2007-2020 广东宝晨空调科技有限公司 All Rights Reserved 粤ICP备2022107769号
友情链接: 成都网站建设公司 网站制作 网站制作公司 营销型网站建设 教育网站设计方案 自适应网站建设 网站制作 手机网站制作 盐亭网站设计 网站建设改版 成都网站建设 成都网站设计公司 营销网站建设 成都网站建设 成都网站建设公司 高端网站建设 成都品牌网站建设 成都网站建设 网站设计 成都做网站建设公司 成都响应式网站建设公司 温江网站设计