Flink集群部署的方式有以下几种,在本文中主要介绍Flink on yarn:
1 | Yarn |

参考:
1 | https://www.slideshare.net/tillrohrmann/redesigning-apache-flinks-distributed-architecture-flink-forward-2017 |
Flink on yarn有2种运行方式:

Flink架构

第一种是Session mode
定义参考:《Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications》
1 | https://books.google.com.sg/books?id=7OiRDwAAQBAJ&pg=PT265&lpg=PT265&dq=flink+job+mode+session+mode&source=bl&ots=0fMg-gnzyf&sig=ACfU3U1-W4Az3WWJiUBwTpm0tGbsi2UpGw&hl=zh-CN&sa=X&ved=2ahUKEwiMwJjS-uDpAhVTIqYKHbgABkwQ6AEwBnoECAoQAQ#v=onepage&q=flink%20job%20mode%20session%20mode&f=false |
即启动一个yarn session,然后submit flink job到这个yarn session上

有2个步骤:
1.启动一个yarn session

2.提交任务到flink集群上

参考:
1 | https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/yarn_setup.html |
第二种是Job mode
即提交一个独立的flink任务到yarn上面,一旦这个job结束了,Flink集群上的资源就会释放

flink kerberos参考
1 | https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/security/security-kerberos.html |
即在conf/flink-conf.yaml中配置
1 | security.kerberos.login.use-ticket-cache: false |