operator用于产生特定的DAG节点
1 | https://airflow.apache.org/docs/apache-airflow/stable/python-api-ref.html#operators |
下面是常用的operator及其用法
BaseOperator#
1 | https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/models/baseoperator.html |
用法：
BaseSensorOperator#
1 | https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/sensors/base.html#BaseSensorOperator.poke |
用法：
MySQLToS3Operator#
这个已经过时了，建议使用SqlToS3Operator
1 | https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_modules/airflow/providers/amazon/aws/transfers/mysql_to_s3.html |
用法：
SqlToS3Operator#
1 | https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/transfer/sql_to_s3.html#howto-operator-sqltos3operator |
用法：
MongoToS3Operator#
1 | https://airflow.apache.org/docs/apache-airflow-providers-amazon/1.2.0/_modules/airflow/providers/amazon/aws/transfers/mongo_to_s3.html |
用法：
BashOperator：执行bash命令#
1 | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/bash.html |
PythonOperator：调用python代码#
1 | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html |
SparkOperators#
1 | https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/operators.html |
EmailOperator：发送邮件#
1 | https://bhavaniravi.com/blog/sending-emails-from-airflow/ |
HTTPOperator : 发送 HTTP 请求#
1 | https://airflow.apache.org/docs/apache-airflow-providers-http/stable/operators.html |
MySqlOperator : 执行 SQL 命令#
1 | https://airflow.apache.org/docs/apache-airflow-providers-mysql/stable/operators.html |
SSHExecuteOperator，airflow1.9.0之后废除，改成 SSHOperator#
1 | https://newbedev.com/airflow-how-to-ssh-and-run-bashoperator-from-a-different-server |
SparkSubmitOperator：提交spark任务#
1 | https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/_api/airflow/providers/apache/spark/operators/spark_submit/index.html#airflow.providers.apache.spark.operators.spark_submit.SparkSubmitOperator |
除了这些基本的构建块之外，还有更多的特定处理器：DockerOperator，HiveOperator，S3FileTransferOperator，PrestoToMysqlOperator，SlackOperator
