从Airflow Celery Executor连接到启用TLS的Elasticache Redis

Question

我目前在Airflow 1.10上使用芹菜执行器。我的经纪人是AWS Elasticache Redis（v.5.0.6）。如何在传输中启用加密？

根据Airflow源代码，需要ssl_keyfile、ssl_certfile和ssl_ca_certs。

elif 'redis://' in broker_url:
    broker_use_ssl = {'ssl_keyfile': conf.get('celery', 'SSL_KEY'),
                      'ssl_certfile': conf.get('celery', 'SSL_CERT'),
                      'ssl_ca_certs': conf.get('celery', 'SSL_CACERT'),
                      'ssl_cert_reqs': ssl.CERT_REQUIRED}
 

https://github.com/apache/airflow/blob/1.10.10/airflow/config_templates/default_celery.py#L68-L72级

但是Elasticache Redis不提供这些TLS证书。官方文档仅解释了redis-cli的解决方案，该解决方案使用TLS隧道。

https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/in-transit-encryption.html#connect-tls

Airflow Celery Executor是否支持Elasticache Redis上传输的加密？如果是，我们如何实施？

Answer 1

以下气流配置适用于elasticache和redis ssl注意redis上的额外

broker_url = rediss://:password@host:port/db?ssl_cert_reqs=required
broker_use_ssl = { "ssl_cert_reqs": ssl.CERT_REQUIRED }

ssl_active = False
ssl_key =
ssl_cert =
ssl_cacert =