如何利用Docker和Compose提供的弹性堆栈(ELK)在logstash中运行多个流水线

Question

我正在使用this_repo开始使用Docker运行ELK。

我的问题是关于docker-compose文件中的logstash映像：

当我在本地运行时，我有3个文件

#general settings
logstash.yml
#pipeline setting
pipeline.yml
#a pipe line configuration
myconf.conf1

# Example of my pipeline.yml
- pipeline.id: my-first-pipeline
  path.config: "/etc/logstash/my-first-pipeline.config"
  pipeline.workers: 2
- pipeline.id: super-swell-pipeline
  path.config: "/etc/logstash/super-swell-pipeline.config"
  queue.type: persisted

  logstash:
    build:
      context: logstash/
      args:
        ELK_VERSION: $ELK_VERSION
    volumes:
      - type: bind
        source: ./logstash/config/logstash.yml
        target: /usr/share/logstash/config/logstash.yml
        read_only: true
      - type: bind
        #can be either a host path or volume name.
        source: ./logstash/pipeline
        #is the container path where the volume is mounted
        target: /usr/share/logstash/pipeline
        read_only: true
    ports:
      - "5000:5000/tcp"
      - "5000:5000/udp"
      - "9600:9600"
    environment:
      LS_JAVA_OPTS: "-Xmx256m -Xms256m"
    networks:
      - elk
    depends_on:
      - elasticsearch

DockerFile：

ARG ELK_VERSION

# https://www.docker.elastic.co/
FROM docker.elastic.co/logstash/logstash:${ELK_VERSION}

# Add your logstash plugins setup here
# Example: RUN logstash-plugin install logstash-filter-json

logstash.yml

## Default Logstash configuration from Logstash base image.
## https://github.com/elastic/logstash/blob/master/docker/data/logstash/config/logstash-full.yml
#
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]

## X-Pack security credentials
#
xpack.monitoring.enabled: false
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password: changeme

Answer 1

您还需要将pipelines.yml文件挂载到容器中。Logstash查找可能的pipelines.yml文件的默认位置是/usr/share/Logstash/config/（与您已经将Logstash.yml文件挂载到的文件夹相同）。

请注意，您还必须将当前的本地pipelines.yml文件更新为容器内管道的正确路径。准确地说，你需要改变

path.config：“/etc/logstash/my-first-pipeline.config”

另外，请查看这些关于使用Docker运行Logstash以及如何配置多个管道的官方指南：

>

https://www.elastic.co/guide/en/logstash/current/docker-config.html#docker-config

https://www.elastic.co/guide/en/logstash/current/multiplelines.html