OpenTelemeter将跟踪导出到弹性APM和弹性OpenDeto

Question

我正在尝试使用python应用程序（基于django）进行测试，以便能够将事务跟踪推送到Elastic APM，稍后可以使用OpenDistro Elastic中的跟踪分析工具查看。

我试过以下方法

方法1：

安装opentelemeter-exporter-otlp

然后，在manage.py文件中，我添加了以下代码，以直接将跟踪发送到弹性APM。

    span_exporter = OTLPSpanExporter(
        endpoint="http://localhost:8200",
        insecure=True
    )

运行代码时，出现以下错误：

Transient error StatusCode.UNAVAILABLE encountered while exporting span batch, retrying in 1s.
Transient error StatusCode.UNAVAILABLE encountered while exporting span batch, retrying in 2s.

方法2：

由于方法1不起作用，我尝试在两者之间使用OpenTelemetry Collector。我用以下方式配置了收集器：

extensions:
  memory_ballast:
    size_mib: 512
  zpages:
    endpoint: 0.0.0.0:55679

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:
  memory_limiter:
    # 75% of maximum memory up to 4G
    limit_mib: 1536
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s

exporters:
  logging:
    logLevel: debug
  otlp/elastic:
    endpoint: "198.19.11.22:8200"
    insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [logging, otlp/elastic]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [logging]

  extensions: [memory_ballast, zpages]

并将我的代码配置为像这样向收集器发送跟踪-

    span_exporter = OTLPSpanExporter(
        endpoint="http://localhost:4317",
        insecure=True
    )

一旦启动程序，收集器日志中就会出现以下错误-

go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send
    go.opentelemetry.io/collector@v0.35.0/exporter/exporterhelper/queued_retry.go:304
go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesExporterWithObservability).send
    go.opentelemetry.io/collector@v0.35.0/exporter/exporterhelper/traces.go:116
go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1
    go.opentelemetry.io/collector@v0.35.0/exporter/exporterhelper/queued_retry.go:155
go.opentelemetry.io/collector/exporter/exporterhelper/internal.ConsumerFunc.Consume
    go.opentelemetry.io/collector@v0.35.0/exporter/exporterhelper/internal/bounded_queue.go:103
go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*BoundedQueue).StartConsumersWithFactory.func1
    go.opentelemetry.io/collector@v0.35.0/exporter/exporterhelper/internal/bounded_queue.go:82
2022-01-05T17:36:55.349Z    error   exporterhelper/queued_retry.go:304  Exporting failed. No more retries left. Dropping data.  {"kind": "exporter", "name": "otlp/elastic", "error": "max elapsed time expired failed to push trace data via OTLP exporter: rpc error: code = Unavailable desc = connection closed", "dropped_items": 1}

我在这里错过了什么？

注意：我使用的是最新版本的opentelemetry sdk和API以及最新版本的collector。

Answer 1

好的，所以使用开放发行版的Elastic来获取跟踪的方法是：

避免使用APM本身。OpenDistro提供了一个名为Data Prepper的工具，必须使用该工具才能将数据（跟踪）从Otel Collector发送到OpenDistro Elastic。

以下是我为Otel Collector所做的配置，用于将数据发送到数据预处理器：

... # other configurations like receivers, etc. 
exporters:
  logging:
    logLevel: debug
  otlp/data-prepper:
    endpoint: "http://<DATA_PREPPER_HOST>:21890"
    tls:
      insecure: true
... # Other configurations like pipelines, etc. 

这就是我如何配置Data Prepper来从Collector接收数据并将其发送到弹性

entry-pipeline:
  delay: "100"
  source:
    otel_trace_source:
      ssl: false
  sink:
    - pipeline:
        name: "raw-pipeline"
raw-pipeline:
  source:
    pipeline:
      name: "entry-pipeline"
  prepper:
     - otel_trace_raw_prepper:
  sink:
    - elasticsearch:
        hosts: [ "http://<ELASTIC_HOST>:9200" ]
        trace_analytics_raw: true