Apache Beam S3文件系统扩展始终需要aws区域输入，即使在我的项目中的其他管道中，我们也不需要aws

Question

我有一个用java编写的项目，其中有多个beam管道，我将这些管道编译成一个jar文件，以便在服务器上执行。目前一切都可以工作，我只是从GCP参考资料中读取，但我只是添加了一个写入S3的管道。S3部分独立工作，但现在当我尝试运行其他只使用GCP的管道时，它会抛出一个异常，因为我没有提供S3选项（尽管我不需要它们）--下面的错误消息。当我只使用GCP资源时，我需要指定一个AWS区域，这似乎有点不合时宜（或者可能是我做错了什么）。是否有一种方法可以只注册我在特定管道中使用的文件系统，而不是在初始化时全面注册所有文件系统？

INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message.

它抛出这个（上面）信息警告，好像可以在AWS区域忽略它，但随后抛出一个异常（下面）。

Exception in thread "main" com.amazonaws.SdkClientException: Could not find region information for 'null' in SDK metadata.

Jan 08, 2019 4:14:00 PM org.apache.beam.sdk.io.aws.s3.S3FileSystem <init>
INFO: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you don't plan to use S3, then ignore this message.
Exception in thread "main" com.amazonaws.SdkClientException: Could not find region information for 'null' in SDK metadata.
    at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
    at com.amazonaws.client.builder.AwsClientBuilder.withRegion(AwsClientBuilder.java:243)
    at org.apache.beam.sdk.io.aws.s3.DefaultS3ClientBuilderFactory.createBuilder(DefaultS3ClientBuilderFactory.java:42)
    at org.apache.beam.sdk.io.aws.s3.S3FileSystem.<init>(S3FileSystem.java:112)
    at org.apache.beam.sdk.io.aws.s3.S3FileSystemRegistrar.fromOptions(S3FileSystemRegistrar.java:39)
    at org.apache.beam.sdk.io.FileSystems.verifySchemesAreUnique(FileSystems.java:489)
    at org.apache.beam.sdk.io.FileSystems.setDefaultPipelineOptions(FileSystems.java:479)
    at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:47)
    at org.apache.beam.sdk.Pipeline.create(Pipeline.java:145)
    at foo.GCSPipeline.runGCSPipeline(GCSPipeline.java:192)
    at foo.GCSPipeline.main(GCSPipeline.java:239)

Answer 1

这是bug，在这里跟踪它:https://issues.apache.org/jira/browse/beam-6266