提交云数据流作业时出错

Question

Exception in thread "main" java.lang.RuntimeException: Failed to create a workflow job: Invalid JSON payload received. Unknown token.
       { 8r W
^
    at com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.run(DataflowPipelineRunner.java:219)
    at com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner.run(BlockingDataflowPipelineRunner.java:96)
    at com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner.run(BlockingDataflowPipelineRunner.java:47)
    at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:145)
    at snippet.WordCount.main(WordCount.java:165)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
  "code" : 400,
  "errors" : [ {
    "domain" : "global",
    "message" : "Invalid JSON payload received. Unknown token.\n\u001F \b\u0000\u0000\u0000\u0000\u0000\u0000\u0000  \t{ 8r\u0000 W\n^",
    "reason" : "badRequest"
  } ],
  "message" : "Invalid JSON payload received. Unknown token.\n\u001F \b\u0000\u0000\u0000\u0000\u0000\u0000\u0000  \t{ 8r\u0000 W\n^",
  "status" : "INVALID_ARGUMENT"
}

Answer 1

为了调试这个问题，我们希望验证所发出的请求是否有效，并找到JSON有效负载的无效部分。为此，我们将：

1. 增加日志记录的冗长
2. 重新运行应用程序并捕获日志
3. 在表示JSON有效负载的日志中找到相关部分
4. 验证JSON有效负载

增加日志记录的冗长性

import java.util.logging.ConsoleHandler;
import java.util.logging.Level;
import java.util.logging.Logger;
public class MyDataflowProgram {
  public static void main(String[] args) {
    ConsoleHandler consoleHandler = new ConsoleHandler();
    consoleHandler.setLevel(Level.ALL);
    Logger googleApiLogger = Logger.getLogger("com.google.api");
    googleApiLogger.setLevel(Level.ALL);
    googleApiLogger.setUseParentHandlers(false);
    googleApiLogger.addHandler(consoleHandler);
    ... Pipeline Construction ...
}

在重新执行数据流作业期间，您需要查找与提交数据流作业相关的日志。这些日志将包含后跟响应的HTTP请求，如下所示：

POST https://dataflow.googleapis.com/v1b3/projects/$GCP_PROJECT_NAME/jobs
Accept-Encoding: gzip
... Additional HTTP headers ...
... JSON request payload for creation ...
{"environment":{"clusterManagerApiService":"compute.googleapis.com","dataset":"bigquery.googleapis.com/cloud_dataflow","sdkPipelineOptions": ...

-------------- RESPONSE --------------
HTTP/1.1 200 OK
... Additional HTTP headers ...
... JSON response payload ...

您对请求有效负载感兴趣，因为您得到的错误表明它是问题的根源。

验证JSON有效负载