Presto群集无法对配置单元定义的表运行查询-“没有可用于运行查询的节点”

Question

我正在运行一个指向EMR上的hive的2节点presto集群，该集群在S3上配置了数据。

配置单元元数据是可见的；在CLI中，我可以描述（表）claim1，并查看有关它的元数据。

2014-05-19T22:19:32.760+0000     INFO   HiveHdfsWalker-144  stdout  22:19:32.760 [HiveHdfsWalker-144] DEBUG c.a.s.s.m.t.XmlResponsesSaxParser - Examining listing for bucket: unzippeddata

2014-05-19T22:19:32.763+0000     INFO   HiveHdfsWalker-144  stdout  22:19:32.763 [HiveHdfsWalker-144] DEBUG com.amazonaws.request - Received successful response: 200, AWS Request ID: 9B844EEC8586FF3B

2014-05-19T22:19:32.766+0000    DEBUG   query-scheduler-8   com.facebook.presto.execution.SqlStageExecution Stage 20140519_221932_00005_mfhtx.1 is FAILED
2014-05-19T22:19:32.766+0000    DEBUG   query-scheduler-6   com.facebook.presto.execution.SqlStageExecution Stage 20140519_221932_00005_mfhtx.0 is FAILED
2014-05-19T22:19:32.768+0000    DEBUG   query-scheduler-7   com.facebook.presto.execution.QueryStateMachine Query 20140519_221932_00005_mfhtx is FAILED
2014-05-19T22:19:32.770+0000    ERROR   Stage-20140519_221932_00005_mfhtx.1-126 com.facebook.presto.execution.SqlStageExecution Error while starting stage 20140519_221932_00005_mfhtx.1
com.facebook.presto.spi.PrestoException: No nodes available to run query
    at com.facebook.presto.util.Failures.checkCondition(Failures.java:79) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.util.Failures.checkCondition(Failures.java:73) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.NodeScheduler$NodeSelector.computeAssignments(NodeScheduler.java:184) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.scheduleSourcePartitionedNodes(SqlStageExecution.java:631) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.startTasks(SqlStageExecution.java:549) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.access$200(SqlStageExecution.java:91) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution$4.run(SqlStageExecution.java:521) [presto-main-0.68.jar:0.68]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
2014-05-19T22:19:32.775+0000     INFO   query-scheduler-7   com.facebook.presto.event.query.QueryMonitor    TIMELINE: Query 20140519_221932_00005_mfhtx :: elapsed 483.00ms :: planning 74.12ms :: scheduling 409.00ms :: running 0.00ms :: finishing 409.00ms :: begin 2014-05-19T22:19:32.285Z :: end 2014-05-19T22:19:32.768Z
2014-05-19T22:19:32.872+0000    DEBUG   task-notification-0 com.facebook.presto.execution.TaskStateMachine  Task 20140519_221932_00005_mfhtx.0.0 is CANCELED
2014-05-19T22:19:32.880+0000    DEBUG   20140519_221932_00005_mfhtx.0.0-0-56    com.facebook.presto.execution.TaskExecutor  Split 20140519_221932_00005_mfhtx.0.0-0 (start = 1400537972443, wall = 437 ms, cpu = 3 ms, calls = 2) is finished

...或交替地：

2014-05-19T22:22:43.972+0000     INFO   HiveHdfsWalker-170  stdout  22:22:43.972 [HiveHdfsWalker-170] DEBUG c.a.s.s.m.t.XmlResponsesSaxParser - Parsing XML response document with handler: class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler

2014-05-19T22:22:43.972+0000     INFO   HiveHdfsWalker-170  stdout  22:22:43.972 [HiveHdfsWalker-170] DEBUG c.a.s.s.m.t.XmlResponsesSaxParser - Examining listing for bucket: unzippeddata

2014-05-19T22:22:43.976+0000     INFO   HiveHdfsWalker-170  stdout  22:22:43.976 [HiveHdfsWalker-170] DEBUG com.amazonaws.request - Received successful response: 200, AWS Request ID: 476D4ABA552DAA66

2014-05-19T22:22:43.979+0000    DEBUG   query-scheduler-17  com.facebook.presto.execution.SqlStageExecution Stage 20140519_222243_00007_mfhtx.1 is FAILED
2014-05-19T22:22:43.979+0000    ERROR   Stage-20140519_222243_00007_mfhtx.1-160 com.facebook.presto.execution.SqlStageExecution Error while starting stage 20140519_222243_00007_mfhtx.1
com.facebook.presto.spi.PrestoException: No nodes available to run query
    at com.facebook.presto.util.Failures.checkCondition(Failures.java:79) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.util.Failures.checkCondition(Failures.java:73) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.NodeScheduler$NodeSelector.computeAssignments(NodeScheduler.java:184) ~[presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.scheduleSourcePartitionedNodes(SqlStageExecution.java:631) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.startTasks(SqlStageExecution.java:549) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution.access$200(SqlStageExecution.java:91) [presto-main-0.68.jar:0.68]
    at com.facebook.presto.execution.SqlStageExecution$4.run(SqlStageExecution.java:521) [presto-main-0.68.jar:0.68]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
2014-05-19T22:22:43.983+0000    DEBUG   query-scheduler-13  com.facebook.presto.execution.SqlStageExecution Stage 20140519_222243_00007_mfhtx.0 is FAILED
2014-05-19T22:22:43.984+0000    DEBUG   query-scheduler-14  com.facebook.presto.execution.QueryStateMachine Query 20140519_222243_00007_mfhtx is FAILED
2014-05-19T22:22:43.990+0000     INFO   query-scheduler-14  com.facebook.presto.event.query.QueryMonitor    TIMELINE: Query 20140519_222243_00007_mfhtx :: elapsed 233.00ms :: planning 60.01ms :: scheduling 173.00ms :: running 0.00ms :: finishing 173.00ms :: begin 2014-05-19T22:22:43.751Z :: end 2014-05-19T22:22:43.984Z
2014-05-19T22:22:44.088+0000    DEBUG   task-notification-3 com.facebook.presto.execution.TaskStateMachine  Task 20140519_222243_00007_mfhtx.0.0 is CANCELED
2014-05-19T22:22:44.102+0000    DEBUG   20140519_222243_00007_mfhtx.0.0-0-50    com.facebook.presto.execution.TaskExecutor  Split 20140519_222243_00007_mfhtx.0.0-0 (start = 1400538163839, wall = 259 ms, cpu = 0 ms, calls = 2) is finished

非协调器节点有时（但并不一致）在其日志中得到几行：

2014-05-19T22:19:32.340+0000    DEBUG   task-notification-10    com.facebook.presto.execution.TaskStateMachine  Task 20140519_222103_00006_mfhtx.0.0 is CANCELED
2014-05-19T22:19:32.352+0000    DEBUG   20140519_222103_00006_mfhtx.0.0-0-48    com.facebook.presto.execution.TaskExecutor  Split 20140519_222103_00006_mfhtx.0.0-0 (start = 1400538063009, wall = 343 ms, cpu = 0 ms, calls = 2) is finished

Answer 1

这个问题过去是由于忘记在工作者上配置datasources引起的，但这不再是一个问题，因为Presto不再需要为此进行显式配置（而且config属性可能会在将来的版本中被删除）。