无法解析给定输入列的“column_name”：SparkSQL

Question

我这里有一段简单的代码：

query = """
    select id, date, type from schema.camps
"""
df = spark.sql(query)

我收到一个错误，上面写着：

>     > "cannot resolve '`id`' given input columns:
>     > [ecs_snapshot, ecs_version, ecs_bundle_type]; line 2   

文件

根据提供的解决方案，我尽了最大努力。有趣的是，我在另一个表上有另一个查询，效果很好。非常感谢您的帮助。提前谢谢。

这是表的架构：

camps(
    
    id numeric(38,0) NOT NULL encode raw,
    name varchar(765) NULL encode zstd,
    type varchar(765) NULL encode zstd,
    YYYY varchar(765) NULL encode zstd,
    ZZZZ varchar(765) NULL encode zstd,
    LLLL varchar(765) NULL encode zstd,
    MMMM numeric(38,0) NULL encode zstd,
    NNNN  varchar(765) NULL encode zstd,
    date timestamp without time zone NULL encode zstd,
    PPPP numeric(38,0) NULL encode az64,
    PRIMARY KEY (marketplace_id, campaign_id)
)

；

Answer 1

请尝试运行代码并显示结果。

import spark.implicits._

val df1 = spark.table("ads.dim_campaigns")
df1.printSchema()
// Please, show result

val df2 = df1.select(
  'campaign_id,
  'external_id,
  'start_date,
  'program_type,
  'advertiser_id
)
df2.printSchema()
// please, show result

Answer 2

营地。表中不存在campaign\u id列。dim\u campaigns

此查询有效

>>> l = [[1],[2],[3]]
>>> df = spark.createDataFrame(l,['col_1'])
>>> df.createOrReplaceTempView('table')
>>> query = """SELECT table_alias.col_1 FROM table table_alias"""
>>> spark.sql(query).show()
+-----+
|col_1|
+-----+
|    1|
|    2|
|    3|
+-----+

此查询给出的错误与您的错误相同（请参见第x列，而不是第1列）

>>> l = [[1],[2],[3]]
>>> df = spark.createDataFrame(l,['col_1'])
>>> df.createOrReplaceTempView('table')
>>> query = """SELECT table_alias.col_x FROM table table_alias"""
>>> spark.sql(query).show()

/.../
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/apache-spark/2.4.5/libexec/python/pyspark/sql/session.py", line 767, in sql
    return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)
  File "/usr/local/Cellar/apache-spark/2.4.5/libexec/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
  File "/usr/local/Cellar/apache-spark/2.4.5/libexec/python/pyspark/sql/utils.py", line 69, in deco
    raise AnalysisException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: "cannot resolve '`table_alias.col_x`' given input columns: [table_alias.col_1];