错误hive.hiveConfig：无法加载org.apache.hadoop.hive.conf.hiveConf。确保HIVE_CONF_DIR设置正确

Question

我正试图将数据从sqoop导入到配置单元

MySQL

use sample;

create table forhive(   id int auto_increment,
    firstname varchar(36),
    lastname varchar(36),
    primary key(id)
    );    

insert into  forhive(firstname, lastname) values("sample","singh");

select * from forhive;

sqoop import --connect jdbc:mysql://********:3306/sample 

--table forhive --split-by id --columns id,firstname,lastname  

--target-dir /home/programmeur_v/forhive 

--hive-import --create-hive-table --hive-table sqp.forhive --username vaibhav -P

这就是我得到的错误

错误日志

18/08/02 19:19:49信息Sqoop.Sqoop：运行Sqoop版本：1.4.7

18/08/02 19:19:56 INFO Manager.sqlManager：正在执行SQL语句：从forhive中选择t.*作为t限制1

18/08/02 19:19:56 INFO Manager.sqlManager：正在执行SQL语句：从forhive中选择t.*作为t限制1

18/08/02 19:19:56 INFO orm.compilationManager:HADOOP_MAPRED_HOME是/home/programmeur_v/softwares/hadoop-2.9.1

18/08/02 19:19:58警告Manager.mysqlManager：看起来您正在从MySQL导入。

18/08/02 19:19:58警告Manager.mysqlManager：此传输可以更快！使用--Direct

18/08/02 19:19:58警告manager.mysqlmanager：执行特定于MySQL的快速路径的选项。

18/08/02 19:19:58 INFO Manager.mysqlManager：将零日期时间行为设置为convertToNull（mysql）

18/08/02 19:19:58 INFO MapReduce.ImportJobBase：开始导入forhive

18/08/02 19:19:58 INFO Configuration.Deprecation：Mapred.jar被否决。而是使用mapreduce.job.jar

18/08/02 19:19:59信息配置。弃用：已弃用Mapred.Map.Tasks。而是使用mapreduce.job.maps

18/08/02 19:19:59 INFO Client.rmProxy：在/0.0.0.0:8032连接到ResourceManager

18/08/02 19:20:02 INFO db.dbinPutFormat：使用读取提交的事务隔离

18/08/02 19:20:02 INFO db.datadrivendbinputformat：boundingvalsquery：从forhive中选择最小(id)，最大(id)

18/08/02 19:20:02 INFO db.IntegerSplitter：拆分大小：0；Num拆分：4从：1到：3

18/08/02 19:20:02 INFO MapReduce.JobSubmitter：拆分数：3

18/08/02 19:20:02 INFO configuration.deprecation：yarn.resourceManager.system-metrics-publisher.enabled是不推荐的。而是使用yarn.system-metrics-publisher.enabl ed

18/08/02 19:20:02 INFO mapreduce.jobsubmitter：提交作业的令牌:job_1533231535061_0006

18/08/02 19:20:03 INFO impl.yarnclientimpl:提交申请application_1533231535061_0006

18/08/02 19:20:03 INFO MapReduce.job：跟踪作业的url:http://instance-1:8088/proxy/application_1533231535061_0006/

18/08/02 19:20:03 INFO MapReduce.job:正在运行的作业:job_1533231535061_0006

18/08/02 19:20:11 INFO MapReduce.Job:job_1533231535061_0006在uber模式下运行：false

18/08/02 19:20:11 INFO MapReduce.job：map 0%reduce 0%

18/08/02 19:20:21 INFO MapReduce.Job：map 33%reduce 0%

18/08/02 19:20:24 INFO MapReduce.job：map 100%reduce 0%

18/08/02 19:20:25 INFO MapReduce.Job:Job job_1533231535061_0006成功完成

18/08/02 19:20:25 INFO MapReduce.作业：计数器：31

        File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=622830
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=295
        HDFS: Number of bytes written=48
        HDFS: Number of read operations=12
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=6
        Job Counters 
        Killed map tasks=1
        Launched map tasks=3
        Other local map tasks=3
        Total time spent by all maps in occupied slots (ms)=27404
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=27404
        Total vcore-milliseconds taken by all map tasks=27404
        Total megabyte-milliseconds taken by all map tasks=28061696
        Map-Reduce Framework
        Map input records=3
        Map output records=3
        Input split bytes=295
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=671
        CPU time spent (ms)=4210
        Physical memory (bytes) snapshot=616452096
        Virtual memory (bytes) snapshot=5963145216
        Total committed heap usage (bytes)=350224384
        File Input Format Counters 
        Bytes Read=0
        File Output Format Counters 
        Bytes Written=48

18/08/02 19:20:25 INFO MapReduce.ImportJobBase：在25.828秒内传输了48个字节（1.8584字节/秒）

18/08/02 19:20:25 INFO MapReduce.ImportJobBase：检索了3条记录。

18/08/02 19:20:25 INFO MapReduce.importJobBase：发布配置单元/HCAT导入作业数据到表forhive的侦听器

18/08/02 19:20:25 INFO Manager.sqlManager：正在执行SQL语句：从forhive中选择t.*作为t限制1

18/08/02 19:20:25 INFO Hive.hiveimport：将上传的数据加载到Hive中

18/08/02 19:20:25错误hive.hiveConfig：无法加载org.apache.hadoop.hive.conf.hiveConf.确保HIVE_CONF_DIR设置正确。

18/08/02 19:20:25错误tool.importtool：Import失败：java.io.ioexception：java.lang.classnotfoundexception：org.apache.sqoop.hive.hive.conf.hiveconf at org.apache.sqoop.hive.hiveimport.gethiveconf（hiveConfig.java:50)at org.apache.sqoop.hive.hiveimport.gethiveeargs（hiveimport.java:392)at）在org.apache.sqoop.hive.hiveimport.importtable（hiveimport.java:241)、在org.apache.sqoop.tool.importtool.importtable（importtool.java:537)、在org.apache.sqoop.tool.importtool.run（importtool.java:628)、在org.apache.sqoop.sqoop.run(sqoop.java:147)、在org.apache.sqoop.util.toolrunner.run 234）在org.apache.sqoop.sqoop.runtool（sqoop.java：243）在org.apache.sqoop.sqoop.main（sqoop.java：252）引起：java.lang.ClassNotFoundException：org.apache.hadoop.hive.conf.hiveconf在java.net.urlClassLoader.findClass(urlClassLoader.java:381)在java.lang.classLoader.loadClass(classLoader.java:424)在sun.misc.launcher$appClassLoader.loadClass(launcher.java:349)在java.lang.Class.ClassClass(classLoader.java:357)在

在谷歌搜索同样的错误后，我也在bashrc中添加了HIVE_CONF_DIR

导出hive_home=/home/programmeur_v/softwares/apache-hive-1.2.2-bin

导出hive_conf_dir=/home/programmeur_v/softwares/apache-hive-1.2.2-bin/conf

导出路径=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$SQOOP_HOME/bin:$HIVE_CONF_DIR

我所有的Hadoop服务也都启动并运行了。

6976 NameNode

7286 SecondaryNameNode

7559 NodeManager

我只是想不出我在这里犯了什么错误。请引导！

Answer 1

通过谷歌下载文件“hive-common-0.10.0.jar”。将其放在“sqoop/lib”文件夹中。这个解决方案对我有效。