Hive的安装及配置
弘康安
一、Hive安装地址
1、Hive官网地址
http://hive.apache.org/
2、文档查看地址
https://cwiki.apache.org/confluence/display/Hive/GettingStarted
3、下载地址
http://archive.apache.org/dist/hive/
二、Hive安装部署
2.1 Hive安装及配置
1、把 apache-hive-1.2.1-bin.tar.gz 上传到 linux 的 /opt/software 目录下
[test@hadoop151 opt]$ cd software/
[test@hadoop151 software]$ ll
总用量 464860
-rw-rw-r-- 1 test test 92834839 2月 14 09:48 apache-hive-1.2.1-bin.tar.gz
-rw-rw-r--. 1 test test 197657687 1月 25 10:41 hadoop-2.7.2.tar.gz
-rw-rw-r--. 1 test test 185515842 1月 25 09:33 jdk-8u144-linux-x64.tar.gz
2、解压 apache-hive-1.2.1-bin.tar.gz 到 /opt/module/ 目录下面
[test@hadoop151 software]$ tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /opt/module/
3、修改 apache-hive-1.2.1-bin.tar.gz 的名称为 hive
[test@hadoop151 module]$ mv apache-hive-1.2.1-bin/ hive
[test@hadoop151 module]$ ll
总用量 16
drwxr-xr-x. 15 test test 4096 2月 3 21:53 hadoop-2.7.2
drwxrwxr-x 8 test test 4096 2月 14 09:49 hive
drwxrwxr-x 2 test test 4096 2月 4 19:16 input
drwxr-xr-x. 8 test test 4096 7月 22 2017 jdk1.8.0_144
4、修改/opt/module/hive/conf 目录下的 hive-env.sh.template 名称为 hive-env.sh
[test@hadoop151 conf]$ mv hive-env.sh.template hive-env.sh
5、配置 hive-env.sh 文件
(1) 配置 HADOOP_HOME 路径
export HADOOP_HOME=/opt/module/hadoop-2.7.2
(2) 配置 HIVE_CONF_DIR 路径
export HIVE_CONF_DIR=/opt/module/hive/conf
2.2 Hadoop集群配置
1、必须启动 hdfs 和 yarn
[test@hadoop151 conf]$ start-dfs.sh
Starting namenodes on [hadoop151]
hadoop151: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-namenode-hadoop151.out
hadoop151: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop151.out
hadoop152: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop152.out
hadoop153: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-datanode-hadoop153.out
Starting secondary namenodes [hadoop153]
hadoop153: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-test-secondarynamenode-hadoop153.out
[test@hadoop152 ~]$ start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-resourcemanager-hadoop152.out
hadoop153: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop153.out
hadoop151: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop151.out
hadoop152: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-test-nodemanager-hadoop152.out
2、在 HDFS 上创建 /tmp 和 /user/hive/warehouse 两个目录并修改他们的同组权限可写(可不操作，系统会自动创建)
[test@hadoop153 ~]$ hadoop fs -mkdir /tmp
[test@hadoop153 ~]$ hadoop fs -mkdir -p /user/hive/warehouse
[test@hadoop153 ~]$ hadoop fs -chmod g+w /tmp
[test@hadoop153 ~]$ hadoop fs -chmod g+w /user/hive/warehouse
2.3 Hive基本操作
1、启动hive
[test@hadoop151 bin]$ hive
Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive>
2、查看数据库
hive> show databases;
OK
default
Time taken: 1.355 seconds, Fetched: 1 row(s)
3、打开默认的数据库
hive> use default;
OK
Time taken: 0.075 seconds
4、显示 default 数据库中的表
hive> show tables;
OK
Time taken: 0.063 seconds
5、创建一张表
hive> create table student(id int, name string);
OK
Time taken: 0.584 seconds
6、显示数据库中有几张表
hive> show tables;
OK
student
Time taken: 0.022 seconds, Fetched: 1 row(s)
7、查看表的结构
hive> desc student;
OK
id int
name string
Time taken: 0.206 seconds, Fetched: 2 row(s)
8、向表中插入数据
hive> insert into student values(1000,"ss");
Query ID = test_20200214104152_6f97830a-f3f8-4c26-9744-c77441cefd05
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1581645695424_0001, Tracking URL = http://hadoop152:8088/proxy/application_1581645695424_0001/
Kill Command = /opt/module/hadoop-2.7.2/bin/hadoop job -kill job_1581645695424_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2020-02-14 10:42:18,853 Stage-1 map = 0%, reduce = 0%
2020-02-14 10:42:38,866 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.2 sec
MapReduce Total cumulative CPU time: 4 seconds 200 msec
Ended Job = job_1581645695424_0001
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to: hdfs://hadoop151:9000/user/hive/warehouse/student/.hive-staging_hive_2020-02-14_10-41-52_166_5255955701507975136-1/-ext-10000
Loading data to table default.student
Table default.student stats: [numFiles=1, numRows=1, totalSize=8, rawDataSize=7]
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 4.2 sec HDFS Read: 3557 HDFS Write: 79 SUCCESS
Total MapReduce CPU Time Spent: 4 seconds 200 msec
OK
Time taken: 49.347 seconds
9、查询表中数据
hive> select * from student;
OK
1000 ss
Time taken: 0.098 seconds, Fetched: 1 row(s)
10、退出hive
hive> quit;
三、使用MySQL存储Metastore
3.1 为何使用MySQL储存Metastore？
如果我们已经打开了一个 hive 窗口，那么当我们再打开一个客户端启动 hive，会产生 java.sql.SQLException 异常
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
原因是：Metastore 默认存储在自带的 derby 数据库中，推荐使用 MySQL 存储 Metastore。
3.2 MySQL安装
1、查看 mysql 是否安装，如果安装了，卸载 mysql
(1) 查看
[test@hadoop151 ~]$ rpm -qa | grep mysql
mysql-libs-5.1.73-7.el6.x86_64
(2) 卸载
[test@hadoop151 ~]$ sudo rpm -e --nodeps mysql-libs-5.1.73-7.el6.x86_64
2、解压 mysql-libs.zip 文件到当前目录
[test@hadoop151 software]$ unzip mysql-libs.zip
Archive: mysql-libs.zip
creating: mysql-libs/
inflating: mysql-libs/MySQL-client-5.6.24-1.el6.x86_64.rpm
inflating: mysql-libs/mysql-connector-java-5.1.27.tar.gz
inflating: mysql-libs/MySQL-server-5.6.24-1.el6.x86_64.rpm
[test@hadoop151 software]$ ll
总用量 540852
-rw-rw-r-- 1 test test 92834839 2月 14 09:48 apache-hive-1.2.1-bin.tar.gz
-rw-rw-r--. 1 test test 197657687 1月 25 10:41 hadoop-2.7.2.tar.gz
-rw-rw-r--. 1 test test 185515842 1月 25 09:33 jdk-8u144-linux-x64.tar.gz
drwxrwxr-x 2 test test 4096 6月 26 2015 mysql-libs
-rw-rw-r-- 1 test test 77807942 2月 14 12:05 mysql-libs.zip
3、进入 mysql-libs 文件夹下
[test@hadoop151 software]$ cd mysql-libs
[test@hadoop151 mysql-libs]$ ll
总用量 76048
-rw-rw-r-- 1 test test 18509960 3月 26 2015 MySQL-client-5.6.24-1.el6.x86_64.rpm
-rw-rw-r-- 1 test test 3575135 12月 1 2013 mysql-connector-java-5.1.27.tar.gz
-rw-rw-r-- 1 test test 55782196 3月 26 2015 MySQL-server-5.6.24-1.el6.x86_64.rpm
3.3 安装MySQL服务器
1、安装 mysql 服务端
[test@hadoop151 mysql-libs]$ sudo rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
2、查看产生的随机密码
[test@hadoop151 mysql-libs]$ sudo cat /root/.mysql_secret
# The random password set for the root user at Fri Feb 14 12:10:37 2020 (local time): b2LD_KwosbrrTEly
3、查看 mysql 状态
[test@hadoop151 mysql-libs]$ sudo service mysql status
MySQL is not running [失败]
4、启动 mysql
[test@hadoop151 mysql-libs]$ sudo service mysql start
Starting MySQL [确定]
[test@hadoop151 mysql-libs]$ sudo service mysql status
MySQL running (7623) [确定]
3.4 安装MySQL客户端
1、安装 mysql 客户端
[test@hadoop151 mysql-libs]$ sudo rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
2、链接 mysql
[test@hadoop151 mysql-libs]$ mysql -uroot -pb2LD_KwosbrrTEly
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
3、修改密码
mysql> SET PASSWORD=PASSWORD('147258');
Query OK, 0 rows affected (0.04 sec)
4、退出 mysql
mysql> quit;
Bye
3.5 MySQL中user表中主机配置
1、进入 mysql
[test@hadoop151 mysql-libs]$ mysql -uroot -p147258
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.24 MySQL Community Server (GPL)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
2、显示数据库
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
4 rows in set (0.09 sec)
mysql>
3、使用 mysql 数据库
mysql> use mysql
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
4、显示 mysql 数据库中的所有表
mysql> show tables;
+---------------------------+
| Tables_in_mysql |
+---------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| innodb_index_stats |
| innodb_table_stats |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slave_master_info |
| slave_relay_log_info |
| slave_worker_info |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
28 rows in set (0.00 sec)
5、展示 user 表的结构
mysql> desc user
-> ;
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Host | char(60) | NO | PRI | | |
| User | char(16) | NO | PRI | | |
| Password | char(41) | NO | | | |
| Select_priv | enum('N','Y') | NO | | N | |
| Insert_priv | enum('N','Y') | NO | | N | |
| Update_priv | enum('N','Y') | NO | | N | |
| Delete_priv | enum('N','Y') | NO | | N | |
| Create_priv | enum('N','Y') | NO | | N | |
| Drop_priv | enum('N','Y') | NO | | N | |
| Reload_priv | enum('N','Y') | NO | | N | |
| Shutdown_priv | enum('N','Y') | NO | | N | |
| Process_priv | enum('N','Y') | NO | | N | |
| File_priv | enum('N','Y') | NO | | N | |
| Grant_priv | enum('N','Y') | NO | | N | |
| References_priv | enum('N','Y') | NO | | N | |
| Index_priv | enum('N','Y') | NO | | N | |
| Alter_priv | enum('N','Y') | NO | | N | |
| Show_db_priv | enum('N','Y') | NO | | N | |
| Super_priv | enum('N','Y') | NO | | N | |
| Create_tmp_table_priv | enum('N','Y') | NO | | N | |
| Lock_tables_priv | enum('N','Y') | NO | | N | |
| Execute_priv | enum('N','Y') | NO | | N | |
| Repl_slave_priv | enum('N','Y') | NO | | N | |
| Repl_client_priv | enum('N','Y') | NO | | N | |
| Create_view_priv | enum('N','Y') | NO | | N | |
| Show_view_priv | enum('N','Y') | NO | | N | |
| Create_routine_priv | enum('N','Y') | NO | | N | |
| Alter_routine_priv | enum('N','Y') | NO | | N | |
| Create_user_priv | enum('N','Y') | NO | | N | |
| Event_priv | enum('N','Y') | NO | | N | |
| Trigger_priv | enum('N','Y') | NO | | N | |
| Create_tablespace_priv | enum('N','Y') | NO | | N | |
| ssl_type | enum('','ANY','X509','SPECIFIED') | NO | | | |
| ssl_cipher | blob | NO | | NULL | |
| x509_issuer | blob | NO | | NULL | |
| x509_subject | blob | NO | | NULL | |
| max_questions | int(11) unsigned | NO | | 0 | |
| max_updates | int(11) unsigned | NO | | 0 | |
| max_connections | int(11) unsigned | NO | | 0 | |
| max_user_connections | int(11) unsigned | NO | | 0 | |
| plugin | char(64) | YES | | mysql_native_password | |
| authentication_string | text | YES | | NULL | |
| password_expired | enum('N','Y') | NO | | N | |
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
43 rows in set (0.00 sec)
6、查询 user 表
mysql> select User, Host, Password from user;
+------+-----------+-------------------------------------------+
| User | Host | Password |
+------+-----------+-------------------------------------------+
| root | localhost | *288A7937D7A48B2C8148F1849013DBD8AA443263 |
| root | hadoop151 | *4D194794583A003975BF7EAE853BCF0FE7DA93FF |
| root | 127.0.0.1 | *4D194794583A003975BF7EAE853BCF0FE7DA93FF |
| root | ::1 | *4D194794583A003975BF7EAE853BCF0FE7DA93FF |
+------+-----------+-------------------------------------------+
4 rows in set (0.00 sec)
7、修改 user 表，把 Host 表内容修改为 %
mysql> update user set host='%' where host='localhost';
Query OK, 1 row affected (0.03 sec)
Rows matched: 1 Changed: 1 Warnings: 0
8、删除 root 用户的其他 host
mysql> delete from user where Host='hadoop151'; delete from user where Host='127.0.0.1'; delete from user where Host='::1';
Query OK, 0 rows affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
Query OK, 1 row affected (0.00 sec)
9、刷新
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
10、退出
mysql>quit;
3.6 拷贝驱动
1、在 /opt/software/mysql-libs 目录下解压 mysql-connector-java-5.1.27.tar.gz 驱动包
[test@hadoop151 mysql-libs]$ tar -zxvf mysql-connector-java-5.1.27.tar.gz -C ./
2、拷贝 mysql-connector-java-5.1.27-bin.jar 到 /opt/module/hive/lib/
[test@hadoop151 mysql-connector-java-5.1.27]$ cp mysql-connector-java-5.1.27-bin.jar /opt/module/hive/lib/
3.7 配置元数据到MySQL
1、在/opt/module/hive/conf 目录下创建一个 hive-site.xml
[test@hadoop151 conf]$ touch hive-site.xml
[test@hadoop151 conf]$ vim hive-site.xml
2、根据官方文档配置参数，拷贝数据到 hive-site.xml 文件中
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+Administration
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop151:3306/metastore?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>147258</value>
<description>password to use against metastore database</description>
</property>
</configuration>
3、配置完毕后，如果启动 hive 异常，可以重新启动虚拟机
3.8 多窗口启动Hive测试
1、再次打开多个窗口，分别启动 hive
2、启动 hive 后，再查看一下数据库，增加了 metastore 数据库
[test@hadoop151 ~]$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.24 MySQL Community Server (GPL)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| metastore |
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.38 sec)
四、HiveJDBC访问
1、启动 hiveserver2 服务
[test@hadoop151 ~]$ hiveserver2
2、启动 beeline
[test@hadoop151 ~]$ beeline
Beeline version 1.2.1 by Apache Hive
beeline>
3、连接 hiveserver2
beeline> !connect jdbc:hive2://hadoop151:10000
Connecting to jdbc:hive2://hadoop151:10000
Enter username for jdbc:hive2://hadoop151:10000: test
Enter password for jdbc:hive2://hadoop151:10000:
Connected to: Apache Hive (version 1.2.1)
Driver: Hive JDBC (version 1.2.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop151:10000> show databases;
+----------------+--+
| database_name |
+----------------+--+
| default |
+----------------+--+
1 row selected (2.266 seconds)
五、Hive常用命令
1、查看命令说明
[test@hadoop151 ~]$ hive -help
usage: hive
-d,--define <key=value> Variable subsitution to apply to hive
commands. e.g. -d A=B or --define A=B
--database <databasename> Specify the database to use
-e <quoted-query-string> SQL from command line
-f <filename> SQL from files
-H,--help Print help information
--hiveconf <property=value> Use value for given property
--hivevar <key=value> Variable subsitution to apply to hive
commands. e.g. --hivevar A=B
-i <filename> Initialization SQL file
-S,--silent Silent mode in interactive shell
-v,--verbose Verbose mode (echo executed SQL to the
console)
2、"-e" 不进入 hive 的交互窗口执行 sql 语句
[test@hadoop151 ~]$ hive -e "select id from student;"
Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
OK
1000
1000
Time taken: 1.561 seconds, Fetched: 2 row(s)
3、"-f" 执行脚本中 sql 语句
(1) 在 /opt/module/datas 目录下创建 hivef.sql 文件，并且编辑文件
[test@hadoop151 datas]$ touch hivef.sql
[test@hadoop151 datas]$ vim hivef.sql
[test@hadoop151 datas]$ cat hivef.sql
select * from student;
(2) 执行文件中的 sql 语句
[test@hadoop151 datas]$ hive -f /opt/module/datas/hivef.sql
Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
OK
1000 ss
1000 ss
Time taken: 1.53 seconds, Fetched: 2 row(s)
(3) 执行文件中的 sql 语句并将结果写入文件中
[test@hadoop151 datas]$ touch hive_result.txt
[test@hadoop151 datas]$ hive -f /opt/module/datas/hivef.sql > /opt/module/datas/hive_result.txt
Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
OK
Time taken: 1.661 seconds, Fetched: 2 row(s)
[test@hadoop151 datas]$ cat hive_result.txt
1000 ss
1000 ss
4、其他命令操作
(1) 在 hive 窗口中查看 hdfs 文件系统
hive> dfs -ls /;
Found 20 items
drwxr-xr-x - test supergroup 0 2020-02-02 14:37 /0202
drwxr-xr-x - test supergroup 0 2020-02-01 13:51 /A
-rw-r--r-- 3 test supergroup 27 2020-02-04 20:02 /A.txt
-rw-r--r-- 3 test supergroup 1366 2020-02-03 21:55 /README.txt
-rw-r--r-- 3 test supergroup 18 2020-02-02 17:32 /abc.txt
-rw-r--r-- 3 test supergroup 16 2020-02-04 20:02 /b.txt
-rw-r--r-- 2 test supergroup 18 2020-02-02 15:14 /bao.txt
-rw-r--r-- 2 test supergroup 18 2020-02-02 17:10 /baozhu.txt
-rw-r--r-- 3 test supergroup 12 2020-02-04 20:02 /c.txt
-rw-r--r-- 3 test supergroup 197657687 2020-01-30 20:50 /hadoop-2.7.2.tar.gz
drwxr-xr-x - test supergroup 0 2020-02-04 19:17 /input
drwxr-xr-x - test supergroup 0 2020-02-04 22:09 /input1
-rw-r--r-- 3 test supergroup 185515842 2020-02-01 12:58 /jdk-8u144-linux-x64.tar.gz
drwxr-xr-x - test supergroup 0 2020-02-04 22:07 /kkk1
drwxr-xr-x - test supergroup 0 2020-02-04 14:33 /module
drwxr-xr-x - test supergroup 0 2020-02-04 19:45 /output1
drwxr-xr-x - test supergroup 0 2020-02-01 13:06 /shuihu
drwxr-xr-x - test supergroup 0 2020-02-04 14:22 /system
drwxrwxr-x - test supergroup 0 2020-02-14 10:34 /tmp
drwxr-xr-x - test supergroup 0 2020-02-14 10:07 /user
(2) 在 hive 窗口中查看本地文件系统
hive> ! ls /opt/module/datas;
hivef.sql
hive_result.txt
(3) 查看在 hive 中输入的所有历史命令
A、进入当前用户的根目录
[test@hadoop151 datas]$ cd ~
B、查看 .hivehistory 文件
[test@hadoop151 ~]$ cat .hivehistory
show databases;
use default;
show tables;
create table student(id int, name string);
show tables;
desc student;
insert into student values(1000,"ss");
select * from student;
quit;
rpm -qa|grep mysql
;
rpm -qa|grep mysql
quit;
quit
;
六、Hive常见属性配置
6.1 Hive数据仓库位置配置
1、Default 数据仓库的最原始位置是在 hdfs 上的：/user/hive/warehouse 路径下。
2、在仓库目录下，没有对默认的数据库 default 创建文件夹。如果某张表属于 default 数据库，直接在数据仓库目录下创建一个文件夹。
3、修改 default 数据仓库原始位置(将 hive-default.xml.template 如下配置信息拷贝到 hive-site.xml 文件中)。
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
配置同组用户有执行权限
hdfs dfs -chmod g+w /user/hive/warehouse
6.2 查询后信息显示配置
1、在 hive-site.xml 文件中添加如下配置信息，就可以实现显示当前数据库，以及查询表的头信息配置。
<property>
<name>hive.cli.print.header</name>
<value>true</value>
</property>
<property>
<name>hive.cli.print.current.db</name>
<value>true</value>
</property>
6.3 Hive 运行日志信息配置
1、Hive 的 log 默认存放在 /tmp/test/hive.log 目录下(当前用户名下)
2、修改 hive 的 log 存放日志到 /opt/module/hive/logs
(1) 修改/opt/module/hive/conf/hive-log4j.properties.template 文件名称为 hive-log4j.properties
(2) 在 hive-log4j.properties 文件中修改 log 存放位置
hive.log.dir=/opt/module/hive/logs
6.4 参数配置方式
1、查看当前所有的配置信息
hive>set;
2、参数的三种配置方式
(1) 配置文件方式
默认配置文件：hive-default.xml
用户自定义配置文件：hive-site.xml
注意：用户自定义配置会覆盖默认配置，另外，Hive 也会读入 Hadoop 的配置，因为 Hive 是作为 Hadoop 的客户端启动的，Hive 的配置会覆盖 Hadoop 的配置。配置文件的设定对本机启动的所有 Hive 进程都有效。
(2) 命令行参数方式
启动 Hive 时，可以在命令行添加 -hiveconf param=value 来设定参数。
例如：
[test@hadoop151 hive]$ bin/hive -hiveconf mapred.reduce.tasks=10;
注意：仅仅对本次 hive 启动有效
查看参数设置：
hive (default)> set mapred.reduce.tasks;
(3) 参数声明方式
可以在 HQL 中使用 set 关键字设定参数
例如：
hive (default)> set mapred.reduce.tasks=100;
注意：仅仅对本次 hive 启动有效。
(4) 三种方式的顺序
上述三种设定方式的优先级依次递增。即配置文件<命令行参数<参数声明。注意某些系统级的参数，例如 log4j 相关的设定，必须用前两种方式设定，因为那些参数的读取在会话建立以前已经完成了。
七、说明
在本次搭建 hive 环境中，必须先搭建好 hadoop 环境，hadoop 版本为 2.7.2，hadoop 环境的搭建可以参考我的另一篇博客：https://blog.csdn.net/a1786742005/article/details/104104983。
本次hive搭建的所有软件包存放在百度网盘链接：
链接：https://pan.baidu.com/s/1QM8Dd7t2FJtYTEfcjgEmsg 提取码：m4i9
类似资料：