登录
当前位置： 首页 > 工具软件 > datart > 使用案例 >

手把手教你datart集成hive3.1.2

卓正业
2023-12-01

datart集成hive3.1.2

datart中是没有hive驱动包的，需要我们自己根据hive版本，放入驱动包

1.放入hive的驱动jar包

cp hive-jdbc-3.1.2.jar /opt/datart/lib/

测试hive数据源连接，报如下 错误信息

2022-09-28 13:34:33.756 ERROR datart.server.job.SchemaSyncJob : source schema sync error 
com.alibaba.druid.pool.GetConnectionTimeoutException: wait millis 5000, active 0, maxActive 8, creating 0, createErrorCount 2
        at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1736)
        at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1408)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1388)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1378)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:99)
        at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.getConn(JdbcDataProviderAdapter.java:277)
        at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.readAllDatabases(JdbcDataProviderAdapter.java:114)
        at datart.data.provider.JdbcDataProvider.readAllDatabases(JdbcDataProvider.java:73)
        at datart.data.provider.ProviderManager.readAllDatabases(ProviderManager.java:87)
        at datart.server.service.impl.DataProviderServiceImpl.readAllDatabases(DataProviderServiceImpl.java:127)
        at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:81)
        at datart.server.job.SchemaSyncJob.execute(SchemaSyncJob.java:72)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: java.lang.NoClassDefFoundError: org/apache/thrift/transport/TTransport
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
        at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1652)
        at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1718)
        at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2785)
Caused by: java.lang.ClassNotFoundException: org.apache.thrift.transport.TTransport
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 4 common frames omitted

2.放hive相关jar包

光有驱动包是不够的，继续放入相关的jar包

 hive-common-3.1.2.jar
 hive-exec-3.1.2.jar
 hive-jdbc-3.1.2.jar
 hive-metastore-3.1.2.jar
 hive-serde-3.1.2.jar
 hive-service-3.1.2.jar
 hive-shims-0.23-3.1.2.jar
 hive-shims-3.1.2.jar
 hive-shims-common-3.1.2.jar
 hive-shims-scheduler-3.1.2.jar

将以上jar包放入datart目录后，包如下错误：

2022-09-28 13:47:08.907 WARN  com.alibaba.druid.pool.DruidDataSource : get connection timeout retry : 1
2022-09-28 13:47:13.909 ERROR datart.server.config.WebExceptionHandler : wait millis 5000, active 0, maxActive 8, creating 0, createErrorCount 2
com.alibaba.druid.pool.GetConnectionTimeoutException: wait millis 5000, active 0, maxActive 8, creating 0, createErrorCount 2
        at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1736)
        at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1408)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1388)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1378)
        at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:99)
        at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.getConn(JdbcDataProviderAdapter.java:277)
        at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.execute(JdbcDataProviderAdapter.java:223)
        at datart.data.provider.jdbc.adapters.JdbcDataProviderAdapter.executeOnSource(JdbcDataProviderAdapter.java:459)
        at datart.data.provider.JdbcDataProvider.execute(JdbcDataProvider.java:95)
        at datart.data.provider.ProviderManager.run(ProviderManager.java:238)
        at datart.data.provider.ProviderManager.execute(ProviderManager.java:123)
        at datart.server.service.impl.DataProviderServiceImpl.testExecute(DataProviderServiceImpl.java:209)
        at datart.server.controller.DataProviderController.testExecute(DataProviderController.java:95)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:197)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:141)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:894)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1060)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:962)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:652)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
        at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:887)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1684)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
        at org.apache.hive.jdbc.Utils.extractURLComponents(Utils.java:472)
        at org.apache.hive.jdbc.Utils.parseURL(Utils.java:332)
        at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
        at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1652)
        at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1718)
        at com.alibaba.druid.pool.DruidDataSource$CreateConnectionThread.run(DruidDataSource.java:2785)
Caused by: java.lang.ClassNotFoundException: org.apache.curator.RetryPolicy
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 7 common frames omitted

3.放入curator相关jar包

查看hive中curator的包

ll | grep curator
#有如下相关的jar
curator-client-2.12.0.jar
curator-framework-2.12.0.jar
curator-recipes-2.12.0.jar

放入datart的lib目录下

cp curator* /opt/datart/lib/

再测试连接，大功告成!

 类似资料：

相关阅读

Vue手把手教你撸一个 beforeEnter 钩子函数手把手教你pycharm专业版安装破解教程(linux版)C语言手把手教你实现贪吃蛇AI（中）C语言手把手教你实现贪吃蛇AI（上）手把手教你Python yLab的绘制折线图的画法

相关文章

快手快star-offer到手教你如何成功拿下大厂Offer！所有面试【前端手撕合集】快手前端|24秋招|面试合集8.10 快手-数据采集-二面凉经

相关问答

我在安装快速把手时出错了WireMock CustomTransformer（车把助手不在bodyFile中工作）如何手动生成google-services.jsonjavaJava有手动垃圾收集吗？你能把一条小溪分成两条小溪吗？

相关文档

PHP 异步编程：手把手教你实现 co 与 KoaBiopython 教程与手册 v1.61+教你成为全栈工程师Bash 手册FreeBSD Porter 手册
快捷导航： 新手教程 算法原理 架构设计 Java进阶 数据库进阶 大厂专栏 面试经验 编程笔记 编程问答 所有专题 文档资料 工具软件 电子书籍 小牛导航
在线工具： 房贷计算器 个税计算器 Linux命令查询 Json格式化 正则表达式 颜色转换 AES加解密 SHA1加密 MD5加密 毒鸡汤 字数统计 随机密码生成 进制转换 Base64编解码 励志句子
Copyright © 2019-2024 小牛知识库@xnip.cn. All Rights Reserved.