使用Hive-JDBC遇到的一些问题解决
2017-12-12 11:23
369 查看
1.java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface
使用JDBC驱动连接到hive的时候,出现了NoClassDefFoundError的错误,提示找不到Iface接口这个类。
Hive版本1.1
2.上述的问题解决之后,又出现了一个新的问题:java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
异常详情:
3. 在使用DatabaseMetaData获取元数据信息时,抛出异常:
"Method
not supported"
这是因为HiveDatabaseMetaData在实现DatabaseMetaData接口时,没有做相应方法的实现,这里我使用到的是一个getUserName接口:
使用JDBC驱动连接到hive的时候,出现了NoClassDefFoundError的错误,提示找不到Iface接口这个类。
Hive版本1.1
java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161) at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43) at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34) at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 44 more根据错误定位到..HiveDriver类105行的位置
public Connection connect(String url, Properties info) throws SQLException { return acceptsURL(url) ? new HiveConnection(url, info) : null; }发现在HiveConnection中有调用TCLIService.Iface接口
... TCLIService.Iface client = new TCLIService.Client(new TBinaryProtocol(transport)); TOpenSessionResp openResp = client.OpenSession(new TOpenSessionReq()); if (openResp != null) { client.CloseSession(new TCloseSessionReq(openResp.getSessionHandle())); } ...在这里抛了找不到Iface的异常,但是点进去TCLIService类中,确是存在Iface接口,所以说找不到不是因为不存在,而是出现重复的类。
@SuppressWarnings({"cast", "rawtypes", "serial", "unchecked"}) @Generated(value = "Autogenerated by Thrift Compiler (0.9.3)") public class TCLIService { public interface Iface { public TOpenSessionResp OpenSession(TOpenSessionReq req) throws org.apache.thrift.TException; public TCloseSessionResp CloseSession(TCloseSessionReq req) throws org.apache.thrift.TException; .....所以去看下引入的依赖包,发现有hive-service和hive-service-rpc两个包中有相同的源码,所以在这里会抛异常,我就先把这其中一个依赖从hive-jdbc中去除掉,但是发现还是抛异常,我这里用的hive-jdbc1.1版本,发现hive-service是2.1版本,所以立马把两个都去除掉,重新引入一个hive-service进来就好拉
<dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-jdbc</artifactId> <exclusions> <exclusion> <groupId>org.apache.hive</groupId> <artifactId>hive-service-rpc</artifactId> </exclusion> <exclusion> <groupId>org.apache.hive</groupId> <artifactId>hive-service</artifactId> </exclusion> </exclusions> <version>1.1.0</version> </dependency> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-service</artifactId> <version>1.1.0</version> </dependency>
2.上述的问题解决之后,又出现了一个新的问题:java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
异常详情:
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:393) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191) at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161) at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68) at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43) at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34) at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 47 more这是因为要缺少一个hadoop-common包,引入即可:
<dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.6.5</version> </dependency>
3. 在使用DatabaseMetaData获取元数据信息时,抛出异常:
"Method
not supported"
这是因为HiveDatabaseMetaData在实现DatabaseMetaData接口时,没有做相应方法的实现,这里我使用到的是一个getUserName接口:
... public String getUserName() throws SQLException { throw new SQLException("Method not supported"); } ...可以看到在HiveDatabaseMetaData中没有对getUserName支持,所以会抛异常,getUserName方法是获取到当前HiveConnection连接的用户名(Schema名称),但是这里不支持,那就得另外想办法,这里可以调用HiveConnection中的一个getConnParams方法得到连接参数,再调用getSchema方法获取到Schema名称。代码:
... HiveDatabaseMetaData hiveMetaData = (HiveDatabaseMetaData) metadata; HiveConnection hiveConn = (HiveConnection) hiveMetaData.getConnection(); String hiveSchema = hiveConn.getSchema(); ...
相关文章推荐
- 在使用putty工具进行远程登录时也许会遇到一些问题,下面列出了一些问题有利于帮助大家解决:
- 11.Cocos2dx2.2下使用JNI技术调用jar包里面的一些方法遇到的一些问题及解决方式。
- XILINX ISE 13.1 使用中遇到的一些问题和解决方法
- tomcat,jboss使用中遇到的一些问题的解决
- 生产环境使用elasticsearch遇到的一些问题以及解决方法
- GitHub使用elasticsearch遇到的一些问题及解决方法
- 使用elasticsearch遇到的一些问题以及解决方法
- [开发总结]WebView使用中遇到的一些问题&解决
- Oracle 11g 开始使用时遇到的一些问题及解决。
- spark 使用中会遇到的一些问题及解决思路
- 生产环境使用elasticsearch遇到的一些问题以及解决方法(不断更新)
- 转 [开发总结]WebView使用中遇到的一些问题&解决
- tomcat,jboss使用中遇到的一些问题的解决
- 生产环境使用elasticsearch遇到的一些问题以及解决方法(不断更新)
- 使用Linq 更新数据库时遇到的一些问题及解决办法
- windows使用libvlc进行网络串流遇到的一些问题及解决方法
- 生产环境使用elasticsearch遇到的一些问题以及解决方法(不断更新)
- 生产环境使用elasticsearch遇到的一些问题以及解决方法(不断更新)
- jquery mobile使用过程中遇到的一些问题及解决方法
- WebView使用中遇到的一些问题&解决