Hive报错，NoSuchMethodError: org.apache.hadoop.mapred.JobConf.unset

  • javenzhen

    2015-09-16 19:22:36
  • 数据分析
  • 原创
1、Hbase中test表就是需要在Hive中查看的表
  1. hbase(main):006:0> scan 'test'
  2.     ROW COLUMN+CELL
  3.      row-01 column=cf1:id, timestamp=1442020353563, value=1
  4.      row-01 column=cf1:name, timestamp=1442020382276, value=aaa
  5.      row-02 column=cf1:id, timestamp=1442020360143, value=2
  6.      row-02 column=cf1:name, timestamp=1442020388494, value=bbb
  7.      row-03 column=cf1:id, timestamp=1442020364496, value=3
  8.      row-03 column=cf1:name, timestamp=1442020393616, value=ccc
  9.      row-04 column=cf1:id, timestamp=1442020369002, value=4
  10.      row-04 column=cf1:name, timestamp=1442020398557, value=ddd
  11.      row-05 column=cf1:id, timestamp=1442020373493, value=5
  12.      row-05 column=cf1:name, timestamp=1442020404131, value=eee
  13.     5 row(s) in 0.0680 seconds
2、在Hive中创建external table
  1. grid@master1:~$ hive
  2.   
  3. Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
  4. hive> create external table hbase_test
  5.     > (key string, id string, name string)
  6.     > stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
  7.     > with serdeproperties ("hbase.columns.mapping" =":key,cf1:id,cf1:name")
  8.     > tblproperties("hbase.table.name" = "test");
  9. OK
  10. Time taken: 2.177 seconds

key代表行；

id是Hbase中表test，cf1列族中的id列

name是Hbase中表test，cf1列族中的name列

3、在Hive中使用select查看创建的表

  1. hive> select * from hbase_test;
  2.     Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JobConf.unset(Ljava/lang/String;)V
  3.             at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushFilters(HiveInputFormat.java:438)
  4.             at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:77)
  5.             at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:456)
  6.             at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
  7.             at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
  8.             at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
  9.             at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
  10.             at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
  11.             at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
  12.             at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
  13.             at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
  14.             at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
  15.             at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
  16.             at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
  17.             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  18.             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  19.             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  20.             at java.lang.reflect.Method.invoke(Method.java:606)
  21.             at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

报错NoSuchMethodError: org.apache.hadoop.mapred.JobConf.unset，输入信息的意思应该是Hbase中没有unset类，我现在使用的 Hive版本是1.2.1，可能是Hive版本过高的原因，更换Hive-0.9版本。


4、换了低版本的Hive之后，再次查看表的数据。

  1. hive> show tables;
  2. OK
  3. hbase_test
  4. Time taken: 0.083 seconds
  5. hive> select * from hbase_test;
  6. OK
  7. row-01 1 aaa
  8. row-02 2 bbb
  9. row-03 3 ccc
  10. row-04 4 ddd
  11. row-05 5 eee
  12. Time taken: 0.329 seconds

请使用浏览器的分享功能分享到微信等