Hadoop版本:2.8.4
HBase版本:2.1.0
Hive版本:2.3.3
Hadoop HA 与 HBase HA配置完成,且可正常使用。
Hive配置完成后,使用Hive Shell 可以查询、创建数据库,也可以创建表,但是在表中插入数据失败,详细如下所示:
Hive日志如下所示:
以下是 Hive日志的文件信息
2018-09-05T15:29:52,851 INFO [main] conf.HiveConf: Using the default value passed in for log id: 8edc6120-e6f3-477f-bf26-52e0e2fa36db
2018-09-05T15:29:52,851 INFO [main] session.SessionState: Updating thread name to 8edc6120-e6f3-477f-bf26-52e0e2fa36db main
2018-09-05T15:29:52,852 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Compiling command(queryId=hadoop_20180905152952_4e1d12b8-06cb-4a91-9fdf-7ca3fafb0051): insert into phil.userinfo values ("abc1", "21")
2018-09-05T15:29:52,854 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Starting Semantic Analysis
2018-09-05T15:29:52,907 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis
2018-09-05T15:29:52,908 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for source tables
2018-09-05T15:29:52,908 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for subqueries
2018-09-05T15:29:52,908 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for destination tables
2018-09-05T15:29:52,919 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis
2018-09-05T15:29:52,960 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for source tables
2018-09-05T15:29:52,960 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for subqueries
2018-09-05T15:29:52,961 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Get metadata for destination tables
2018-09-05T15:29:52,976 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] common.FileUtils: Creating directory if it doesn't exist: hdfs://namenodeHaclusterDev/user/hive/warehouse/phil.db/userinfo/.hive-staging_hive_2018-09-05_15-29-52_852_2117193599271128337-1
2018-09-05T15:29:52,996 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan.
2018-09-05T15:29:52,996 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ppd.OpProcFactory: Processing for FS(2)
2018-09-05T15:29:52,997 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ppd.OpProcFactory: Processing for SEL(1)
2018-09-05T15:29:52,997 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ppd.OpProcFactory: Processing for TS(0)
2018-09-05T15:29:53,029 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] optimizer.GenMRFileSink1: using CombineHiveInputformat for the merge job
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] parse.CalcitePlanner: Completed plan generation
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Semantic Analysis Completed
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:values__tmp__table__2.tmp_values_col1, type:string, comment:null), FieldSchema(name:values__tmp__table__2.tmp_values_col2, type:string, comment:null)], properties:null)
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Completed compiling command(queryId=hadoop_20180905152952_4e1d12b8-06cb-4a91-9fdf-7ca3fafb0051); Time taken: 0.239 seconds
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2018-09-05T15:29:53,091 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Executing command(queryId=hadoop_20180905152952_4e1d12b8-06cb-4a91-9fdf-7ca3fafb0051): insert into phil.userinfo values ("abc1", "21")
2018-09-05T15:29:53,092 WARN [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
2018-09-05T15:29:53,092 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
2018-09-05T15:29:53,092 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Query ID = hadoop_20180905152952_4e1d12b8-06cb-4a91-9fdf-7ca3fafb0051
2018-09-05T15:29:53,092 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Total jobs = 3
2018-09-05T15:29:53,093 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Launching Job 1 out of 3
2018-09-05T15:29:53,094 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode
2018-09-05T15:29:53,095 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Task: Number of reduce tasks is set to 0 since there's no reduce operator
2018-09-05T15:29:53,101 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Context: New scratch dir is hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/hive_2018-09-05_15-29-52_852_2117193599271128337-1
2018-09-05T15:29:53,103 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] mr.ExecDriver: Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
2018-09-05T15:29:53,103 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: Processing alias values__tmp__table__2
2018-09-05T15:29:53,103 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: Adding 1 inputs; the first input is hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/_tmp_space.db/Values__Tmp__Table__2
2018-09-05T15:29:53,103 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: Content Summary not cached for hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/_tmp_space.db/Values__Tmp__Table__2
2018-09-05T15:29:53,104 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Context: New scratch dir is hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/hive_2018-09-05_15-29-52_852_2117193599271128337-1
2018-09-05T15:29:53,111 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.SerializationUtilities: Serializing MapWork using kryo
2018-09-05T15:29:53,135 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: Serialized plan (via FILE) - name: null size: 3.88KB
2018-09-05T15:29:53,173 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] fs.FSStatsPublisher: created : hdfs://namenodeHaclusterDev/user/hive/warehouse/phil.db/userinfo/.hive-staging_hive_2018-09-05_15-29-52_852_2117193599271128337-1/-ext-10001
2018-09-05T15:29:53,215 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: PLAN PATH = hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/hive_2018-09-05_15-29-52_852_2117193599271128337-1/-mr-10004/91ef904a-f4de-4a49-a656-0dcaafcdbb56/map.xml
2018-09-05T15:29:53,383 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.Utilities: PLAN PATH = hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/hive_2018-09-05_15-29-52_852_2117193599271128337-1/-mr-10004/91ef904a-f4de-4a49-a656-0dcaafcdbb56/map.xml
2018-09-05T15:29:53,383 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] io.CombineHiveInputFormat: Total number of paths: 1, launching 1 threads to check non-combinable ones.
2018-09-05T15:29:53,384 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/_tmp_space.db/Values__Tmp__Table__2; using filter path hdfs://namenodeHaclusterDev/tmp/hive/hadoop/8edc6120-e6f3-477f-bf26-52e0e2fa36db/_tmp_space.db/Values__Tmp__Table__2
2018-09-05T15:29:53,388 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] io.CombineHiveInputFormat: number of splits 1
2018-09-05T15:29:53,389 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] io.CombineHiveInputFormat: Number of all splits 1
2018-09-05T15:29:53,701 ERROR [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] exec.TaskRunner: Error in executeTask
java.lang.NoSuchFieldError: DEFAULT
at org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:322) ~[hadoop-mapreduce-client-common-2.8.4.jar:?]
at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:440) ~[hadoop-mapreduce-client-jobclient-2.8.4.jar:?]
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:314) ~[hadoop-mapreduce-client-jobclient-2.8.4.jar:?]
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:240) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_65]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_65]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840) ~[hadoop-common-2.8.4.jar:?]
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_65]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_65]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840) ~[hadoop-common-2.8.4.jar:?]
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561) ~[phoenix-4.14.0-HBase-1.4-hive.jar:?]
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:411) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:151) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2183) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1839) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1526) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) ~[hive-cli-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) ~[hive-cli-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) ~[hive-cli-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) ~[hive-cli-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) ~[hive-cli-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) ~[hive-cli-2.3.3.jar:2.3.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_65]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_65]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_65]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_65]
at org.apache.hadoop.util.RunJar.run(RunJar.java:234) ~[hadoop-common-2.8.4.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:148) ~[hadoop-common-2.8.4.jar:?]
2018-09-05T15:29:53,701 ERROR [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. DEFAULT
2018-09-05T15:29:53,701 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] ql.Driver: Completed executing command(queryId=hadoop_20180905152952_4e1d12b8-06cb-4a91-9fdf-7ca3fafb0051); Time taken: 0.61 seconds
2018-09-05T15:29:53,706 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] conf.HiveConf: Using the default value passed in for log id: 8edc6120-e6f3-477f-bf26-52e0e2fa36db
2018-09-05T15:29:53,706 INFO [8edc6120-e6f3-477f-bf26-52e0e2fa36db main] session.SessionState: Resetting thread name to main