Can't create a partition in hive

hive

#1

at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:259)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:205)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3972)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1130)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:851)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)


#2

Where are you trying this?


#3

Im trying to use this, in hdfs, /user/drestrepohinc/data/raw


#4

Can you share the script you are trying to create hive table?


#5

Alter table avro_orders add partition (year=2018) location ‘/user/drestrepohinc/data/raw/’;


#6

It is not how partition table will be created. Can you tell me your database name? I will give the script.


#7

Database: drestrepohinc__retail_db
Table: avro_orders


#8

There are several issues in your approach

  • Your database is pointing to your home directory. It will not work in our lab as it is multi tenant environment
  • Table is external, it might not work in our lab as it is multi tenant environment.
  • Table is not partitioned
  • Table already have data, in partitioned tables main directory of the table should not have data
  • year column does not exist in the table

Here is the code to create partitioned table in a newly created database:

create database drestrepohinc;
use drestrepohinc;
CREATE TABLE `avro_orders_partitioned`(
  `ingestion_year` int COMMENT '',
  `ingestion_month` int COMMENT '',
  `ingestion_day` int COMMENT '',
  `order_id` int COMMENT '',
  `order_date` bigint COMMENT '',
  `order_customer_id` int COMMENT '',
  `order_status` string COMMENT '')
PARTITIONED BY (year int)
STORED AS avro;

Alter table avro_orders_partitioned add partition (year=2018);

As the user have lab, we are able to quickly troubleshoot the issue and provide the solution



#9

thank you so much, that was very helpfull