Can't create a partition in hive

at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:259)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:205)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1827)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:108)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3972)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1130)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:851)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)

Where are you trying this?

Im trying to use this, in hdfs, /user/drestrepohinc/data/raw

Can you share the script you are trying to create hive table?

Alter table avro_orders add partition (year=2018) location ‘/user/drestrepohinc/data/raw/’;

It is not how partition table will be created. Can you tell me your database name? I will give the script.

Database: drestrepohinc__retail_db
Table: avro_orders

There are several issues in your approach

  • Your database is pointing to your home directory. It will not work in our lab as it is multi tenant environment
  • Table is external, it might not work in our lab as it is multi tenant environment.
  • Table is not partitioned
  • Table already have data, in partitioned tables main directory of the table should not have data
  • year column does not exist in the table

Here is the code to create partitioned table in a newly created database:

create database drestrepohinc;
use drestrepohinc;
CREATE TABLE `avro_orders_partitioned`(
  `ingestion_year` int COMMENT '',
  `ingestion_month` int COMMENT '',
  `ingestion_day` int COMMENT '',
  `order_id` int COMMENT '',
  `order_date` bigint COMMENT '',
  `order_customer_id` int COMMENT '',
  `order_status` string COMMENT '')
PARTITIONED BY (year int)
STORED AS avro;

Alter table avro_orders_partitioned add partition (year=2018);

As the user have lab, we are able to quickly troubleshoot the issue and provide the solution


thank you so much, that was very helpfull

What is the purpose of adding "COMMENT ’ ’ " after each of the data types?