Sqoop import to Hive table in parquet format


#1

Hi, I have got two doubts
1.I was trying to import orders table from mysql to hive, in parquet format, but it failed.
Please suggest if there is anything wrong/needs to be modified. .

This is my code below,

sqoop import
–connect jdbc:mysql://ms.itversity.com:3306/retail_db
–username retail_user
–password itversity
–table orders
–hive-import
–create-hive-table
–hive-database aparna
–hive-table orders_parquet
–as-parquetfile
-m 1

2.How shall we validate the output, when we import data from mysql to hdfs ,in avro/parquet format in snappy/gzip compression mode.

Thanks in advance,
Aparna


#2

It throws the following error

18/05/15 16:49:10 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.UnknownFormatException: Unknown format for serde:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
org.kitesdk.data.UnknownFormatException: Unknown format for serde:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
at org.kitesdk.data.spi.hive.HiveUtils.descriptorForTable(HiveUtils.java:108)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:274)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.resolveNamespace(HiveAbstractMetadataProvider.java:255)
at org.kitesdk.data.spi.hive.HiveAbstractMetadataProvider.exists(HiveAbstractMetadataProvider.java:159)
at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.exists(FileSystemDatasetRepository.java:257)
at org.kitesdk.data.Datasets.exists(Datasets.java:629)
at org.kitesdk.data.Datasets.exists(Datasets.java:646)
at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:117)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.main(Sqoop.java:243)