Mapred/Hive is stuck at 33%

Mapred/Hive is stuck at 33%
0.0 0


I’m unable to run the Hive query and it stuck at Mapreduce phase stage -2 Mapper 100% and Reducer 33%.
Workaround .
Earlier,I was also getting a GC overlimit Exhaust , Out of memory error also . I have set below parameter to hive query and still stuck at Mapper and reduce 33 %
set"-XX:NewRatio=12 -Xmx4096m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseGCOverheadLimit -XX:+UseConcMarkSweepGC"

Reducer is set to 1 by default.
Jobtracker Logs snip:
INFO org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 100 segments left of total size: 141733920768 bytes . Stuck at this log point

Note: Data size may be huge in insert query.

Do I need to set sort.mb parameter in Hive or Set the number of reducer in hive or increase reducer size from 256 to 1G or Any alternative fix for this please . Kindly provide the exact numbers to set

Trying for more than two months to get the solution .
DN Ram : 90G *40


What is the approximate/average record size of the intermediate map output and the io.sort.mb in this case?


We are not sure. This is specific to your application.

You might give a try by increasing number of reducers understanding the data.