Spark jobs freezing the Yarn queue


#1

you got 2 jobs running by anilkb1105 that are using tons of resources and freezing the cluter

appattempt_1538287994192_3128_000001 running 11 containers with 11 cores and 41984 MB of memory per core

appattempt_1538287994192_3103_000001 running 11 containers with 11 cores and 41984 MB of memory per core

they are both stuck at 34.2 competion, and I believe are freezing the cluster, can someone please take a look?

jobs submitted at Fri Oct 5 08:17:33 -0400 2018 and Fri Oct 5 09:13:59 -0400 2018

Thanks!


#2

now it is totally maxxed out; all cores and all memory is being used.


#3

ok, it seems to be resolved now, thanks!


#4