Scala Spark App submitted to yarn-cluster and unregistered with SUCCEEDED without doing anything -
goal run our scala spark app jar on yarn-cluster mode. works standalone cluster mode , yarn-client, reason not run completion yarn-cluster mode. details the last portion of code seems execute on assigning initial value dataframe when reading input file. looks not after that. none of logs abnormal , there no warns or errors either. gets unregistered status succeeded , gets killed. on other deployment mode (eg. yarn-client, standalone cluster mode) runs smoothly completion. 15/07/22 15:57:00 info yarn.applicationmaster: unregistering applicationmaster succeeded i have ran job on spark 1.3.x , 1.4.x on vanilla spark/yarn cluster , cdh 5.4.3 cluster well. same results. possibly issue? job run command below , input file accessible through hdfs. bin/spark-submit --master yarn-cluster --class assocapp ../associationrulesscala/target/scala-2.10/associationrule_2.10.4-1.0.0.snapshot.jar hdfs://sparkmaster-hk:9000/user/root/breastcancer.csv code snippets this code in area d...